Traffic light recognition model training method, device and traffic light recognition method and device

Abstract

This application discloses a traffic light recognition model training method and apparatus, as well as a traffic light recognition method and apparatus. The method includes: acquiring training sample images, specifically including traffic light images and non-traffic light images; using the training sample images, training a feature extraction network for extracting image features containing multi-dimensional information and a first feature recognition network for identifying whether image features are traffic light features in the traffic light recognition model; based on the trained feature extraction network, training a second feature recognition network for recognizing traffic light colors and a third feature recognition network for recognizing traffic light attributes using traffic light images; and finally outputting the trained traffic light recognition model. This application, through an improved network structure, can learn all the information about traffic lights, and by splitting the feature recognition network into three branches for separate training, it can learn the inherent characteristics and relationships of the data, thereby improving the recognition performance of the traffic light recognition model.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a traffic light recognition model training method and device, and a traffic light recognition method and device. Background Art

[0002] In the field of autonomous driving visual perception, the detection and recognition of traffic lights is of great significance to autonomous driving capabilities. Traffic lights, as small target detection objects, usually have only a few to a dozen pixels of information expression in the original image. In addition, it is impossible to correctly identify and provide detailed information such as left turns and U-turns to the autonomous driving decision-making module with the existing detection model alone. A post-processing recognition module is often required to further identify the traffic lights, thereby improving the recognition accuracy of traffic lights.

[0003] There are two main existing post-processing solutions for traffic lights. One is the traditional post-processing solution, which uses the HSV (Hue-Saturation-Value) color space and RGB (red-green-blue) information to count the proportion of the red component, yellow component, and green component of the traffic light, thereby classifying and identifying the traffic light. The other is the post-processing solution for traffic lights based on deep learning, which uses the existing deep learning training recognition model to identify traffic lights.

[0004] The above-mentioned traditional traffic light post-processing recognition scheme is only effective in recognizing the color of traffic lights, but its generalization is poor. For example, due to weather factors, the color cannot be within the specified color threshold, resulting in a certain error in the recognition result, and the specific attribute information of traffic lights such as left turn, right turn, and U-turn cannot be recognized.

[0005] The traffic light post-processing recognition scheme based on deep learning can not only classify the color of traffic lights, but also classify the attributes, such as left turn, right turn, U-turn, etc. However, the existing recognition model ignores the unique intrinsic data characteristics and internal connections of traffic light data, and only has ordinary statistical characteristics. Therefore, the recognition accuracy of the trained recognition model still needs to be improved. Summary of the invention

[0006] The embodiments of the present application provide a traffic light recognition model training method and device, and a traffic light recognition method and device to improve the recognition accuracy of the traffic light recognition model.

[0007] The present application embodiment adopts the following technical solutions:

[0008] In a first aspect, an embodiment of the present application provides a traffic light recognition model training method, wherein the method comprises:

[0009] Acquire training sample images, wherein the training sample images include traffic light images and non-traffic light images;

[0010] Using the training sample images, training a feature extraction network and a first feature recognition network of a traffic light recognition model, wherein the feature extraction network is used to extract image features containing multi-dimensional information, and the first feature recognition network is used to identify whether the image features are traffic light features;

[0011] Based on the trained feature extraction network, the second feature recognition network and the third feature recognition network of the traffic light recognition model are trained using the traffic light image, wherein the second feature recognition network is used to recognize the color of the traffic light, and the third feature recognition network is used to recognize the attribute of the traffic light;

[0012] Based on the trained feature extraction network, the trained first feature recognition network, the trained second feature recognition network and the trained third feature recognition network, a trained traffic light recognition model is output.

[0013] Optionally, the feature extraction network includes a feature extraction basic network and a feature extraction enhancement network, and the training of the feature extraction network and the first feature recognition network of the traffic light recognition model using the training sample image includes:

[0014] Using the feature extraction basic network, performing convolution processing on the training sample image to obtain a first convolution processing result;

[0015] Using the feature extraction and enhancement network, the first convolution processing result is enhanced to obtain a feature enhancement processing result;

[0016] Using the first feature recognition network, performing feature recognition on the feature enhancement processing result to obtain a feature recognition result;

[0017] Determining a loss function value according to the feature recognition result and the sample label of the training sample image;

[0018] Optimize the parameters of the feature extraction network and the parameters of the first feature recognition network according to the loss function value.

[0019] Optionally, the using the feature extraction basic network to perform convolution processing on the training sample image to obtain a first convolution processing result includes:

[0020] Performing mixed dilated convolution processing on the training sample image to obtain a mixed dilated convolution processing result;

[0021] Performing 3x3 convolution processing on the training sample image to obtain a 3x3 convolution processing result;

[0022] Concatenate the mixed dilated convolution processing result and the 3x3 convolution processing result according to the channel dimension to obtain a concatenated convolution processing result;

[0023] The concatenated convolution processing result is used as the first convolution processing result.

[0024] Optionally, the feature extraction and enhancement network includes a first enhancement network branch, a second enhancement network branch and a third enhancement network branch, and the feature extraction and enhancement network is used to enhance the first convolution processing result to obtain a feature enhancement processing result including:

[0025] Performing a 1x1 convolution process on the first convolution process result to obtain a second convolution process result;

[0026] Using the first enhancement network branch, the second convolution processing result is enhanced to obtain a first feature enhancement processing result;

[0027] Using the second enhancement network branch, the second convolution processing result is enhanced to obtain a second feature enhancement processing result;

[0028] The first feature enhancement processing result and the second feature enhancement processing result are fused by using the third enhancement network branch to obtain a third feature enhancement processing result.

[0029] Optionally, the using the first enhancement network branch to enhance the second convolution processing result to obtain a first feature enhancement processing result includes:

[0030] Performing 1x1 convolution processing on the second convolution processing result to obtain a third convolution processing result;

[0031] Processing the third convolution processing result by using a nonlinear activation function to obtain a first processing result of the nonlinear activation function;

[0032] Perform matrix dot multiplication on the first processing result of the nonlinear activation function and the second convolution processing result according to the channel dimension to obtain the first feature enhancement processing result, which includes the enhancement processing result of the traffic light position feature.

[0033] Optionally, the using the second enhancement network branch to enhance the second convolution processing result to obtain a second feature enhancement processing result includes:

[0034] Performing global average pooling processing on the second convolution processing result to obtain a global average pooling processing result;

[0035] Inputting the global average pooling processing result into a fully connected layer to obtain an output result of the fully connected layer;

[0036] Processing the output result of the fully connected layer using a nonlinear activation function to obtain a second processing result of the nonlinear activation function;

[0037] The second processing result of the nonlinear activation function is multiplied by the second convolution processing result to obtain the second feature enhancement processing result, wherein the second feature enhancement processing result includes an enhancement processing result of a channel importance feature.

[0038] Optionally, the using the third enhancement network branch to fuse the first feature enhancement processing result and the second feature enhancement processing result to obtain a third feature enhancement processing result includes:

[0039] Based on the residual network, the first feature enhancement processing result and the second feature enhancement processing result are spliced ​​to obtain a comprehensive feature enhancement processing result;

[0040] Performing a 1x1 convolution process on the comprehensive feature enhancement processing result to obtain a fourth convolution processing result;

[0041] By using the skip connection of the residual network, the fourth convolution processing result and the second convolution processing result are added according to the channel dimension to obtain the added convolution processing result;

[0042] The added convolution processing result is subjected to 3x3 convolution processing to obtain the feature enhancement processing result.

[0043] In a second aspect, an embodiment of the present application further provides a traffic light recognition method, wherein the method comprises:

[0044] Obtain an image to be recognized;

[0045] Using a traffic light recognition model to recognize the image to be recognized, to obtain a traffic light recognition result;

[0046] The traffic light recognition model is trained based on any of the aforementioned traffic light recognition model training methods.

[0047] In a third aspect, an embodiment of the present application further provides a traffic light recognition model training device, wherein the device comprises:

[0048] A first acquisition unit, configured to acquire a training sample image, wherein the training sample image includes a traffic light image and a non-traffic light image;

[0049] A first training unit is used to train a feature extraction network and a first feature recognition network of a traffic light recognition model using the training sample image, wherein the feature extraction network is used to extract image features containing multi-dimensional information, and the first feature recognition network is used to identify whether the image features are traffic light features;

[0050] A second training unit is used to train a second feature recognition network and a third feature recognition network of the traffic light recognition model based on the trained feature extraction network and using the traffic light image, wherein the second feature recognition network is used to recognize the color of the traffic light and the third feature recognition network is used to recognize the attribute of the traffic light;

[0051] An output unit is used to output a trained traffic light recognition model based on the trained feature extraction network, the trained first feature recognition network, the trained second feature recognition network and the trained third feature recognition network.

[0052] In a fourth aspect, an embodiment of the present application further provides a traffic light recognition device, wherein the device comprises:

[0053] A second acquisition unit, used for acquiring an image to be recognized;

[0054] A recognition unit, used to recognize the image to be recognized by using a traffic light recognition model to obtain a traffic light recognition result;

[0055] Wherein, the traffic light recognition model is trained based on the aforementioned traffic light recognition model training device.

[0056] In a fifth aspect, an embodiment of the present application further provides an electronic device, including:

[0057] Processor; and

[0058] A memory arranged to store computer executable instructions, which, when executed, cause the processor to perform any of the aforementioned traffic light recognition model training methods or traffic light recognition methods.

[0059] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes any of the aforementioned traffic light recognition model training methods or traffic light recognition methods.

[0060] At least one of the above technical solutions adopted in the embodiment of the present application can achieve the following beneficial effects: the traffic light recognition model training method of the embodiment of the present application first obtains a training sample image, and the training sample image includes a traffic light image and a non-traffic light image; then, the feature extraction network and the first feature recognition network of the traffic light recognition model are trained using the training sample image, the feature extraction network is used to extract image features containing multi-dimensional information, and the first feature recognition network is used to identify whether the image feature is a traffic light feature; then, based on the trained feature extraction network, the second feature recognition network and the third feature recognition network of the traffic light recognition model are trained using the traffic light image, the second feature recognition network is used to identify the color of the traffic light, and the third feature recognition network is used to identify the attribute of the traffic light; finally, based on the trained feature extraction network, the trained first feature recognition network, the trained second feature recognition network and the trained third feature recognition network, the trained traffic light recognition model is output. The traffic light recognition model training method of the embodiment of the present application can learn all-round traffic light information through the improved feature extraction network structure, and split the feature recognition network into three branch networks based on the traffic light recognition characteristics for separate training, so that the inherent characteristics and inherent connections of the traffic light data can be learned, and the recognition accuracy of the traffic light recognition model is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0062] Figure 1 A flow chart of a traffic light recognition model training method in an embodiment of the present application;

[0063] Figure 2 This is a schematic diagram of the overall network structure of a traffic light recognition model in an embodiment of the present application;

[0064] Figure 3 This is a schematic diagram of the overall structure of a feature extraction enhancement network in an embodiment of the present application;

[0065] Figure 4 A flow chart of a traffic light recognition method in an embodiment of the present application;

[0066] Figure 5 This is a structural schematic diagram of a traffic light recognition model training device in an embodiment of the present application;

[0067] Figure 6 This is a schematic diagram of the structure of a traffic light recognition device in an embodiment of the present application;

[0068] Figure 7A schematic diagram of the structure of an electronic device in an embodiment of the present application

[0069] Figure 8 It is a schematic diagram of the structure of another electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0071] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0072] The present application embodiment provides a traffic light recognition model training method, such as Figure 1 As shown, a flow chart of a traffic light recognition model training method in an embodiment of the present application is provided, and the method at least includes the following steps S110 to S140:

[0073] Step S110: acquiring training sample images, where the training sample images include traffic light images and non-traffic light images.

[0074] When training a traffic light recognition model, the embodiment of the present application needs to first obtain training sample images, specifically including traffic light images, i.e., images containing real traffic light information, and non-traffic light images, i.e., images that do not contain real traffic light information.

[0075] The above training sample images can be pre-labeled according to whether they contain traffic light information and the type of traffic light information contained. For example, a set of labels is set to 1 for traffic light and 0 for non-traffic light to distinguish traffic light images from non-traffic light images. Furthermore, for traffic light images, specific traffic light types can also be labeled, including color classification and attribute classification, etc. For example, a set of color labels is set to 0 for red, 1 for yellow, 2 for green, and 3 for other colors, and a set of attribute labels is set to 0 for left turn, 1 for straight, 2 for right turn, 3 for U-turn, 4 for pedestrian, 5 for bicycle, 6 for countdown board, etc. The above three sets of labeled labels are used as the recognition capabilities of the three supervisory signal training models.

[0076] Step S120, using the training sample image, training the feature extraction network and the first feature recognition network of the traffic light recognition model, the feature extraction network is used to extract image features containing multi-dimensional information, and the first feature recognition network is used to identify whether the image features are traffic light features.

[0077] Considering that the traffic light detection model may detect false positive traffic signs, such as red lanterns and red cross knots hanging on the roadside, these targets are similar to traffic lights when observed from a distance and can easily confuse the recognition model. Therefore, when training the traffic light recognition model, the embodiment of the present application first trains the feature extraction network and the first feature recognition network.

[0078] The feature extraction network can be regarded as a backbone network, which is used to extract image features containing multi-dimensional information in the training sample images. The first feature recognition network Head_is is mainly used to detect whether the input training sample images contain traffic light information. When training these two networks, the input data includes traffic light images and non-traffic light images. The supervision signal is 1 for traffic light images and 0 for non-traffic light images. When Head_is reaches the specified number of iterations and the accuracy reaches the specified requirements, the training tasks of the two networks are completed. At this time, it can be considered that both backbone and Head_is have the ability to recognize traffic lights.

[0079] Step S130, based on the trained feature extraction network, the second feature recognition network and the third feature recognition network of the traffic light recognition model are trained using the traffic light image, wherein the second feature recognition network is used to recognize the color of the traffic light, and the third feature recognition network is used to recognize the attributes of the traffic light.

[0080] After the training of backbone and Head_is networks is completed, the network parameters of backbone and Head_is can be frozen, and the parameters of the second feature recognition network Head_color and the third feature recognition network Head_content can be trained. The second feature recognition network Head_color is mainly used to recognize the color of traffic lights in traffic light images, and the third feature recognition network Head_content is mainly used to recognize the attributes of traffic lights in traffic light images.

[0081] When training Head_color and Head_content, the input data is a traffic light image. The supervision signal consists of two parts. The supervision signal of Head_color is 0 for red, 1 for yellow, 2 for green, and 3 for other colors. The supervision signal of Head_content is 0 for left turn, 1 for straight, 2 for right turn, 3 for U-turn, 4 for pedestrian, 5 for bicycle, and 6 for countdown sign. When Head_color and Head_content reach the specified number of iterations and the accuracy reaches the specified requirements, the training tasks of the two networks are completed. At this time, it can be considered that Head_color has the ability to recognize the color of traffic lights, and Head_content has the ability to recognize the attributes of traffic lights.

[0082] The embodiment of the present application splits the feature recognition network into three branches: Head_is, Head_color, and Head_content. The role of Head_is is to directly filter out traffic light images and non-traffic light images. For non-traffic light images, there is no need to pay attention to the output results of Head_color and Head_content. The design of Head_color and Head_content is to learn the internal connection of traffic light features, such as the common feature of green light straight and green light left turn is green, and the common feature of red light straight and green light straight is straight. When split into two branches, the model can learn the correlation between data without information confusion. Compared with the method of using only one feature recognition network to directly classify and obtain recognition results of combined attribute granularity, such as red light straight, the learning samples of the Head_color branch and Head_content branch of the embodiment of the present application are increased in disguise, thereby enhancing the learning intensity.

[0083] Step S140, outputting a trained traffic light recognition model based on the trained feature extraction network, the trained first feature recognition network, the trained second feature recognition network, and the trained third feature recognition network.

[0084] After completing the training of the above-mentioned feature extraction network, the first feature recognition network, the second feature recognition network and the third feature recognition network, the training process of the traffic light recognition model of the embodiment of the present application is completed, and finally the trained traffic light recognition model is output.

[0085] The traffic light recognition model training method of the embodiment of the present application can learn comprehensive traffic light information through an improved feature extraction network structure, and split the feature recognition network into three branch networks based on the traffic light recognition characteristics and train them separately, so as to learn the intrinsic characteristics and internal connections of the traffic light data and improve the recognition accuracy of the traffic light recognition model.

[0086] In one embodiment of the present application, the feature extraction network includes a feature extraction basic network and a feature extraction enhancement network, and the use of the training sample image to train the feature extraction network and the first feature recognition network of the traffic light recognition model includes: using the feature extraction basic network to perform convolution processing on the training sample image to obtain a first convolution processing result; using the feature extraction enhancement network to perform enhancement processing on the first convolution processing result to obtain a feature enhancement processing result; using the first feature recognition network to perform feature recognition on the feature enhancement processing result to obtain a feature recognition result; determining a loss function value based on the feature recognition result and the sample label of the training sample image; and optimizing the parameters of the feature extraction network and the parameters of the first feature recognition network based on the loss function value.

[0087] The feature extraction network backbone of the embodiment of the present application specifically includes two parts: a feature extraction basic network and a feature extraction enhancement network. The feature extraction basic network can be regarded as the front end of the backbone, which is used to extract the basic feature information of the training sample image. The feature extraction enhancement network can be regarded as the back end of the backbone, which is used to perform reinforcement learning on the intrinsic characteristics and intrinsic connections of the features extracted by the feature extraction basic network, so as to further improve the cognitive ability of the full information of the traffic light.

[0088] When training the feature extraction network backbone and the first feature recognition network Head_is, the above-mentioned feature extraction basic network is first used to perform convolution processing on the training sample image to obtain the first convolution processing result, and then the above-mentioned feature extraction enhancement network is used to perform feature enhancement processing on the first convolution processing result to strengthen the learning of the intrinsic characteristics and intrinsic connections of the features to obtain the feature enhancement processing result, and then the first feature recognition network Head_is is used to perform feature recognition on the feature enhancement processing result to obtain the feature recognition result, specifically including the recognition result of whether the training sample image contains traffic light information. Finally, the above feature recognition result is compared with the pre-annotated label in the training sample image (1 is a traffic light image, 0 is not a traffic light image), so as to determine the network loss function value, and then update the network parameters by back propagation based on the loss function value until the network converges.

[0089] In one embodiment of the present application, the use of the feature extraction basic network to perform convolution processing on the training sample image to obtain a first convolution processing result includes: performing mixed dilated convolution processing on the training sample image to obtain a mixed dilated convolution processing result; performing 3x3 convolution processing on the training sample image to obtain a 3x3 convolution processing result; splicing the mixed dilated convolution processing result with the 3x3 convolution processing result according to the channel dimension to obtain a spliced ​​convolution processing result; and using the spliced ​​convolution processing result as the first convolution processing result.

[0090] When the embodiment of the present application uses the feature extraction basic network to perform convolution processing on the training sample image, two convolution methods, Hybrid Dilated Convolution (HDC) and 3x3 convolution (conv3x3), are mainly used. The two convolution methods perform the first step of downsampling feature extraction with a step size of 2. The role of HDC is to expand the field of view through the expansion factor, thereby improving the richness of information, and the role of 3x3 convolution is to increase the theoretical receptive field. For example, for a training image with an input size of (b, 64, 64, 3), two image features of (b, 32, 32, 16) can be obtained after HDC and 3x3 convolution processing respectively.

[0091] After HDC and 3x3 convolution processing, the image features (b, 32, 32, 16) extracted by the hybrid dilated convolution processing and the image features (b, 32, 32, 16) extracted by the 3x3 convolution processing can be concatenated according to the channel dimension to obtain the concatenated image features (b, 32, 32, 32), which further increases the number of feature channels and thus improves the richness of the image information.

[0092] In order to facilitate the understanding of the above embodiments, Figure 2 As shown, a schematic diagram of the overall network structure of a traffic light recognition model in an embodiment of the present application is provided. The traffic light recognition model in the embodiment of the present application mainly includes two parts: a feature extraction network backbone and a feature recognition network. The feature extraction network backbone part mainly includes two branches: a feature extraction basic network and a feature extraction enhancement network signal_block. The feature extraction basic network is the front end of the backbone. It mainly performs the first step of downsampling feature extraction through HDC mixed hole convolution and 3x3 convolution with a step size of 2, and then concatenates the extracted features (b, 32, 32, 16) according to the channel dimension to obtain the spliced ​​image features (b, 32, 32, 32) and send them to signal_block.

[0093] In order to improve the training effect of the model, the embodiment of the present application adopts three signal_blocks connected in sequence, and the three signal_blocks have the same network structure but different parameters. After the image feature (b, 32, 32, 32) is processed by the first signal_block, the image feature (b, 16, 16, 64) can be obtained, and then after the second signal_block is processed, the image feature (b, 8, 8, 128) can be obtained. After the third signal_block is processed, the image feature (b, 4, 4, 512) can be obtained. That is, each time the signal_block is processed, the image resolution will be reduced exponentially and the number of channels of the image feature will be increased exponentially, thereby achieving a better feature reinforcement learning effect. Of course, the specific number of signal_block processes can be flexibly adjusted by those skilled in the art according to the model training effect, and no specific limitation is made here.

[0094] After signal_block processing, we can enter the feature recognition stage. The feature recognition network specifically includes three branches: Head_is, Head_color and Head_content. The network structures of the three branches are the same. For example, the network structure of conv1x1 (1x1 convolution) + GAP (Global Average Pool) + conv1x1 + H-swish (non-linear activation function) + FC (Fully Connected layers) can be used. However, the network parameters of the three branches are different, thereby achieving different feature recognition capabilities.

[0095] Specifically, by inputting the feature enhancement processing result into the Head_is branch, we can obtain the recognition result of whether the training sample image is a traffic light image (b, 2); by inputting the feature enhancement processing result into the Head_color branch, we can obtain the recognition result of the traffic light color in the traffic light image (b, 4); by inputting the feature enhancement processing result into the Head_content branch, we can obtain the recognition result of the traffic light attribute in the traffic light image (b, 7).

[0096] In one embodiment of the present application, the feature extraction and enhancement network includes a first enhancement network branch, a second enhancement network branch and a third enhancement network branch, and the use of the feature extraction and enhancement network to enhance the first convolution processing result to obtain a feature enhancement processing result includes: performing 1x1 convolution processing on the first convolution processing result to obtain a second convolution processing result; using the first enhancement network branch to enhance the second convolution processing result to obtain a first feature enhancement processing result; using the second enhancement network branch to enhance the second convolution processing result to obtain a second feature enhancement processing result; using the third enhancement network branch to fuse the first feature enhancement processing result and the second feature enhancement processing result to obtain a third feature enhancement processing result.

[0097] The feature extraction enhancement network of the embodiment of the present application specifically includes a first enhancement network branch, a second enhancement network branch and a third enhancement network branch, and the three branches focus on learning different feature recognition capabilities. Specifically, when performing feature enhancement processing, the first convolution processing result obtained in the above embodiment, such as (b, h, w, c), is first subjected to a 1x1 convolution processing (conv1x1) to obtain a second convolution processing result, which is used as the basis for the subsequent three branch enhancement processing.

[0098] First, for the first reinforcement network branch, its main function is to strengthen the learning of the relative position information of the traffic lights in the image. Therefore, by using the first reinforcement network branch to perform reinforcement learning of the relative position information of the traffic lights on the second convolution processing result, the first feature reinforcement processing result can be obtained. Secondly, for the second reinforcement network branch, its main function is to identify important features and strengthen the learning of important features. Therefore, by using the second reinforcement network branch to perform reinforcement learning of important features on the second convolution processing result, the second feature reinforcement processing result can be obtained. Finally, for the third reinforcement network branch, its main function is to integrate the reinforcement processing results of the first reinforcement network branch and the second reinforcement network branch, improve information flow, and avoid gradient disappearance and degradation problems.

[0099] In one embodiment of the present application, the use of the first enhancement network branch to enhance the second convolution processing result to obtain a first feature enhancement processing result includes: performing 1x1 convolution processing on the second convolution processing result to obtain a third convolution processing result; processing the third convolution processing result using a nonlinear activation function to obtain a first processing result of the nonlinear activation function; performing matrix dot multiplication of the first processing result of the nonlinear activation function and the second convolution processing result according to the channel dimension to obtain the first feature enhancement processing result, and the first feature enhancement processing result includes an enhancement processing result of a traffic light position feature.

[0100] like Figure 3 As shown, a schematic diagram of the overall structure of a feature extraction enhancement network in an embodiment of the present application is provided. Figure 3 The upper left half of the figure represents the first enhanced network branch of the embodiment of the present application, which mainly includes the processing of conv1x1+H-swish.

[0101] Considering that the characteristic information of traffic lights is strongly correlated with the relative position information in addition to the chromaticity information, for example, a three-light traffic light has a red light on the top, a yellow light in the middle, and a green light on the bottom, and their shared background light board is generally black, the design of the first reinforcement network branch is mainly to strengthen the learning of the relative position information of traffic lights.

[0102] Specifically, a 1x1 convolution is first performed on the second convolution processing result obtained in the above embodiment to obtain a third convolution processing result (b, h, w, 1), and then the third convolution processing result (b, h, w, 1) is processed using the nonlinear activation function H-swish to obtain the first processing result of the nonlinear activation function. The H-swish here can be flexibly replaced with other types of nonlinear activation functions according to actual training requirements, and is not specifically limited here. Finally, the output result of the above conv1x1+H-swish is matrix-multiplied with the second convolution processing result according to each feature point of each channel to improve the channel information interaction capability, thereby enhancing the learning and cognitive ability of position information.

[0103] In one embodiment of the present application, the second convolution processing result is enhanced by the second enhancement network branch to obtain a second feature enhancement processing result, including: performing global average pooling processing on the second convolution processing result to obtain a global average pooling processing result; inputting the global average pooling processing result into a fully connected layer to obtain an output result of the fully connected layer; processing the output result of the fully connected layer by a nonlinear activation function to obtain a second processing result of the nonlinear activation function; multiplying the second processing result of the nonlinear activation function by the second convolution processing result to obtain the second feature enhancement processing result, wherein the second feature enhancement processing result includes an enhancement processing result of a channel importance feature.

[0104] like Figure 3 As shown, Figure 3 The lower left half of represents the second enhanced network branch of the embodiment of the present application, which mainly includes the processing of GAP+FC+H-swish, thereby obtaining the second feature enhancement processing result (b, 1, 1, c).

[0105] Taking into account the different learning capabilities of the convolution kernels of each feature channel, the obtained feature importances are also different. Therefore, the embodiment of the present application can learn the importance coefficient of each feature channel through the processing of GAP+FC+H-swish, and then multiply the importance coefficient of each feature channel with the result of the second convolution processing according to the dimension of each feature channel, that is, each feature is multiplied by a corresponding importance coefficient, thereby improving the channel information interaction ability, thereby enhancing the learning and cognitive ability of important features.

[0106] In one embodiment of the present application, the third enhancement network branch is used to fuse the first feature enhancement processing result and the second feature enhancement processing result to obtain the third feature enhancement processing result, including: based on the residual network, the first feature enhancement processing result and the second feature enhancement processing result are spliced ​​to obtain a comprehensive feature enhancement processing result; the comprehensive feature enhancement processing result is subjected to 1x1 convolution processing to obtain a fourth convolution processing result; the fourth convolution processing result and the second convolution processing result are added according to the channel dimension through the jump connection of the residual network to obtain the added convolution processing result; the added convolution processing result is subjected to 3x3 convolution processing to obtain the feature enhancement processing result.

[0107] like Figure 3 As shown, Figure 3 The right half of represents the third enhanced network branch of the embodiment of the present application. The third enhanced network branch uses a residual method to concatenate the output results of the first enhanced network branch and the second enhanced network branch to obtain image features (b, h, w, 2c), and then perform a 1x1 convolution (conv1x1), and add the output features (b, h, w, c) and the previous second convolution processing results according to the dimensions of each feature channel. Finally, the added convolution processing results (b, h, w, c) are subjected to a 3x3 convolution (conv3x3) to obtain image features (b, h / 2, w / 2, 2c), thereby further reducing the image resolution and increasing the number of feature channels. The above process uses a residual jump connection to allow the feature information of the previous module to flow into the next module without hindrance, improve information flow, and avoid gradient disappearance and degradation problems.

[0108] In one embodiment of the present application, the traffic light image includes a left turn light image and a right turn light image, and the right turn light image is obtained in the following manner: acquiring the left turn light image; horizontally mirroring the left turn light image to obtain the right turn light image; and modifying the attribute label corresponding to the left turn light image to the attribute label corresponding to the right turn light image.

[0109] In actual scenarios, since the right turn light data in the traffic light data is usually extremely scarce, the embodiment of the present application first trains the data containing the left turn light in the current batch of traffic light data during the training process of Head_color and Head_content, and then dynamically copies the batch of data for a second time, and horizontally mirrors the left turn light image. The supervisory signal Head_color remains unchanged, and Head_content is adjusted from 0 representing a left turn to 2 representing a right turn, thereby enhancing the training of the right turn light data.

[0110] The present application also provides a method for identifying a traffic light. Figure 4 As shown, a flow chart of a traffic light recognition method in an embodiment of the present application is provided, and the method at least includes the following steps S410 to S420:

[0111] Step S410, obtaining an image to be recognized;

[0112] Step S420, using a traffic light recognition model to recognize the image to be recognized, to obtain a traffic light recognition result;

[0113] The traffic light recognition model is trained based on any of the aforementioned traffic light recognition model training methods.

[0114] The application process of the traffic light recognition model is basically the same as the above training process. The main difference is that there is no supervision signal in the model application stage. By inputting the image to be recognized into the traffic light recognition model, the traffic light recognition result can be directly output, such as whether the image contains a traffic light and the specific color and attributes of the traffic light.

[0115] The present application also provides a traffic light recognition model training device 500. Figure 5 As shown, a schematic diagram of the structure of a traffic light recognition model training device in an embodiment of the present application is provided, and the device 500 includes:

[0116] A first acquisition unit 510 is used to acquire a training sample image, where the training sample image includes a traffic light image and a non-traffic light image;

[0117] A first training unit 520, used to train a feature extraction network and a first feature recognition network of a traffic light recognition model using the training sample image, wherein the feature extraction network is used to extract image features containing multi-dimensional information, and the first feature recognition network is used to identify whether the image features are traffic light features;

[0118] A second training unit 530 is used to train a second feature recognition network and a third feature recognition network of the traffic light recognition model based on the trained feature extraction network and using the traffic light image, wherein the second feature recognition network is used to recognize the color of the traffic light and the third feature recognition network is used to recognize the attribute of the traffic light;

[0119] The output unit 540 is used to output the trained traffic light recognition model based on the trained feature extraction network, the trained first feature recognition network, the trained second feature recognition network and the trained third feature recognition network.

[0120] In one embodiment of the present application, the feature extraction network includes a feature extraction basic network and a feature extraction enhancement network, and the first training unit 520 is specifically used to: use the feature extraction basic network to perform convolution processing on the training sample image to obtain a first convolution processing result; use the feature extraction enhancement network to perform enhancement processing on the first convolution processing result to obtain a feature enhancement processing result; use the first feature recognition network to perform feature recognition on the feature enhancement processing result to obtain a feature recognition result; determine the loss function value based on the feature recognition result and the sample label of the training sample image; and optimize the parameters of the feature extraction network and the parameters of the first feature recognition network based on the loss function value.

[0121] In one embodiment of the present application, the first training unit 520 is specifically used to: perform mixed dilated convolution processing on the training sample image to obtain a mixed dilated convolution processing result; perform 3x3 convolution processing on the training sample image to obtain a 3x3 convolution processing result; splice the mixed dilated convolution processing result with the 3x3 convolution processing result according to the channel dimension to obtain a spliced ​​convolution processing result; and use the spliced ​​convolution processing result as the first convolution processing result.

[0122] In one embodiment of the present application, the feature extraction enhancement network includes a first enhancement network branch, a second enhancement network branch and a third enhancement network branch, and the first training unit 520 is specifically used to: perform 1x1 convolution processing on the first convolution processing result to obtain a second convolution processing result; use the first enhancement network branch to enhance the second convolution processing result to obtain a first feature enhancement processing result; use the second enhancement network branch to enhance the second convolution processing result to obtain a second feature enhancement processing result; use the third enhancement network branch to fuse the first feature enhancement processing result and the second feature enhancement processing result to obtain a third feature enhancement processing result.

[0123] In one embodiment of the present application, the first training unit 520 is specifically used to: perform 1x1 convolution processing on the second convolution processing result to obtain a third convolution processing result; process the third convolution processing result using a nonlinear activation function to obtain a first processing result of the nonlinear activation function; perform matrix dot multiplication of the first processing result of the nonlinear activation function and the second convolution processing result according to the channel dimension to obtain the first feature enhancement processing result, and the first feature enhancement processing result includes an enhancement processing result of a traffic light position feature.

[0124] In one embodiment of the present application, the first training unit 520 is specifically used to: perform global average pooling processing on the second convolution processing result to obtain a global average pooling processing result; input the global average pooling processing result into a fully connected layer to obtain an output result of the fully connected layer; process the output result of the fully connected layer using a nonlinear activation function to obtain a second processing result of the nonlinear activation function; multiply the second processing result of the nonlinear activation function by the second convolution processing result to obtain the second feature enhancement processing result, wherein the second feature enhancement processing result includes an enhancement processing result of a channel importance feature.

[0125] In one embodiment of the present application, the first training unit 520 is specifically used to: based on the residual network, splice the first feature enhancement processing result and the second feature enhancement processing result to obtain a comprehensive feature enhancement processing result; perform 1x1 convolution processing on the comprehensive feature enhancement processing result to obtain a fourth convolution processing result; through the jump connection of the residual network, add the fourth convolution processing result and the second convolution processing result according to the channel dimension to obtain the added convolution processing result; perform 3x3 convolution processing on the added convolution processing result to obtain the feature enhancement processing result.

[0126] It can be understood that the above-mentioned traffic light recognition model training device can implement the various steps of the traffic light recognition model training method provided in the aforementioned embodiment. The relevant explanations about the traffic light recognition model training method are applicable to the traffic light recognition model training device and will not be repeated here.

[0127] The present application also provides a traffic light recognition device 600. Figure 6 As shown, a schematic diagram of the structure of a traffic light recognition device in an embodiment of the present application is provided, and the device 600 includes:

[0128] The second acquisition unit 610 is used to acquire an image to be recognized;

[0129] The recognition unit 620 is used to recognize the image to be recognized by using a traffic light recognition model to obtain a traffic light recognition result;

[0130] Wherein, the traffic light recognition model is trained based on the aforementioned traffic light recognition model training device.

[0131] It can be understood that the above-mentioned traffic light recognition device can implement each step of the traffic light recognition method provided in the above-mentioned embodiment, and the relevant explanations about the traffic light recognition method are applicable to the traffic light recognition device and will not be repeated here.

[0132] Figure 7 and Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 7 and Figure 8 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0133] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 and Figure 8 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0134] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0135] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a traffic light recognition model training device or a traffic light recognition device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0136] Acquire training sample images, wherein the training sample images include traffic light images and non-traffic light images;

[0137] Using the training sample images, training a feature extraction network and a first feature recognition network of a traffic light recognition model, wherein the feature extraction network is used to extract image features containing multi-dimensional information, and the first feature recognition network is used to identify whether the image features are traffic light features;

[0138] Based on the trained feature extraction network, the second feature recognition network and the third feature recognition network of the traffic light recognition model are trained using the traffic light image, wherein the second feature recognition network is used to recognize the color of the traffic light, and the third feature recognition network is used to recognize the attribute of the traffic light;

[0139] Based on the trained feature extraction network, the trained first feature recognition network, the trained second feature recognition network and the trained third feature recognition network, a trained traffic light recognition model is output.

[0140] Or, to do the following:

[0141] Obtain an image to be recognized;

[0142] Using a traffic light recognition model to recognize the image to be recognized, to obtain a traffic light recognition result;

[0143] The traffic light recognition model is trained based on any of the aforementioned traffic light recognition model training methods.

[0144] The above application Figure 1 The traffic light recognition model training device disclosed in the embodiment shown or the present application Figure 4The method performed by the traffic light recognition device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0145] The electronic device may also perform Figure 1 Traffic light recognition model training device or execution Figure 4 The method disclosed in the embodiment shown is executed by the traffic light recognition device, and the traffic light recognition model training device is implemented in Figure 1 The function of the embodiment shown or the traffic light recognition device is Figure 4 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0146] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method performed by the traffic light recognition model training device in the illustrated embodiment is specifically used to perform:

[0147] Acquire training sample images, wherein the training sample images include traffic light images and non-traffic light images;

[0148] Using the training sample images, training a feature extraction network and a first feature recognition network of a traffic light recognition model, wherein the feature extraction network is used to extract image features containing multi-dimensional information, and the first feature recognition network is used to identify whether the image features are traffic light features;

[0149] Based on the trained feature extraction network, the second feature recognition network and the third feature recognition network of the traffic light recognition model are trained using the traffic light image, wherein the second feature recognition network is used to recognize the color of the traffic light, and the third feature recognition network is used to recognize the attribute of the traffic light;

[0150] Based on the trained feature extraction network, the trained first feature recognition network, the trained second feature recognition network and the trained third feature recognition network, a trained traffic light recognition model is output.

[0151] Alternatively, the electronic device is caused to execute Figure 4 The method performed by the traffic light recognition device in the illustrated embodiment is specifically used to perform:

[0152] Obtain an image to be recognized;

[0153] Using a traffic light recognition model to recognize the image to be recognized, to obtain a traffic light recognition result;

[0154] The traffic light recognition model is trained based on any of the aforementioned traffic light recognition model training methods.

[0155] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0157] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0158] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0159] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0160] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0161] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0162] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0163] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0164] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A traffic light recognition model training method, wherein: The method comprises: Acquire training sample images, wherein the training sample images include traffic light images and non-traffic light images; Using the training sample images, training a feature extraction network and a first feature recognition network of a traffic light recognition model, wherein the feature extraction network is used to extract image features containing multi-dimensional information, and the first feature recognition network is used to identify whether the image features are traffic light features; Based on the trained feature extraction network, the second feature recognition network and the third feature recognition network of the traffic light recognition model are trained using the traffic light image, wherein the second feature recognition network is used to recognize the color of the traffic light, and the third feature recognition network is used to recognize the attribute of the traffic light; Outputting a trained traffic light recognition model based on the trained feature extraction network, the trained first feature recognition network, the trained second feature recognition network, and the trained third feature recognition network; The feature extraction network includes a feature extraction basic network and a feature extraction enhancement network, and the training of the feature extraction network and the first feature recognition network of the traffic light recognition model using the training sample image includes: Using the feature extraction basic network, performing convolution processing on the training sample image to obtain a first convolution processing result; Using the feature extraction and enhancement network, the first convolution processing result is enhanced to obtain a feature enhancement processing result; Using the first feature recognition network, performing feature recognition on the feature enhancement processing result to obtain a feature recognition result; Determining a loss function value according to the feature recognition result and the sample label of the training sample image; Optimizing the parameters of the feature extraction network and the parameters of the first feature recognition network according to the loss function value; The feature extraction enhancement network includes a first enhancement network branch, a second enhancement network branch and a third enhancement network branch. The first enhancement network branch is used to enhance the learning of the relative position of traffic lights, the second enhancement network branch is used to identify important features and enhance the learning of important features, and the third enhancement network branch is used to fuse the enhancement processing results of the first enhancement network branch and the second enhancement network branch.

2. The method of claim 1, wherein: The step of performing convolution processing on the training sample image using the feature extraction basic network to obtain a first convolution processing result includes: Performing mixed dilated convolution processing on the training sample image to obtain a mixed dilated convolution processing result; Performing 3x3 convolution processing on the training sample image to obtain a 3x3 convolution processing result; Concatenate the mixed dilated convolution processing result and the 3x3 convolution processing result according to the channel dimension to obtain a concatenated convolution processing result; The concatenated convolution processing result is used as the first convolution processing result.

3. The method of claim 1, wherein: The step of using the feature extraction enhancement network to enhance the first convolution processing result to obtain a feature enhancement processing result includes: Performing a 1x1 convolution process on the first convolution process result to obtain a second convolution process result; Using the first enhancement network branch, the second convolution processing result is enhanced to obtain a first feature enhancement processing result; Using the second enhancement network branch, the second convolution processing result is enhanced to obtain a second feature enhancement processing result; The first feature enhancement processing result and the second feature enhancement processing result are fused by using the third enhancement network branch to obtain a third feature enhancement processing result.

4. The method of claim 3, wherein: The step of performing enhancement processing on the second convolution processing result by using the first enhancement network branch to obtain a first feature enhancement processing result includes: Performing 1x1 convolution processing on the second convolution processing result to obtain a third convolution processing result; Processing the third convolution processing result by using a nonlinear activation function to obtain a first processing result of the nonlinear activation function; Perform matrix dot multiplication on the first processing result of the nonlinear activation function and the second convolution processing result according to the channel dimension to obtain the first feature enhancement processing result, which includes the enhancement processing result of the traffic light position feature.

5. The method of claim 3, wherein: The step of performing enhancement processing on the second convolution processing result by using the second enhancement network branch to obtain a second feature enhancement processing result includes: Performing global average pooling processing on the second convolution processing result to obtain a global average pooling processing result; Inputting the global average pooling processing result into a fully connected layer to obtain an output result of the fully connected layer; Processing the output result of the fully connected layer using a nonlinear activation function to obtain a second processing result of the nonlinear activation function; The second processing result of the nonlinear activation function is multiplied by the second convolution processing result to obtain the second feature enhancement processing result, wherein the second feature enhancement processing result includes an enhancement processing result of a channel importance feature.

6. The method of claim 3, wherein: The step of fusing the first feature enhancement processing result and the second feature enhancement processing result using the third enhancement network branch to obtain a third feature enhancement processing result includes: Based on the residual network, the first feature enhancement processing result and the second feature enhancement processing result are spliced ​​to obtain a comprehensive feature enhancement processing result; Performing a 1x1 convolution process on the comprehensive feature enhancement processing result to obtain a fourth convolution processing result; By using the skip connection of the residual network, the fourth convolution processing result and the second convolution processing result are added according to the channel dimension to obtain the added convolution processing result; The added convolution processing result is subjected to 3x3 convolution processing to obtain the third feature enhancement processing result.

7. A method for identifying a traffic light, wherein: The method comprises: Obtain an image to be recognized; Using a traffic light recognition model to recognize the image to be recognized, to obtain a traffic light recognition result; Wherein, the traffic light recognition model is trained based on the traffic light recognition model training method according to any one of claims 1-6.

8. A traffic light recognition model training device, wherein: The device comprises: A first acquisition unit, configured to acquire a training sample image, wherein the training sample image includes a traffic light image and a non-traffic light image; A first training unit is used to train a feature extraction network and a first feature recognition network of a traffic light recognition model using the training sample image, wherein the feature extraction network is used to extract image features containing multi-dimensional information, and the first feature recognition network is used to identify whether the image features are traffic light features; A second training unit is used to train a second feature recognition network and a third feature recognition network of the traffic light recognition model based on the trained feature extraction network and using the traffic light image, wherein the second feature recognition network is used to recognize the color of the traffic light and the third feature recognition network is used to recognize the attribute of the traffic light; An output unit, used for outputting a trained traffic light recognition model based on the trained feature extraction network, the trained first feature recognition network, the trained second feature recognition network, and the trained third feature recognition network; The feature extraction network includes a feature extraction basic network and a feature extraction enhancement network, and the first training unit is specifically used for: Using the feature extraction basic network, performing convolution processing on the training sample image to obtain a first convolution processing result; Using the feature extraction and enhancement network, the first convolution processing result is enhanced to obtain a feature enhancement processing result; Using the first feature recognition network, performing feature recognition on the feature enhancement processing result to obtain a feature recognition result; Determining a loss function value according to the feature recognition result and the sample label of the training sample image; Optimizing the parameters of the feature extraction network and the parameters of the first feature recognition network according to the loss function value; The feature extraction enhancement network includes a first enhancement network branch, a second enhancement network branch and a third enhancement network branch. The first enhancement network branch is used to enhance the learning of the relative position of traffic lights, the second enhancement network branch is used to identify important features and enhance the learning of important features, and the third enhancement network branch is used to fuse the enhancement processing results of the first enhancement network branch and the second enhancement network branch.

9. A traffic light recognition device, wherein: The device comprises: A second acquisition unit, used for acquiring an image to be recognized; A recognition unit, used to recognize the image to be recognized by using a traffic light recognition model to obtain a traffic light recognition result; Wherein, the traffic light recognition model is trained based on the traffic light recognition model training device according to claim 8.

Images