Traffic light detection method, electronic device, and storage medium

Abstract

Embodiments of the present application disclose a traffic light detection method, an electronic device and a storage medium. The method comprises: receiving a first image to be detected; encoding position information of pixel points in the first image to obtain a second image containing position encoding information; inputting the second image into a traffic light detection model for processing to obtain a detection result output by the traffic light detection model, wherein the detection result comprises: whether a traffic light exists in the first image and position information of the traffic light.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to a traffic light detection method, electronic device, and storage medium. Background Technology

[0002] Traffic light detection is a crucial task in the fields of assisted driving and autonomous driving, playing a vital role in ensuring vehicle safety. Current general-purpose object detection algorithms have achieved high metrics and accuracy on datasets like COCO. However, traffic light detection differs significantly from general object detection. Due to the inherent characteristics of its task, traffic light detection has a lower tolerance for false detections. Furthermore, traffic lights typically need to be detected at a distance, resulting in generally small targets, increasing detection difficulty and the false detection rate. Therefore, effectively reducing the false detection rate and improving detection performance is a critical challenge that must be addressed in traffic light detection. Summary of the Invention

[0003] This application provides a traffic light detection method, electronic device, and storage medium to solve the technical problem of high false detection rate of traffic lights in the prior art.

[0004] According to a first aspect of this application, a traffic light detection method is disclosed, the method comprising:

[0005] Receive the first image to be detected;

[0006] The position information of pixels in the first image is encoded to obtain a second image containing position encoding information;

[0007] The second image is input into the traffic light detection model for processing to obtain the detection result output by the traffic light detection model. The detection result includes: whether there is a traffic light in the first image and the location information of the traffic light.

[0008] According to a second aspect of this application, an electronic device is disclosed, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the traffic light detection method as described in the first aspect.

[0009] According to a third aspect of this application, a computer-readable storage medium is disclosed having a computer program / instructions stored thereon, which, when executed by a processor, implements the traffic light detection method as described in the first aspect.

[0010] According to a fourth aspect of this application, a computer program product is disclosed, comprising a computer program / instructions that, when executed by a processor, implement the traffic light detection method as described in the first aspect.

[0011] In this embodiment, a first image to be detected is received, and the position information of pixels in the first image is encoded to obtain a second image containing position encoding information. The second image is then input into a traffic light detection model for processing to obtain the detection result output by the traffic light detection model. The detection result includes whether a traffic light exists in the first image and the position information of the traffic light. In this embodiment, since traffic lights have significant positional features in images, that is, their positional distribution is relatively fixed, encoding the position information of the image to be detected based on this feature and inputting the image containing position encoding information into the traffic light detection model for processing can improve the detection accuracy of traffic light positions in the image, thereby reducing the false detection rate at non-traffic light positions. Attached Figure Description

[0012] Figure 1 This is a flowchart of a traffic light detection method provided in an embodiment of this application;

[0013] Figure 2 This is a flowchart of the training process of the traffic light detection model provided in the embodiments of this application;

[0014] Figure 3 This is a flowchart of another traffic light detection method provided in the embodiments of this application;

[0015] Figure 4 This is a flowchart of the training process of the traffic light recognition model provided in the embodiments of this application;

[0016] Figure 5 This is a schematic diagram of the structure of a traffic light detection device provided in an embodiment of this application;

[0017] Figure 6 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0020] In recent years, significant progress has been made in research on technologies based on artificial intelligence, such as computer vision, deep learning, machine learning, image processing, and image recognition. Artificial intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies, and application systems to simulate and extend human intelligence. AI is a comprehensive discipline involving numerous technologies, including chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, and neural networks. Computer vision, as an important branch of AI, specifically enables machines to recognize the world. Computer vision technologies typically include face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, object detection, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, and robot navigation and localization. With the research and advancement of artificial intelligence technology, this technology has been applied in numerous fields, such as security, urban management, traffic management, building management, park management, facial recognition access control, facial recognition attendance, logistics management, warehouse management, robotics, intelligent marketing, computational photography, mobile imaging, cloud services, smart homes, wearable devices, autonomous driving, autonomous driving, smart healthcare, facial payment, facial unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile internet, live streaming, beautification, makeup, medical aesthetics, and intelligent temperature measurement.

[0021] Taking the fields of assisted driving or autonomous driving as an example, traffic light detection is an extremely important task in these fields, playing a crucial role in the safe operation of vehicles. Current general-purpose object detection algorithms have achieved high metrics and accuracy on datasets such as COCO. However, traffic light detection differs significantly from general object detection. Due to the inherent characteristics of its task requirements, traffic light detection has a lower tolerance for false detections. Furthermore, traffic lights typically need to be detected at a long distance, resulting in generally small targets, making detection more difficult and leading to a higher false detection rate.

[0022] To address the aforementioned technical problems, embodiments of this application provide a traffic light detection method, an electronic device, and a storage medium.

[0023] The following is a description of a traffic light detection method provided in an embodiment of this application.

[0024] Figure 1 This is a flowchart of a traffic light detection method provided in an embodiment of this application, such as... Figure 1As shown, the method may include the following steps: step 101, step 102, and step 103, wherein,

[0025] In step 101, the first image to be detected is received.

[0026] In this embodiment of the application, the first image can be an RGB format image or an image in other formats, such as a YUV format image. This embodiment of the application does not limit the specific format.

[0027] In step 102, the position information of the pixels in the first image is encoded to obtain a second image containing position encoding information.

[0028] Considering that traffic lights have significant positional characteristics in images, i.e., their positional distribution is relatively fixed and they generally appear within a fixed positional range in the image, such as in the middle or upper part of the image from 1 / 3 to 2 / 3, and generally do not appear at the very top or very bottom edges, in order to avoid misidentification, in this embodiment of the application, the positional information of each pixel in the first image can be encoded to focus on the distribution position of traffic lights in the image.

[0029] In this embodiment, after encoding the position information of each pixel in the first image to obtain the second image, each pixel in the second image corresponds to multiple position encoding information to ensure that the encoding information can comprehensively reflect the positional features of the pixel. For example, each pixel in the second image corresponds to two, three, four, or more position encoding information, and different position encoding information is generated based on different encoding methods and / or encoding parameters.

[0030] Optionally, in one specific implementation, each pixel in the second image corresponds to four location encoding information.

[0031] In this embodiment of the application, the four positional encoding information can be divided into two groups during encoding, and the two groups adopt different encoding methods. At this time, the four positional encoding information corresponding to each pixel is generated through two encoding methods. That is, the positional features of each pixel in the image can be expressed in different dimensions to further ensure that the encoding information can fully reflect the positional features of the pixel.

[0032] In this embodiment of the application, when the first image is a three-channel color image, each pixel in the second image includes pixel values ​​of seven channels, wherein the pixel values ​​of the seven channels include pixel values ​​of three color channels and position encoding information of four position channels, and each of the four position channels corresponds to one of the position encoding information of the four position encoding information.

[0033] In one example, the first image is an RGB image, meaning that each pixel in the first image includes pixel values ​​for three color channels: the R channel, the G channel, and the B channel. After encoding the position information in the first image, each pixel in the resulting second image includes pixel values ​​for seven channels: the R, G, and B channels, P1, P2, P3, and P4. Among these, P1 to P4 represent four positional encoding information.

[0034] In some embodiments, step 102 may include the following steps (not shown in the figures): step 1021 and step 1022, wherein,

[0035] In step 1021, for each pixel in the first image, based on the pixel's position information and the sine trigonometric function, the first encoding information and the second encoding information corresponding to the pixel are determined, and based on the pixel's position information and the cosine trigonometric function, the third encoding information and the fourth encoding information corresponding to the pixel are determined; wherein, the encoding periods of the first encoding information and the second encoding information are different, and the encoding periods of the third encoding information and the fourth encoding information are different.

[0036] In one example, the first and second encoded information are sinω1t and sinω2t, respectively, and the third and fourth encoded information are cosω1t and cosω2t, respectively. Here, t is the position coordinate of the pixel, and ω1 and ω2 represent different encoding periods. Different encoding periods ω represent different trigonometric functions, which is to avoid the same encoded value for the same pixel position.

[0037] In step 1022, the first and second encoding information are used as the position encoding information for the odd-numbered positions in the four position channels of the pixel, and the third and fourth encoding information are used as the position encoding information for the even-numbered positions in the four position channels of the pixel, to obtain a second image containing position encoding information.

[0038] In one example, the positional encoding information of the four positional channels of each pixel in the second image can be [sinω1t, cosω1t, sinω2t, cosω2t].

[0039] It is understandable that, in addition to this, the first and second encoding information can also be used as the position encoding information for the even-numbered bits in the four position channels of a pixel, and the third and fourth encoding information can be used as the position encoding information for the odd-numbered bits in the four position channels of a pixel. For example, in a specific example, the position encoding information for the four position channels of each pixel in the second image can be [cosω1t, sinω1t, cosω2t, sinω2t].

[0040] In step 103, the second image is input into the traffic light detection model for processing to obtain the detection results output by the traffic light detection model. The detection results include whether there is a traffic light in the first image and the location information of the traffic light.

[0041] In this embodiment of the application, when training the traffic light detection model, each pixel position of the images in the training set can be explicitly encoded. The encoded image is then used as input and fed into the network for learning, thereby effectively embedding the position information into the model. By visualizing the final results, it is found that false detections at non-traffic light locations are significantly reduced.

[0042] Accordingly, such as Figure 2 As shown, the training process of the traffic light detection model may include the following steps: step 201, step 202, and step 203, wherein,

[0043] In step 201, a first training set is obtained, wherein the first training set includes: initial sample images and traffic light location annotation information of the initial sample images.

[0044] In this embodiment of the application, the initial sample image can be an image taken by a user holding a mobile phone and pointing at a traffic light, or it can be an image containing the influence of traffic lights acquired by other image acquisition devices.

[0045] In this embodiment of the application, the traffic light location labeling information may include: the location coordinates of the image area where the traffic light is located.

[0046] In step 202, the position information of pixels in the initial sample image is encoded to obtain a target sample image containing position encoding information.

[0047] In this embodiment of the application, the encoding method of the initial sample image is similar to that of the first image, and will not be described again here.

[0048] In step 203, the target sample image is input into the initial detection model for processing to obtain the first prediction result output by the initial detection model. Based on the first prediction result and the traffic light location annotation information, the parameters in the initial detection model are adjusted until the model converges to obtain the traffic light detection model. The first prediction result includes whether there is a traffic light in the target sample image and the location information of the traffic light.

[0049] As can be seen, in this embodiment of the application, when training the traffic light detection model, each pixel position of the images in the training set can be explicitly encoded, and the encoded image can be used as input to the network for learning, thereby effectively embedding the position information into the model, so that when using the traffic light detection model to detect traffic lights in other images, the false detection rate can be reduced.

[0050] In some embodiments, the traffic light detection model may consist of a feature extraction network, an FPN network, a classification network, and a coordinate regression network. In this case, the traffic light detection model may include the following steps when processing the second image:

[0051] The semantic features of the second image are extracted using a feature extraction network, which can be ResNet or DarkNet.

[0052] The semantic features of the second image are reused through the FPN network to obtain the target features. The FPN network is used to enhance small features and to interact features at different levels to share features / information.

[0053] The target features are processed by a classification network and a coordinate regression network to output the detection results. The classification network is used to determine whether it is a traffic light, and the coordinate regression network is used to predict the position of the traffic light, that is, the position coordinates of the traffic light in the image.

[0054] As can be seen from the above embodiments, in this embodiment, a first image to be detected is received, the position information of the pixels in the first image is encoded to obtain a second image containing position encoding information, and the second image is input into a traffic light detection model for processing to obtain the detection result output by the traffic light detection model. The detection result includes whether a traffic light exists in the first image and the position information of the traffic light. In this embodiment, since traffic lights have significant positional features in the image, that is, their positional distribution is relatively fixed, encoding the position information of the image to be detected based on this feature and inputting the image containing position encoding information into the traffic light detection model for processing can improve the detection accuracy of traffic light positions in the image, thereby reducing the false detection rate at non-traffic light positions.

[0055] Besides traffic light detection, traffic light recognition is also a crucial task in the fields of assisted driving and autonomous driving. Currently, for traffic light recognition, in real-world scenarios, due to factors such as distance, lighting, and camera imaging, traffic lights exhibit various differences in their imaging. Some samples are easily detected and correctly classified by the model; these are called "easy samples." Conversely, some samples are difficult to detect and correctly classify; these are called "difficult samples." In existing technologies, during the training process of traffic light recognition models, all input samples are treated equally. This leads to the model's inability to consider samples of different difficulty levels during convergence, ultimately resulting in poor performance on difficult samples and a high false recognition rate. To address the above technical problems, the embodiments of this application... Figure 1 Based on the illustrated embodiment, another traffic light detection method is proposed.

[0056] Figure 3 This is a flowchart of another traffic light detection method provided in the embodiments of this application, such as... Figure 3 As shown, the method may include the following steps: step 301, step 302, step 303, step 304, and step 305, wherein,

[0057] In step 301, the first image to be detected is received.

[0058] In step 302, the position information of the pixels in the first image is encoded to obtain a second image containing position encoding information.

[0059] In step 303, the second image is input into the traffic light detection model for processing to obtain the detection results output by the traffic light detection model. The detection results include whether there is a traffic light in the first image and the location information of the traffic light.

[0060] Steps 301 to 303 in the embodiments of this application are... Figure 1 Steps 101 to 103 in the illustrated embodiment are similar and will not be repeated here.

[0061] In step 304, if the detection result indicates that there is a traffic light in the first image, a third image of the area where the traffic light is located is extracted from the first image based on the location information of the traffic light in the detection result.

[0062] In this embodiment of the application, if a traffic light is present in the first image, a third image of the area where the traffic light is located can be extracted from the first image through a "cutout" operation.

[0063] In step 305, the third image is input into the traffic light recognition model for processing to obtain the recognition result output by the traffic light recognition model, wherein the recognition result includes the color information of the traffic light.

[0064] In existing technologies, difficult and easy samples are mixed together as input when training traffic light recognition models, without actively distinguishing between them. This results in a small proportion of difficult samples and insufficient training on them. To address the imbalance between difficult and easy samples in traffic light recognition, this application employs a balanced training scheme for the traffic light recognition model. During training, this scheme selects difficult samples based on the model's performance and continues training on these samples. This allows the model to accommodate samples of varying difficulty, thereby enabling the final model to have high generalization ability across different samples.

[0065] Accordingly, such as Figure 4 As shown, the training process of the traffic light recognition model may include the following steps: steps 401, 402, 403, and 404, wherein,

[0066] In step 401, a second training set is obtained, wherein the second training set includes: sample traffic light images and traffic light color labeling information of the sample traffic light images.

[0067] In this embodiment of the application, the sample traffic light images in the second training set can be traffic light images extracted from the initial sample images of the first training set.

[0068] In this embodiment of the application, the traffic light color labeling information may include: the color of the traffic light, such as whether the traffic light is red, yellow, green or black (i.e., when the light is off).

[0069] In one example, the color label information for traffic lights is in the format (D1, D2, D3, D4), where D1 represents red, D2 represents yellow, D3 represents green, and D4 represents black. If the traffic light is red, the color label information is (1,0,0,0); if the traffic light is yellow, the color label information is (0,1,0,0).

[0070] In step 402, the sample traffic light image is input into the initial recognition model for processing to obtain the second prediction result output by the initial recognition model. Based on the second prediction result and the traffic light color labeling information, the parameters in the initial recognition model are adjusted to obtain the intermediate recognition model. The second prediction result includes the probability that the traffic light color in the sample traffic light image is any of the various colors.

[0071] In this embodiment of the application, the format of the second prediction result is (M1, M2, M3, M4), where M1 to M4 represent the probabilities of the traffic light being red, yellow, green, and black.

[0072] In this embodiment, after the initial recognition network has been trained to a certain extent, the sample images previously used in the model training are filtered based on the trained model (i.e., the intermediate recognition model) to identify which samples are difficult and which are easy. In the next round of training, the selected difficult samples are used to train the model. The purpose is to increase the proportion of difficult and easy samples in the training data, force the network to converge to the difficult samples, and improve the model's ability to recognize difficult samples.

[0073] In step 403, difficult sample traffic light images are selected from the sample traffic light images based on the intermediate recognition model.

[0074] In some embodiments, step 403 may include the following steps (not shown in the figures): steps 4031, 4032, and 4033, wherein,

[0075] In step 4031, the sample traffic light image is input into the intermediate recognition model for processing to obtain the third prediction result output by the intermediate recognition model. The third prediction result includes the probability that the traffic light color in the sample traffic light image is any color.

[0076] In this embodiment of the application, the format of the third prediction result is the same as that of the second prediction result, which can be (M1, M2, M3, M4), where M1 to M4 represent the probabilities of the traffic light being red, yellow, green, and black.

[0077] In step 4032, the loss value corresponding to the sample traffic light image is calculated based on the third prediction result and the traffic light color label information.

[0078] In this embodiment of the application, the loss value corresponding to the sample traffic light image can be calculated using the cross-entropy method for the third prediction result and the traffic light color labeling information. That is, if the third prediction result is (M1, M2, M3, M4) and the traffic light color labeling information is (D1, D2, D3, D4), then the loss value is M1*D1+M2*D2+M3*D3+M4*D4.

[0079] In step 4033, difficult sample traffic light images are selected from the sample traffic light images based on the loss value.

[0080] In some embodiments, hard examples can be screened directly based on the loss value. In this case, step 4033 above includes:

[0081] Based on the magnitude of the loss value, the sample traffic light images used in training the intermediate recognition model are sorted in descending order, and the top N sample traffic light images are determined as hard sample traffic light images, where N is a second value greater than zero.

[0082] In other embodiments, to filter out as many difficult examples as possible where the predicted results differ from the labeled results, a weighting coefficient greater than 1 can be added to the loss value when the predicted results differ from the labeled results. The loss value is then multiplied by the weighting coefficient to obtain the final loss value. Based on the final loss value, the images previously used in model training are sorted from largest to smallest, and the images at the top of the sort are identified as difficult examples. In this case, step 4033 includes:

[0083] Based on the coefficient matrix, the loss values ​​corresponding to the sample traffic light images are corrected. According to the magnitude of the corrected values, the sample traffic light images participating in the training of the intermediate recognition model are sorted in descending order. The sample traffic light images ranked in the top I are determined as the difficult sample traffic light images. The coefficient matrix includes the weight coefficients corresponding to different traffic light recognition error situations, and I is the third value that is greater than zero.

[0084] In this embodiment, traffic light recognition errors may include: a red light being recognized as a green light, a red light being recognized as a yellow light, a red light being recognized as off, a yellow light being recognized as a green light, a yellow light being recognized as a red light, a yellow light being recognized as off, a green light being recognized as a red light, a green light being recognized as a yellow light, a green light being recognized as off, an off light being recognized as a green light, an off light being recognized as a red light, and an off light being recognized as a yellow light. Each traffic light recognition error will have a corresponding weighting coefficient.

[0085] In step 404, the intermediate recognition model is trained based on the traffic light images of difficult examples. The above screening and training process is repeated until the accuracy of the trained model on the test set is greater than M, thus obtaining the traffic light recognition model, where M is a first value greater than zero and less than 1. For example, M can be 0.99.

[0086] As can be seen from the above embodiments, in this embodiment, when recognizing traffic light images, the traffic light recognition model used can take into account samples of different difficulty levels during its training process. Therefore, using this model for traffic light detection can reduce the false detection rate.

[0087] Figure 5 This is a schematic diagram of the structure of a traffic light detection device provided in an embodiment of this application, as shown below. Figure 5 As shown, the traffic light detection device 500 may include:

[0088] Receiver module 501 is used to receive the first image to be detected;

[0089] Encoding module 502 is used to encode the position information of pixels in the first image to obtain a second image containing position encoding information;

[0090] The detection module 503 is used to input the second image into the traffic light detection model for processing and obtain the detection result output by the traffic light detection model. The detection result includes: whether there is a traffic light in the first image and the location information of the traffic light.

[0091] As can be seen from the above embodiments, in this embodiment, a first image to be detected is received, the position information of the pixels in the first image is encoded to obtain a second image containing position encoding information, and the second image is input into a traffic light detection model for processing to obtain the detection result output by the traffic light detection model. The detection result includes whether a traffic light exists in the first image and the position information of the traffic light. In this embodiment, since traffic lights have significant positional features in the image, that is, their positional distribution is relatively fixed, encoding the position information of the image to be detected based on this feature and inputting the image containing position encoding information into the traffic light detection model for processing can improve the detection accuracy of traffic light positions in the image, thereby reducing the false detection rate at non-traffic light positions.

[0092] Optionally, as an embodiment, each pixel in the second image corresponds to multiple location encoding information.

[0093] Optionally, as an embodiment, each pixel in the second image corresponds to four location encoding information.

[0094] Optionally, as an embodiment, the four positional encoding information corresponding to each pixel is generated through two encoding methods.

[0095] Optionally, as an embodiment, each pixel in the second image includes pixel values ​​in seven channels; wherein the pixel values ​​in the seven channels include pixel values ​​in three color channels and position encoding information in four position channels.

[0096] Optionally, as an embodiment, the encoding module 502 may include:

[0097] The encoding submodule is used to determine, for each pixel in the first image, first encoding information and second encoding information corresponding to the pixel based on the pixel's position information and a sine trigonometric function, and third encoding information and fourth encoding information corresponding to the pixel based on the pixel's position information and a cosine trigonometric function; wherein the encoding periods of the first encoding information and the second encoding information are different, and the encoding periods of the third encoding information and the fourth encoding information are different;

[0098] The first and second encoding information are used as the position encoding information for the odd-numbered positions in the four position channels of the pixel, and the third and fourth encoding information are used as the position encoding information for the even-numbered positions in the four position channels of the pixel, to obtain a second image containing position encoding information.

[0099] Optionally, as an embodiment, the traffic light detection device 500 may further include:

[0100] The extraction module is used to extract a third image of the area where the traffic light is located from the first image based on the location information of the traffic light in the detection result;

[0101] The recognition module is used to input the third image into the traffic light recognition model for processing and obtain the recognition result output by the traffic light recognition model, wherein the recognition result includes the color information of the traffic light.

[0102] Alternatively, as an embodiment, the traffic light detection model is trained through the following process:

[0103] Obtain a first training set, wherein the first training set includes: an initial sample image and traffic light location annotation information of the initial sample image;

[0104] The position information of pixels in the initial sample image is encoded to obtain a target sample image containing position encoding information;

[0105] The target sample image is input into an initial detection model for processing to obtain a first prediction result output by the initial detection model. Based on the first prediction result and the traffic light location annotation information, the parameters in the initial detection model are adjusted until the model converges to obtain the traffic light detection model. The first prediction result includes whether there is a traffic light in the target sample image and the location information of the traffic light.

[0106] Optionally, as an embodiment, the traffic light recognition model is trained through the following process:

[0107] Obtain a second training set, wherein the second training set includes: sample traffic light images and traffic light color labeling information of the sample traffic light images;

[0108] The sample traffic light image is input into an initial recognition model for processing to obtain a second prediction result output by the initial recognition model. Based on the second prediction result and the traffic light color labeling information, the parameters in the initial recognition model are adjusted to obtain an intermediate recognition model. The second prediction result includes the probability that the traffic light color in the sample traffic light image is any of the various colors.

[0109] Based on the intermediate recognition model, difficult sample traffic light images are selected from the sample traffic light images;

[0110] The intermediate recognition model is trained based on the difficult sample traffic light images. The above screening and training process is repeated until the accuracy of the trained model on the test set is greater than M, thus obtaining the traffic light recognition model, where M is a first value greater than zero and less than 1.

[0111] Optionally, as an embodiment, the difficult sample traffic light images in the sample traffic light images are obtained through the following process:

[0112] The sample traffic light image is input into the intermediate recognition model for processing to obtain a third prediction result output by the intermediate recognition model. The third prediction result includes the probability that the traffic light color in the sample traffic light image is any color.

[0113] Based on the third prediction result and the traffic light color labeling information, calculate the loss value corresponding to the sample traffic light image;

[0114] Based on the loss value, difficult sample traffic light images are selected from the sample traffic light images.

[0115] Optionally, as an embodiment, the difficult sample traffic light images in the sample traffic light images are obtained through the following process:

[0116] Based on the magnitude of the loss value, the sample traffic light images used to train the intermediate recognition model are sorted in descending order, and the top N sample traffic light images are determined as hard example sample traffic light images, where N is a second value greater than zero; or...

[0117] Based on the coefficient matrix, the loss value corresponding to the sample traffic light image is corrected. According to the magnitude of the corrected value, the sample traffic light images participating in the training of the intermediate recognition model are sorted in descending order. The sample traffic light images ranked in the top I are determined as difficult sample traffic light images. The coefficient matrix includes weight coefficients corresponding to different traffic light recognition error situations, and I is a third value greater than zero.

[0118] Any step and specific operation in any step of the traffic light detection method provided in this application can be completed by the corresponding module in the traffic light detection device. The process of the corresponding operation completed by each module in the traffic light detection device is referred to the process of the corresponding operation described in the embodiment of the traffic light detection method.

[0119] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0120] Figure 6This is a structural block diagram of an electronic device according to an embodiment of this application. The electronic device includes a processing component 622, which further includes one or more processors, and memory resources represented by memory 632 for storing instructions executable by the processing component 622, such as application programs. The application programs stored in memory 632 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 622 is configured to execute instructions to perform the methods described above.

[0121] The electronic device may also include a power supply component 626 configured to perform power management of the electronic device, a wired or wireless network interface 650 configured to connect the electronic device to a network, and an input / output (I / O) interface 658. The electronic device may operate on an operating system stored in memory 632, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or similar.

[0122] According to yet another embodiment of this application, this application also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps in the traffic light detection method as described in any of the above embodiments.

[0123] According to yet another embodiment of this application, this application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps in the traffic light detection method as described in any of the above embodiments.

[0124] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0125] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0127] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0128] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0129] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0130] The traffic light detection method, electronic device, and storage medium provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A traffic light detection method, characterized in that, The method includes: Receive the first image to be detected; The position information of pixels in the first image is encoded to obtain a second image containing position encoding information; The second image is input into the traffic light detection model for processing to obtain the detection result output by the traffic light detection model. The detection result includes: whether there is a traffic light in the first image and the location information of the traffic light. Encoding the position information of pixels in the first image to obtain a second image containing position encoding information includes: For each pixel in the first image, based on the pixel's position information and a sine trigonometric function, the first encoding information and the second encoding information corresponding to the pixel are determined; based on the pixel's position information and a cosine trigonometric function, the third encoding information and the fourth encoding information corresponding to the pixel are determined; wherein, the encoding periods of the first encoding information and the second encoding information are different, and the encoding periods of the third encoding information and the fourth encoding information are different; The first and second encoding information are used as the position encoding information for the odd-numbered positions in the four position channels of the pixel, and the third and fourth encoding information are used as the position encoding information for the even-numbered positions in the four position channels of the pixel, to obtain a second image containing position encoding information.

2. The method according to claim 1, characterized in that, Each pixel in the second image corresponds to multiple location encoding information.

3. The method according to claim 1 or 2, characterized in that, Each pixel in the second image corresponds to four location encoding information.

4. The method according to claim 3, characterized in that, The four positional encoding information corresponding to each pixel is generated through two encoding methods.

5. The method according to claim 4, characterized in that, Each pixel in the second image includes pixel values ​​from seven channels; The pixel values ​​of the seven channels include pixel values ​​of three color channels and position encoding information of four position channels.

6. The method according to claim 1, characterized in that, When the detection result indicates the presence of a traffic light in the first image, after the step of inputting the second image into the traffic light detection model for processing and obtaining the detection result output by the traffic light detection model, the method further includes: Based on the location information of the traffic lights in the detection results, a third image of the area where the traffic lights are located is extracted from the first image; The third image is input into the traffic light recognition model for processing to obtain the recognition result output by the traffic light recognition model, wherein the recognition result includes the color information of the traffic light.

7. The method according to any one of claims 1-2 and 4-6, characterized in that, The traffic light detection model is trained through the following process: Obtain a first training set, wherein the first training set includes: an initial sample image and traffic light location annotation information of the initial sample image; The position information of pixels in the initial sample image is encoded to obtain a target sample image containing position encoding information; The target sample image is input into an initial detection model for processing to obtain a first prediction result output by the initial detection model. Based on the first prediction result and the traffic light location annotation information, the parameters in the initial detection model are adjusted until the model converges to obtain the traffic light detection model. The first prediction result includes whether there is a traffic light in the target sample image and the location information of the traffic light.

8. The method according to claim 6, characterized in that, The traffic light recognition model is trained through the following process: Obtain a second training set, wherein the second training set includes: sample traffic light images and traffic light color labeling information of the sample traffic light images; The sample traffic light image is input into an initial recognition model for processing to obtain a second prediction result output by the initial recognition model. Based on the second prediction result and the traffic light color labeling information, the parameters in the initial recognition model are adjusted to obtain an intermediate recognition model. The second prediction result includes the probability that the traffic light color in the sample traffic light image is any of the various colors. Based on the intermediate recognition model, difficult sample traffic light images are selected from the sample traffic light images; The intermediate recognition model is trained based on the difficult sample traffic light images. The above screening and training process is repeated until the accuracy of the trained model on the test set is greater than M, thus obtaining the traffic light recognition model, where M is a first value greater than zero and less than 1.

9. The method according to claim 8, characterized in that, The step of filtering difficult traffic light images from the sample traffic light images based on the intermediate recognition model includes: The sample traffic light image is input into the intermediate recognition model for processing to obtain a third prediction result output by the intermediate recognition model. The third prediction result includes the probability that the traffic light color in the sample traffic light image is any color. Based on the third prediction result and the traffic light color labeling information, calculate the loss value corresponding to the sample traffic light image; Based on the loss value, difficult sample traffic light images are selected from the sample traffic light images.

10. The method according to claim 9, characterized in that, The step of filtering difficult traffic light images from the sample traffic light images based on the loss value includes: Based on the magnitude of the loss value, the sample traffic light images used to train the intermediate recognition model are sorted in descending order, and the top N sample traffic light images are determined as hard example sample traffic light images, where N is a second value greater than zero; or... Based on the coefficient matrix, the loss value corresponding to the sample traffic light image is corrected. According to the magnitude of the corrected value, the sample traffic light images participating in the training of the intermediate recognition model are sorted in descending order. The sample traffic light images ranked in the top I are determined as difficult sample traffic light images. The coefficient matrix includes weight coefficients corresponding to different traffic light recognition error situations, and I is a third value greater than zero.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1-10.

12. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1-10.

13. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1-10.

Images