Signal light detection method, device and computer readable storage medium
By identifying and predicting the status changes of the signal lights, combining spatial and time domain information, the accuracy of signal light detection is improved, and the problem of low signal light detection accuracy in the prior art is solved, thereby achieving higher detection accuracy and cost-effectiveness.
Patent Information
- Application Number
- CN202110939496.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-16
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-08-16
AI Technical Summary
In order to simultaneously distinguish vehicle type, vehicle color, license plate number, faces in the car, etc. in the smart traffic police scene, the video streaming and snapping images require a long exposure time, resulting in low detection accuracy of signal lights. It is prone to mis-checking.
By acquiring the image to be detected and its previous historical images, the status change results of the target signal light are identified, and the current status of the signal light is predicted based on spatial and time domain information to reduce the error detection rate.
It improves the accuracy of signal light detection and reduces the error detection rate. It is suitable for any application scenario without adding filters, reducing the cost of hardware and manual debugging.
Smart Images

Figure CN113723229B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a signal light detection method, device and computer-readable storage medium. Background Art
[0002] In intelligent traffic electronic police scenarios, traffic light detection technology is crucial to regulating vehicle behavior. However, in order to simultaneously identify vehicle type, vehicle color, license plate number, faces inside the vehicle and other information, existing cameras require long exposure times for video streams and captured images, resulting in low traffic light detection accuracy and prone to false detections. Summary of the invention
[0003] The present application provides a signal light detection method, device and computer-readable storage medium, which can improve the accuracy of signal light detection.
[0004] In a first aspect, an embodiment of the present application provides a method for detecting a traffic light, the method comprising: acquiring an image to be detected and a historical image that is chronologically preceding the image to be detected, the image to be detected and the historical image both including a target traffic light; identifying the image to be detected and obtaining a first state of the target traffic light in the image to be detected; based on the historical image and the image to be detected, identifying a state change result of the target traffic light from the historical image to the image to be detected; predicting a second state of the target traffic light in the image to be detected based on a historical state of a traffic light logic area corresponding to the target traffic light in the historical image and the state change result; and obtaining a current state of the target traffic light in the image to be detected based on the first state and the second state.
[0005] A second aspect of an embodiment of the present application provides a signal light detection device, which includes a processor, a memory, and a communication circuit. The processor is coupled to the memory and the communication circuit, respectively. The memory stores program data. The processor implements the steps in the above method by executing the program data in the memory.
[0006] A third aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program can be executed by a processor to implement the steps in the above method.
[0007] The beneficial effect is: the present application first obtains the first state based on space, color and other information, and obtains the second state based on time domain information, that is, the first state and the second state are state information obtained from different angles, and then based on the first state and the second state, obtains the current state of the target signal light in the image to be detected. Compared with relying only on the first state, it can reduce false detection and improve detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:
[0009] Figure 1 It is a flow chart of an implementation method of a signal light detection method of the present application;
[0010] Figure 2 It is an application scenario corresponding to Figure 1 Schematic diagram of the detection method framework;
[0011] Figure 3 It is a flow chart of another implementation method of the signal light detection method of the present application;
[0012] Figure 4 yes Figure 1 A schematic diagram of the process of step S150 in an application scenario;
[0013] Figure 5 It is a structural schematic diagram of an implementation method of a detection device for a signal light of the present application;
[0014] Figure 6 It is a structural schematic diagram of another embodiment of the detection device of the signal light of the present application;
[0015] Figure 7 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION
[0016] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0017] See also Figure 1 , Figure 11 is a flow chart of an embodiment of a method for detecting a signal light of the present application, the method comprising:
[0018] S110: Acquire an image to be detected and a historical image that is before the image to be detected in time sequence, wherein both the image to be detected and the historical image include a target signal light.
[0019] Specifically, the shooting time of the historical image is earlier, and the shooting time of the image to be detected is later, but both include the target signal light.
[0020] In an application scenario, the historical image and the image to be detected belong to the image data in the same video stream. In this case, the historical image can be the previous frame of the image to be detected, or can be an image that is separated from the image to be detected by several frames, for example, the interval between the historical image and the image to be detected is one frame, two frames, or four frames. For the sake of convenience, the following description is based on the historical image being the previous frame of the image to be detected.
[0021] Among them, the state of the target signal light changes at most once from the historical image to the image to be detected.
[0022] Among them, the target signal light of the present application emits monochromatic light, or emits light of different colors at different times. For example, the target signal light can only emit red light, green light or yellow light, or the target signal light can convert between the three colors of red light, green light and yellow light.
[0023] It is understandable that when the target signal light can only emit monochromatic light, it needs to cooperate with other signal lights to complete a complete signal light logic (the complete signal light logic is: red light switches to green light, green light switches to yellow light). When the target signal light can switch between red light, green light and yellow light, one target signal light can realize a complete signal light logic.
[0024] Among them, the number of target traffic lights in the image to be detected and the historical image can be one or more than two. When the number of target traffic lights is more than two, the subsequent steps are performed separately for different target traffic lights to obtain the current state of the corresponding target traffic light in the image to be detected.
[0025] S120: Identify the image to be detected and obtain a first state of the target signal light in the image to be detected.
[0026] Specifically, the image to be detected is recognized to obtain a first state of the target signal light in the image to be detected.
[0027] Since the first state is obtained by directly recognizing the image to be detected, the first state is obtained by using information such as color and space of the image to be detected.
[0028] In one application scenario, combined with Figure 2 In order to improve the efficiency and accuracy of recognition, step S120 of identifying the image to be detected includes: sending the image to be detected to a pre-trained state model for recognition to obtain the first state of the target signal light in the image to be detected. The specific structure of the state model is introduced below.
[0029] S130: Based on the historical image and the image to be detected, identifying the state change result of the target traffic light from the historical image to the image to be detected.
[0030] Specifically, by comparing the historical image with the image to be detected, the state change result of the target traffic light from the historical image to the image to be detected can be obtained.
[0031] It can be understood that the state change result is the state change result of the target traffic light from the shooting time of the historical image to the shooting time of the image to be detected.
[0032] Among them, since the target signal light emits monochromatic light, or emits light of different colors at different times, the state change result may be from off to bright, color change, no color change, or from bright to off.
[0033] Among them, since the state change result is obtained based on the historical image and the image to be detected, the state change result is obtained by using the time domain information.
[0034] In an application scenario, continue to combine Figure 2 In order to improve the efficiency and accuracy of recognition, step S130 of identifying the state change result includes: sending the historical image and the image to be detected to a pre-trained conversion model for recognition, and obtaining the state change result of the target traffic light from the historical image to the image to be detected.
[0035] Specifically, the historical image and the image to be detected are simultaneously fed into the conversion model to obtain the state change result of the target signal light. In an application scenario, the historical image and the image to be detected are channel-merged and then fed into the conversion model for recognition.
[0036] Among them, the state model and the transition model are both pre-trained network models with a high recognition rate. They can accurately detect the location, status and other information of traffic lights, and can greatly remove interference from non-signal light areas such as car lights, body color, and street lights. There is no need to refer to red and green color information to set threshold parameters, control the camera's long and short exposure time, add filters, etc., which can reduce the workload of manual parameter debugging.
[0037] At the same time, the state model and the transition model mainly include multiple convolutional layers, activation layers, pooling layers, upsampling layers, and cascade processing layers. Among them, the processing of the convolutional layers, activation layers, pooling layers, upsampling layers, and cascade processing layers is the same as that of the prior art, and will not be repeated here.
[0038] In one application scenario, before an image is input into a state model or a conversion model, color conversion is required to convert the image from YUV data to RGB data.
[0039] In another application scenario, the state model does not directly output the first state, but outputs multiple feature data representing the first state. In this case, after the state model outputs the feature data, the output feature data must also be post-processed. For example, feature data with high confidence levels are first screened out, and then a non-maximum suppression algorithm is used to filter out repeated or similar feature data, and the accurate first state is obtained based on the final feature data.
[0040] Similarly, the feature data output by the conversion model also needs to be post-processed to obtain accurate state change results.
[0041] Among them, when the number of target signal lights in the image to be detected is more than two, in order to improve the detection efficiency, a region of interest can be set in the image to be detected, and one region of interest corresponds to a complete signal light logic area (the signal light logic area is: an area that can present a complete signal light logic). For example, when the target signal light can only emit monochromatic light, one region of interest includes target signal lights that emit red light, green light and yellow light respectively. When the target signal light can emit red light, green light and yellow light, one region of interest can include only one target signal light. At the same time, different regions of interest in the image to be detected will not cross each other to avoid logical confusion.
[0042] When determining the region of interest, the region of interest may be set to be larger than the corresponding signal light logic area, so as to avoid the region of interest being out of the signal light logic area due to camera shaking or swaying, thereby enhancing the stability of the entire method.
[0043] After determining the regions of interest, clustering is performed on the regions of interest through a clustering algorithm, and the regions of interest are finally divided into several categories. In one application scenario, after clustering, two regions of interest whose distance is less than a distance threshold are in the same category, otherwise they are in different categories; in another application scenario, after clustering, in the same category, for any region of interest, there is a region of interest whose distance is less than the distance threshold, and in different categories, the distance between any two regions of interest is not less than the distance threshold.
[0044] The distance between the same position points of the two regions of interest may be calculated and determined as the distance between the two regions of interest. For example, the distance between the upper left vertices of the two regions of interest may be determined as the distance between the two regions of interest.
[0045] After determining the categories of the regions of interest, a detection box is set for each category, and the detection box selects all regions of interest of the corresponding category. It can be understood that the number of detection boxes is the same as the number of categories of the regions of interest. In an application scenario, the sizes of the detection boxes are set to be the same.
[0046] It should be noted that, for the detection frame in the image to be detected, there is also a corresponding detection frame in the historical image. The two detection frames correspond here to each other, which means that the two detection frames include the same target signal light. When the shooting angles of the image to be detected and the historical image are the same, the positions and numbers of the detection frames in the image to be detected and the historical image are exactly the same. At this time, you only need to determine the position of the detection frame in one of the image to be detected and the historical image, and then apply the determined detection frame to the other.
[0047] After the detection frame is determined, the images of each detection frame in the image to be detected and the historical image are extracted, and then the target is recognized based on the extracted images. For example, the extracted images are sent to the state model and the conversion model for target recognition.
[0048] That is to say, during one recognition process, multiple target signal lights can be recognized simultaneously, thereby improving the detection efficiency.
[0049] At the same time, since multiple target signal lights can be recognized at one time, the final recognition result includes the results corresponding to multiple target signal lights. In order to facilitate subsequent processing, after the recognition is completed, the recognition results will be assigned to the corresponding regions of interest. In other words, when using the state model and the transition model for target recognition, the recognition results output by the state model and the transition model will finally be assigned to the corresponding regions of interest for subsequent processing.
[0050] S140: Predicting a second state of the target signal light in the image to be detected based on the historical state of the signal light logic area corresponding to the target signal light in the historical image and the state change result.
[0051] Specifically, the signal light logic area corresponding to the target signal light refers to the signal light logic area where the target signal light is located, wherein a signal light logic area can present the signal light logic of a red light switching to a green light, and a green light switching to a yellow light.
[0052] From the above analysis, it can be seen that when the target signal light can only emit monochromatic light, the signal light logic area corresponding to the target signal light includes other signal lights in addition to the target signal light. When the target signal light can convert red light, green light and yellow light, the signal light logic area corresponding to the target signal light may only include the target signal light.
[0053] According to the historical states of the signal light logic area corresponding to the target signal light in the historical image and the state change result, the second state of the target signal light in the image to be detected can be predicted.
[0054] For example, when the state change result is that the color of the target signal light changes, and the historical state of the target signal light in the historical image is a red light, it can be predicted that the second state of the target signal light in the image to be detected is a green light. Or, when the state change result is that the target signal light changes from off to on, and the historical state of the signal light logic area corresponding to the target signal light in the historical image is a yellow light, it can be predicted that the second state of the target signal light in the image to be detected is a red light.
[0055] S150: Based on the first state and the second state, obtain the current state of the target signal light in the image to be detected.
[0056] Specifically, the first state is obtained using spatial, color and other information, while the second state is obtained using time domain information. Both are descriptions of the current state of the target signal light in the image to be detected, and both have the probability of error. Therefore, the first state and the second state are integrated to obtain the current state of the target signal light, which can improve the detection accuracy.
[0057] In the prior art, the first state is generally directly determined as the current state of the target signal light in the image to be detected. However, in some application scenarios, due to the long exposure time, the red light and the yellow light in the image are easily confused. Therefore, the prior art approach has a high false detection rate. The present application combines the first state and the second state to obtain the current state of the target signal light in the image to be detected. The first state is obtained using spatial, color and other information, and the second state is obtained using time domain information. The two are state information obtained from different angles and are not directly dependent on the first state or the second state, which can reduce false detections and improve detection accuracy.
[0058] At the same time, the method of the present application does not require the installation of any filter solutions, and can be used with general electric police surveillance cameras. It will not change the overall brightness of the image, nor will it affect the quality of the image, and can reduce the cost of hardware and manual debugging.
[0059] At the same time, the method of the present application is not limited by the scenario and is applicable to any application scenario.
[0060] It is understandable that when the next frame of the image to be detected is subjected to traffic light detection, the image to be detected will become a historical image, and information such as the state of the target traffic light detected this time will become historical information for the next frame of the image.
[0061] Considering that not all states are prone to false detection, Figure 3 In other embodiments, before step S130 identifies the state change result, it also includes:
[0062] S121: Determine whether the first state of the target signal light in the image to be detected is a predetermined state.
[0063] If it is determined to be the predetermined state, step S130 is executed to identify the state change result; otherwise, step S122 is entered.
[0064] S122: Based on the first state, obtain the current state of the target signal light in the image to be detected.
[0065] Specifically, the predetermined state is a state that is prone to false detection. If the first state is the predetermined state, then the current state of the target signal light in the image to be detected is obtained based on the first state and the second state, otherwise, the current state of the target signal light is obtained directly based on the first state.
[0066] The step of directly obtaining the current state of the target signal light according to the first state includes: determining the first state as the current state of the target signal light.
[0067] In an application scenario, the predetermined state is a light-emitting state, that is, if step S120 identifies that the target signal light is in a light-emitting state in the image to be detected, then steps S130 to S150 are executed, otherwise step S122 is executed.
[0068] In another application scenario, the predetermined state is emitting yellow light or emitting red light. That is to say, if step S120 identifies that the state of the target signal light in the image to be detected is emitting yellow light or emitting red light, then steps S130 to S150 are executed. That is to say, at this time, in order to avoid confusion between yellow light and red light, the current state of the target signal light in the image to be detected is obtained based on the first state and the second state. Otherwise, when the first state is identified as emitting green light or being in an off state, the first state is directly used as the current state of the target signal light.
[0069] In this embodiment, combined with Figure 4 In this embodiment, step S150 includes:
[0070] S151: Based on the confidence of identifying the first state, obtain a first confidence corresponding to the first state.
[0071] Specifically, the confidence level of identifying the first state is the confidence level that the state of the target signal light in the image to be detected is the identified first state. The higher the confidence level, the higher the probability that the state of the target signal light in the image to be detected is the first state.
[0072] After the confidence level of identifying the first state is obtained, the confidence level of identifying the first state is converted into the first confidence level according to a preset strategy.
[0073] When the first state is obtained by using the state model for identification, the confidence level of identifying the first state is the confidence level output by the state model.
[0074] S152: Based on the confidence of the recognition state change result, obtain a second confidence corresponding to the second state.
[0075] Specifically, the confidence level of the recognition state change result is the confidence level that the change result of the target signal light from the historical image to the image to be detected is the recognized state change result. The higher the confidence level, the higher the probability that the change result of the target signal light from the historical image to the image to be detected is the recognized state change result.
[0076] After the confidence level of the recognition state change result is obtained, the confidence level of the recognition state change result is converted into a second confidence level according to a preset strategy.
[0077] When the state change result is obtained by using the conversion model for identification, the confidence of the identified state change result is the confidence of the conversion model output.
[0078] S153: Based on the first confidence level and the second confidence level, determine one of the first state and the second state as the current state of the target signal light in the image to be detected.
[0079] In one application scenario, the first confidence is positively correlated with the confidence of identifying the first state, and the second confidence is positively correlated with the confidence of the identification state change result. At this time, if the first confidence is greater than the second confidence, the first state is determined as the current state of the target signal light in the image to be detected, otherwise the second state is determined as the current state of the target signal light in the image to be detected.
[0080] In another application scenario, the first confidence is negatively correlated with the confidence of identifying the first state, and the second confidence is negatively correlated with the confidence of the identification state change result. At this time, if the first confidence is greater than the second confidence, the second state is determined as the current state of the target signal light in the image to be detected, otherwise the first state is determined as the current state of the target signal light in the image to be detected.
[0081] In this embodiment, when the first state is finally determined as the current state, the confidence of identifying the first state is also determined as the confidence of the current state; when the second state is finally determined as the current state, the second confidence is determined as the confidence of the current state.
[0082] For example, when the first state is determined as the current state of the target signal light, the confidence output by the state model is used as the confidence of the current state of the target signal light; when the second state is determined as the current state of the target signal light, the second confidence is directly used as the confidence of the current state of the target signal light.
[0083] In one application scenario, step S151 of determining the first confidence level includes: multiplying the confidence level of identifying the first state by a first preset factor to determine the first confidence level.
[0084] Specifically, the first confidence level is determined using the following formula:
[0085] conf1=conf det *α, where conf1 is the first confidence level, conf det To identify the confidence of the first state, α is a first preset factor.
[0086] Among them, since when the first state is finally determined as the current state, the confidence of identifying the first state is also determined as the confidence of the current state, after the first confidence conf1 is calculated, if the first state is finally determined as the current state, the first confidence is also restored at this time, and the result of conf1 / α (that is, the confidence of identifying the first state conf det ), determined as the confidence of the current state.
[0087] In an application scenario, the first preset factor is greater than zero and less than one, wherein the first preset factor can be set according to actual needs and is not specifically limited here.
[0088] In another application scenario, step S152 of determining the second confidence level includes: when the historical state of the target signal light in the historical image is a lighting state, obtaining the second confidence level based on the confidence level of the recognition state change result and the confidence level of the historical state of the target signal light.
[0089] Specifically, when the historical state of the target signal light in the historical image is a lighting state, the second confidence is obtained by combining the confidence of the recognition state change result and the confidence of the historical state of the target signal light.
[0090] For example, when the historical state of the target signal light is a lighting state, and the state change result is that the color of the target signal light has not changed, the product of the confidence of the identification state change result, the confidence of the historical state of the target signal light, and a second preset factor is determined as the second confidence; when the historical state of the target signal light is a lighting state, and the state change result is that the color of the target signal light has changed, the average of the confidence of the identification state change result and the confidence of the historical state of the target signal light is determined as the second confidence.
[0091] Specifically, when the historical state of the target signal light is a lighting state, and the state change result is that the color of the target signal light does not change, the second confidence level is determined using the following formula:
[0092] conf2=conf pre *conf uch *β, where conf2 is the second confidence level, conf pre is the confidence of the historical state of the target signal light, conf uch is the confidence level of the state change result (the state change result is that the color of the target signal light has not changed), and β is the second preset factor. In an application scenario, the second preset factor is greater than zero and less than one, wherein the second preset factor can be set according to actual needs and is not specifically limited here.
[0093] When the historical state of the target signal light is the lighting state, and the state change result is that the color of the target signal light changes, the second confidence level is determined using the following formula:
[0094] conf2=(conf pre +conf ch ) / 2, where conf2 is the second confidence level, conf pre is the confidence of the historical state of the target signal light, conf ch It is the confidence level of the recognition state change result (in this case, the state change result is the color change of the target signal light).
[0095] In another application scenario, the step of determining the second confidence level in step S152 includes: when the historical state of the target traffic light in the historical image is an off state, determining the confidence level of the recognition state change result as the second confidence level.
[0096] Specifically, when the historical state of the target signal light in the historical image is the off state, the confidence level of the recognition state change result is directly determined as the second confidence level. At this time, the second confidence level is determined using the following formula:
[0097] conf2=conf lgt, where conf2 is the second confidence level, conf lgt is the confidence level of the recognition state change result (at this time, the historical state of the target signal light in the historical image is off).
[0098] When the second state is finally determined as the current state, the second confidence level is determined as the confidence level of the current state, that is, conf2 is determined as the confidence level of the current state, and confidence level restoration is not required at this time.
[0099] In this embodiment, in addition to detecting the state of the target signal light, the target signal light is also positioned. At this time, while obtaining the first state of the target signal light in the image to be detected in step S120, it also includes: predicting the first position of the target signal light in the image to be detected; while identifying the state change result of the target signal light from the historical image to the image to be detected in step S130, it also includes: predicting the second position of the target signal light in the image to be detected. It can be understood that the first position and the second position are both descriptions of the position of the target signal light in the image to be detected. Among them, the first position and the second position are both represented by the position of the identification box of the target signal light.
[0100] At this time, after the first state is determined as the current state, the current position of the target signal light in the image to be detected is obtained based on the first position; after the second state is determined as the current state, the current position of the target signal light in the image to be detected is obtained based on the second position.
[0101] Specifically, when the first state is determined as the current state, it means that the accuracy of step S120 in identifying the first state is high, and based on the first position, the current position of the target signal light in the image to be detected is obtained; and when the second state is determined as the current state, it means that the accuracy of step S130 in identifying the state change result is high, and based on the second position, the current position of the target signal light in the image to be detected is obtained.
[0102] In this embodiment, before obtaining the first confidence in step S151, it also includes: determining whether there is a standard position of the target signal light; if so, performing a first correction on the confidence of identifying the first state based on the degree of deviation between the first position and the standard position, and determining the confidence after the first correction as the confidence of identifying the first state.
[0103] Specifically, the standard position of the target signal light may be preset manually or obtained through calculation, wherein the calculation process may be referred to below.
[0104] The degree of deviation between the first position and the standard position indicates the accuracy of identifying the first state, that is, the farther the first position is from the standard position, the lower the possibility that the state of the target signal light in the image to be detected is the first state.
[0105] The intersection-and-joint ratio of the first position and the standard position may be calculated to characterize the degree of deviation between the first position and the standard position.
[0106] In one application scenario, the confidence of identifying the first state is corrected using the following formula:
[0107] conf m =conf n *cos(1-iou), where conf n is the confidence of the first state of recognition before correction, conf m is the confidence of the corrected recognition of the first state, and iou is the intersection-and-union ratio of the first position to the standard position.
[0108] After the correction is performed, the confidence level of the identification of the first state is updated to the first corrected confidence level, and then step S151 is executed.
[0109] It should be noted that, when the first state is subsequently determined as the current state, since the confidence of identifying the first state has been updated to the first corrected confidence, the first corrected confidence is determined as the confidence of the current state.
[0110] In this embodiment, after step S152, it also includes: determining whether there is a standard position of the target signal light; if so, performing a second correction on the second confidence level based on the degree of deviation between the second position and the standard position, and determining the confidence level after the second correction as the second confidence level.
[0111] Specifically, the degree of deviation between the second position and the standard position indicates the probability that the state of the target signal light in the image to be detected is the second state. Specifically, the farther the second position is from the standard position, the lower the possibility that the state of the target signal light in the image to be detected is the second state.
[0112] Similar to the degree of deviation between the first position and the standard position, the intersection-and-union ratio between the second position and the standard position may also be calculated to characterize the degree of deviation between the second position and the standard position.
[0113] In one application scenario, the second confidence level is corrected using the following formula:
[0114] conf p =conf q *cos(1-iou), where conf q is the second confidence before correction, conf p is the second confidence after correction, and iou is the intersection-over-union ratio of the second position to the standard position.
[0115] After correction, the second confidence level is updated to the second corrected confidence level, and then step S153 is performed.
[0116] It should be noted that, when the second state is subsequently determined as the current state, since the second confidence has been updated to the second corrected confidence, the second corrected confidence is determined as the confidence of the current state.
[0117] It can be understood that when there is no standard position of the target signal light, there is no need to calibrate the confidence level and the second confidence level for identifying the first state, and steps S151 to S153 are directly executed.
[0118] At the same time, in other embodiments, the confidence of identifying the first state may be corrected only when the degree of deviation between the first position and the standard position exceeds a set threshold, otherwise no correction is performed; or the second confidence may be corrected only when the degree of deviation between the second position and the standard position exceeds a set threshold, otherwise no correction is performed.
[0119] Alternatively, in other embodiments, only one of the confidence level for identifying the first state and the second confidence level may be corrected, while the other one is not corrected.
[0120] In this embodiment, the step of obtaining the current position of the target signal light in the image to be detected based on the first position includes:
[0121] When there is a standard position of the target signal light, the current position of the target signal light in the image to be detected is obtained based on the first position and the standard position; when there is no standard position, the first position is determined as the current position of the target signal light in the image to be detected.
[0122] The step of obtaining the current position of the target signal light in the image to be detected based on the second position includes:
[0123] When there is a standard position of the target signal light, the current position of the target signal light in the image to be detected is obtained based on the second position and the standard position; when there is no standard position, the second position is determined as the current position of the target signal light in the image to be detected.
[0124] Specifically, when there is a standard position of the target signal light, it is necessary to determine the accuracy of the first position or the second position, and then obtain the current position of the target signal light in the image to be detected based on the first position and the standard position, or obtain the current position of the target signal light in the image to be detected based on the second position and the standard position. When there is no standard position of the target signal light, the first position or the second position is directly determined as the current position of the target signal light in the image to be detected.
[0125] In one application scenario, the step of obtaining the current position of the target signal light in the image to be detected based on the first position and the standard position includes:
[0126] If the degree of deviation between the first position and the standard position is less than a first degree threshold, the first position is determined as the current position of the target signal light in the image to be detected; otherwise, the standard position is determined as the current position of the target signal light in the image to be detected.
[0127] The step of obtaining the current position of the target signal light in the image to be detected based on the second position and the standard position includes:
[0128] If the deviation degree between the second position and the standard position is less than a second degree threshold, the second position is determined as the current position of the target signal light in the image to be detected; otherwise, the standard position is determined as the current position of the target signal light in the image to be detected.
[0129] Specifically, when the degree of deviation between the first position and the standard position is less than a first degree threshold, it means that the accuracy of the first position is relatively high, and the first position can be directly determined as the current position of the target signal light in the image to be detected; otherwise, it means that the accuracy of the first position is relatively low, and the first position is inaccurate. At this time, the standard position is directly determined as the current position of the target signal light in the image to be detected.
[0130] Similarly, when the degree of deviation between the second position and the standard position is less than the second degree threshold, it means that the accuracy of the second position is relatively high, and the second position can be directly determined as the current position of the target signal light in the image to be detected; otherwise, it means that the accuracy of the second position is relatively low and the second position is inaccurate. At this time, the standard position is directly determined as the current position of the target signal light in the image to be detected.
[0131] That is, when the accuracy of determining the first position or the second position is low, the first position or the second position is directly eliminated, and the standard position is used as the current position of the target signal light, thereby ensuring the detection accuracy of the target signal light.
[0132] Wherein, if the degree of deviation between the first position and the standard position is less than a first degree threshold, after determining the first position as the current position of the target signal light in the image to be detected, the method further includes: updating the standard position based on the first position.
[0133] If the degree of deviation between the second position and the standard position is less than a second degree threshold, then after determining the second position as the current position of the target signal light in the image to be detected, the method further includes: updating the standard position based on the second position.
[0134] Specifically, when the accuracy of the first position is high, after the first position is determined as the current position of the target signal light, the standard position needs to be updated; when the accuracy of the second position is high, after the second position is determined as the current position of the target signal light, the standard position needs to be updated.
[0135] Among them, the standard position can be updated using the following formula:
[0136] Among them, dbbox represents the coordinates of the first position or the coordinates of the second position, qt represents the number of calculations of the standard position, and cbbox represents the coordinates of the standard position.
[0137] After the standard position is updated, qt=qt+1, that is, the number of calculations of the standard position is increased by one.
[0138] In this embodiment, when there is no standard position, after obtaining the current state of the target signal light in the image to be detected, the method further includes:
[0139] Based on the historical position of the target signal light and the current position of the target signal light in the image to be detected, the standard position of the target signal light is obtained to serve as the standard position of the target signal light in the next detection process.
[0140] Specifically, the current position of the target signal light in the image to be detected is the final position of the target signal light in the image to be detected.
[0141] When there is no standard position, after obtaining the current position of the target signal light in the image to be detected, the standard position is determined according to the final position of the target signal light in the image that has been determined.
[0142] It is understandable that when the next frame of the image to be detected is subjected to traffic light detection, the current position of the target traffic light in the image to be detected will become the historical position of the target traffic light.
[0143] Among them, the method for determining the standard position may include: calculating the average value of the final positions of all the determined target signal lights in the image, or, it may also include: clustering the final positions of the determined target signal lights in the image to obtain the standard position, wherein the clustering processing here can adopt the mean shift clustering technology.
[0144] Before obtaining the standard position of the target signal light based on the historical position of the target signal light and the current position of the target signal light in the image to be detected, the method further includes:
[0145] Determine whether the number of historical positions of the target signal light reaches a number threshold;
[0146] If it is reached, the standard position of the target signal light is obtained based on the historical position of the target signal light and the current position of the target signal light in the image to be detected.
[0147] Specifically, only when the number of historical positions of the target signal light reaches a number threshold, a standard position is generated; otherwise, a standard position is not generated.
[0148] See also Figure 5 , Figure 5 It is a structural schematic diagram of an embodiment of a detection device for a signal light of the present application. The detection device 200 for a signal light includes a processor 210, a memory 220 and a communication circuit 230. The processor 210 is coupled to the memory 220 and the communication circuit 230, respectively. The memory 220 stores program data. The processor 210 implements the steps in any of the above-mentioned implementation methods by executing the program data in the memory 220. The detailed steps can be found in the above-mentioned implementations and will not be repeated here.
[0149] The signal light detection device 200 may be any device with image processing capability, such as a computer or a mobile phone, and is not limited here.
[0150] See also Figure 6 , Figure 6 It is a structural diagram of another embodiment of the signal light detection device of the present application. The signal light detection device 300 includes an image acquisition module 310, a first recognition module 320, a second recognition module 330, a state prediction module 340 and a current state module 350.
[0151] The image acquisition module 310 is used to acquire the image to be detected and the historical images that are before the image to be detected in time sequence. Both the image to be detected and the historical images include the target signal light.
[0152] The first recognition module 320 is connected to the image acquisition module 310 and is used to recognize the image to be detected and obtain the first state of the target signal light in the image to be detected.
[0153] The second recognition module 330 is connected to the image acquisition module 310 and is used to recognize the state change result of the target traffic light from the historical image to the image to be detected based on the historical image and the image to be detected.
[0154] The state prediction module 340 is connected to the second recognition module 330 and the image acquisition module 310 at the same time, and is used to predict the second state of the target signal light in the image to be detected based on the historical state of the signal light logic area corresponding to the target signal light in the historical image and the state change result;
[0155] The current state module 350 is connected to the first recognition module 320 and the state prediction module 340 at the same time, and is used to obtain the current state of the target signal light in the image to be detected based on the first state and the second state.
[0156] The signal light detection device 300 may be any device with image processing capability, such as a computer or a mobile phone, and is not limited here.
[0157] Among them, the image acquisition module 310, the first recognition module 320, the second recognition module 330, the state prediction module 340 and the current state module 350 in the traffic light detection device 300 cooperate with each other to execute the steps in the traffic light detection method in any of the above-mentioned embodiments, wherein the detailed method steps can be referred to the above-mentioned embodiments and will not be repeated here.
[0158] See also Figure 7 , Figure 7 The computer-readable storage medium 400 stores a computer program 410, which can be executed by a processor to implement the steps in any of the above methods.
[0159] Among them, the computer-readable storage medium 400 can specifically be a device that can store the computer program 410, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or it can also be a server that stores the computer program 410. The server can send the stored computer program 410 to other devices for execution, or it can also run the stored computer program 410 by itself.
[0160] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A method for detecting a signal light, It is characterized in that The method comprises: Acquire an image to be detected and a historical image that precedes the image to be detected in time sequence, wherein both the image to be detected and the historical image include a target signal light; Identify the image to be detected, and obtain a first state of the target signal light in the image to be detected; Based on the historical image and the image to be detected, identifying a state change result of the target signal light from the historical image to the image to be detected; Predicting a second state of the target signal light in the image to be detected based on a historical state of the signal light logic area corresponding to the target signal light in the historical image and the state change result; Based on the first state and the second state, obtaining a current state of the target signal light in the image to be detected; Wherein, while obtaining the first state of the target signal light in the image to be detected, the method further includes: predicting the first position of the target signal light in the image to be detected; While identifying the state change result of the target signal light from the historical image to the image to be detected, the method further includes: predicting a second position of the target signal light in the image to be detected; The method further comprises: After determining the first state as the current state, obtaining the current position of the target signal light in the image to be detected based on the first position; After the second state is determined as the current state, the current position of the target signal light in the image to be detected is obtained based on the second position.
2. The method according to claim 1, It is characterized in that Before identifying the state change result of the target signal light from the historical image to the image to be detected based on the historical image and the image to be detected, the method further includes: Determining whether the first state of the target signal light in the image to be detected is a predetermined state; If it is determined to be the predetermined state, the step of identifying the state change result of the target signal light from the historical image to the image to be detected based on the historical image and the image to be detected is performed; Otherwise, based on the first state, the current state of the target signal light in the image to be detected is obtained.
3. The method according to claim 1, It is characterized in that The step of obtaining the current state of the target signal light in the image to be detected based on the first state and the second state includes: Based on the confidence level of identifying the first state, obtaining a first confidence level corresponding to the first state; Based on the confidence of the result of identifying the state change, obtaining a second confidence corresponding to the second state; Based on the first confidence level and the second confidence level, one of the first state and the second state is determined as the current state of the target signal light in the image to be detected.
4. The method according to claim 3, It is characterized in that The step of obtaining a first confidence level corresponding to the first state based on identifying the confidence level of the first state comprises: The product of the confidence of identifying the first state and a first preset factor is determined as the first confidence.
5. The method according to claim 3, It is characterized in that The step of obtaining a second confidence level corresponding to the second state based on the confidence level of the state change result identified comprises: When the historical state of the target signal light in the historical image is a lighting state, the second confidence is obtained based on the confidence of the result of identifying the state change and the confidence of the historical state of the target signal light.
6. The method according to claim 5, It is characterized in that The step of obtaining the second confidence level based on the confidence level of the state change result and the confidence level of the historical state of the target signal light when the historical state of the target signal light in the historical image is a lighting state comprises: When the historical state of the target signal light is a lighting state, and the state change result is that the color of the target signal light does not change, the product of the confidence of identifying the state change result, the confidence of the historical state of the target signal light, and a second preset factor is determined as the second confidence; When the historical state of the target signal light is a lighting state and the state change result is a change in color of the target signal light, an average value of the confidence of identifying the state change result and the confidence of the historical state of the target signal light is determined as the second confidence.
7. The method according to claim 3, It is characterized in that The step of obtaining a second confidence level corresponding to the second state based on the confidence level of the state change result identified comprises: When the historical state of the target signal light in the historical image is an off state, the confidence level of the result of identifying the state change is determined as the second confidence level.
8. The method according to claim 3, It is characterized in that The method further comprises: When determining the first state as the current state, determining the confidence level of identifying the first state as the confidence level of the current state; When the second state is determined as the current state, the second confidence level is determined as the confidence level of the current state.
9. The method according to claim 1, It is characterized in that Before obtaining a first confidence corresponding to the first state based on the confidence of identifying the first state, the method further includes: determining whether a standard position of the target signal light exists; If so, performing a first correction on the confidence level for identifying the first state based on the degree of deviation between the first position and the standard position, and determining the confidence level after the first correction as the confidence level for identifying the first state; After obtaining a second confidence level corresponding to the second state based on the confidence level of the state change result being identified, the method further includes: determining whether a standard position of the target signal light exists; If so, a second correction is performed on the second confidence level based on the degree of deviation between the second position and the standard position, and the confidence level after the second correction is determined as the second confidence level.
10. The method according to claim 1, It is characterized in that The step of obtaining the current position of the target signal light in the image to be detected based on the first position includes: When there is a standard position of the target signal light, obtaining a current position of the target signal light in the image to be detected based on the first position and the standard position; When the standard position does not exist, determining the first position as the current position of the target signal light in the image to be detected; The step of obtaining the current position of the target signal light in the image to be detected based on the second position includes: When there is a standard position of the target signal light, obtaining a current position of the target signal light in the image to be detected based on the second position and the standard position; When the standard position does not exist, the second position is determined as the current position of the target signal light in the image to be detected.
11. The method according to claim 10, It is characterized in that The step of obtaining the current position of the target signal light in the image to be detected based on the first position and the standard position includes: If the degree of deviation between the first position and the standard position is less than a first degree threshold, the first position is determined as the current position of the target signal light in the image to be detected; otherwise, the standard position is determined as the current position of the target signal light in the image to be detected; The step of obtaining the current position of the target signal light in the image to be detected based on the second position and the standard position includes: If the degree of deviation between the second position and the standard position is less than a second degree threshold, the second position is determined as the current position of the target signal light in the image to be detected; otherwise, the standard position is determined as the current position of the target signal light in the image to be detected.
12. The method according to claim 11, It is characterized in that After determining the first position as the current position of the target signal light in the image to be detected if the degree of deviation between the first position and the standard position is less than a first degree threshold, the method further includes: updating the standard position based on the first position; After determining the second position as the current position of the target signal light in the image to be detected if the degree of deviation between the second position and the standard position is less than a second degree threshold, the method further includes: updating the standard position based on the second position.
13. The method according to claim 9 or 10, It is characterized in that When the standard position does not exist, after obtaining the current state of the target signal light in the image to be detected, the method further includes: Based on the historical position of the target signal light and the current position of the target signal light in the image to be detected, the standard position of the target signal light is obtained to serve as the standard position of the target signal light in the next detection process.
14. The method according to claim 13, It is characterized in that Before obtaining the standard position of the target signal light based on the historical position of the target signal light and the current position of the target signal light in the image to be detected, the method further includes: Determining whether the number of historical positions of the target signal light reaches a number threshold; If so, the step of obtaining the standard position of the target signal light based on the historical position of the target signal light and the current position of the target signal light in the image to be detected is performed.
15. The method according to claim 1, It is characterized in that The step of identifying the image to be detected and obtaining the first state of the target signal light in the image to be detected includes: Sending the image to be detected into a pre-trained state model for recognition to obtain the first state of the target signal light in the image to be detected; And / or, the step of identifying the state change result of the target signal light from the historical image to the image to be detected based on the historical image and the image to be detected includes: The historical image and the image to be detected are sent to a pre-trained conversion model for recognition, and the state change result of the target traffic light from the historical image to the image to be detected is obtained.
16. A detection device for a signal light, It is characterized in that The signal light detection device includes a processor, a memory and a communication circuit, the processor is coupled to the memory and the communication circuit respectively, the memory stores program data, and the processor implements the steps in the method as described in any one of claims 1 to 15 by executing the program data in the memory.
17. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and the computer program can be executed by a processor to implement the steps in the method according to any one of claims 1 to 15.
