Traffic light color and countdown determination method, device, equipment, medium and product

Abstract

The invention provides a traffic light color and countdown determination method, device and equipment, a medium and a product, which can be used in the field of intelligent traffic. In the method, after image features and time features are determined according to an obtained current frame of traffic light image and a shooting moment of the current frame of traffic light image, the features and the features are input into a classification model, and a light color corresponding to the frame of traffic light image is obtained. And determining countdown of the light color corresponding to the current frame of traffic light image according to the shooting time, the light color corresponding to the current frame of traffic light image and whether the light color corresponding to the previous frame of traffic light image of the current frame of traffic light image is obtained. According to the scheme, the light color is determined according to the image features and the time features, and the countdown is determined according to whether the light color corresponding to the previous frame of traffic light image is acquired, so that the light color and the countdown of the traffic light are determined in a scene in which data in the signal machine cannot be acquired, and the accuracy of the light color and the countdown is also improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation, and in particular to a method, device, equipment, medium and product for determining the color and countdown of a traffic light. Background Art

[0002] With the development of science and technology, autonomous driving technology has emerged, improving vehicle safety. In autonomous driving technology, how to obtain traffic signal data at each intersection is crucial. Traffic signal data includes the light color and countdown of traffic lights.

[0003] In the prior art, the method for determining the light color and countdown of a traffic light is usually to obtain and analyze the data from the traffic light signal. In the scenario where the data from the traffic light signal cannot be obtained, the light color and countdown of the traffic light cannot be determined.

[0004] Therefore, there is an urgent need for a method for determining the color and countdown of a traffic light, which can determine the color and countdown of a traffic light in a scenario where data from a traffic light cannot be obtained. Summary of the Invention

[0005] The embodiments of the present application provide a method, device, equipment, medium and product for determining the color and countdown of a traffic light, which are used to solve the problem in the prior art that the color and countdown of a traffic light cannot be determined in a scenario where data from a traffic light cannot be obtained.

[0006] In a first aspect, an embodiment of the present application provides a method for determining the color and countdown of a traffic light, including:

[0007] Determining image features and time features based on the acquired current frame traffic light image and the shooting time of the current frame traffic light image;

[0008] Inputting the image features and the time features into a classification model to obtain the light color corresponding to the traffic light image of the current frame; the classification model is a pre-trained machine learning model that determines the light color based on the image features and time features;

[0009] The countdown of the light color corresponding to the current frame of traffic light image is determined according to the shooting time, the light color corresponding to the current frame of traffic light image, and whether the light color corresponding to the previous frame of traffic light image of the current frame of traffic light image is obtained.

[0010] In a specific embodiment, determining the image features and time features based on the acquired current frame traffic light image and the shooting time of the current frame traffic light image includes:

[0011] Using a target detection model to detect the position of the traffic light in the current frame of the traffic light image to determine the position of the traffic light; the target detection model is a pre-trained neural network model that determines the position of the traffic light based on the traffic light image;

[0012] Determining the image feature according to the current frame traffic light image and the traffic light position;

[0013] The time feature is determined according to the shooting moment.

[0014] In a specific embodiment, the image feature includes a first image feature and a second image feature, and determining the image feature based on the current frame traffic light image and the traffic light position includes:

[0015] Cutting out a traffic light area image from the current frame traffic light image according to the traffic light position;

[0016] Determining, based on the traffic light area image, the first image features for characterizing red, green, blue, hue, saturation, lightness, red-green ratio, and green-blue ratio;

[0017] The second image feature for characterizing the saturation difference and the brightness difference between the traffic light area and the overall image is determined based on the current frame traffic light image and the traffic light area image.

[0018] In a specific embodiment, the time features include features for characterizing month, week, and hour periods.

[0019] In a specific embodiment, determining the countdown for the light color corresponding to the current traffic light image frame based on the shooting time, the light color corresponding to the current traffic light image frame, and whether the light color corresponding to the previous traffic light image frame before the current traffic light image frame is obtained includes:

[0020] If a light color corresponding to a previous traffic light image frame of the current traffic light image frame is obtained, then determining the countdown time based on whether the light colors corresponding to the current traffic light image frame and the previous traffic light image frame are the same according to the shooting time;

[0021] If the light color corresponding to the previous frame of traffic light image of the current frame of traffic light image is not obtained, the start time of the light color corresponding to the stored current frame of traffic light image is updated according to the shooting time, and the countdown is determined according to the updated start time, the light color cycle of the light color corresponding to the stored current frame of traffic light image, and the shooting time.

[0022] In a specific embodiment, determining the countdown based on whether the light colors corresponding to the current frame of the traffic light image and the previous frame of the traffic light image are the same according to the shooting time includes:

[0023] If the light color corresponding to the current frame of traffic light image is the same as that corresponding to the previous frame of traffic light image, obtaining the start time and light color cycle of the light color corresponding to the stored current frame of traffic light image;

[0024] The countdown is determined according to the start time, the light color cycle and the shooting time.

[0025] In a specific embodiment, the method further comprises:

[0026] If the light color corresponding to the current frame of traffic light image is different from that corresponding to the previous frame of traffic light image, updating the start time of the light color corresponding to the stored current frame of traffic light image according to the shooting time;

[0027] The countdown is determined according to the updated start time, the light color cycle of the light color corresponding to the stored current frame traffic light image, and the shooting time.

[0028] In a specific embodiment, the method further comprises:

[0029] If the light color corresponding to the traffic light image of the current frame is green, updating the stored light color cycle corresponding to red according to the shooting time and the stored start time corresponding to red;

[0030] If the light color corresponding to the traffic light image of the current frame is yellow, the stored light color cycle corresponding to green is updated according to the shooting time and the stored start time corresponding to green.

[0031] In a second aspect, an embodiment of the present application provides a device for determining a green light color and a countdown, comprising:

[0032] a processing module, configured to determine image features and time features based on the acquired current frame traffic light image and the shooting time of the current frame traffic light image;

[0033] a classification module, configured to input the image features and the time features into a classification model to obtain the light color corresponding to the traffic light image in the current frame; the classification model is a pre-trained machine learning model that determines the light color based on the image features and the time features;

[0034] The processing module is also used to determine the countdown of the light color corresponding to the current frame of traffic light image based on the shooting time, the light color corresponding to the current frame of traffic light image, and whether the light color corresponding to the previous frame of traffic light image of the current frame of traffic light image is obtained.

[0035] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0036] Processor, memory, communication interface;

[0037] The memory is used to store executable instructions of the processor;

[0038] The processor is configured to execute the traffic light color and countdown determination method described in any one of the first aspects by executing the executable instructions.

[0039] In a fourth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for determining the color and countdown of a traffic light as described in any one of the first aspects.

[0040] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising a computer program, which, when executed by a processor, is used to implement the method for determining the color and countdown of a traffic light as described in any one of the first aspects.

[0041] The methods, devices, equipment, media and products for determining the color and countdown of traffic lights provided in the embodiments of the present application determine the image features and time features based on the acquired current frame traffic light image and the shooting time of the current frame traffic light image, and then input the features and the features into a classification model to obtain the light color corresponding to the frame traffic light image. Then, based on the shooting time, the light color corresponding to the current frame traffic light image, and whether the light color corresponding to the previous frame traffic light image of the current frame traffic light image is obtained, the countdown of the light color corresponding to the current frame traffic light image is determined. This solution determines the light color based on image features and time features, and determines the countdown based on the determined light color, the shooting time, and whether the light color corresponding to the previous frame traffic light image is obtained. This not only realizes the determination of the light color and countdown of traffic lights in scenarios where data from the traffic light cannot be obtained, but also improves the accuracy of the determined light color and countdown. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0043] Figure 1 Schematic diagram of the application scenario of the method for determining traffic light color and countdown provided in this application;

[0044] Figure 2A flow chart of Example 1 of the method for determining the color and countdown of a traffic light provided in this application;

[0045] Figure 3a A flow chart of Example 2 of the method for determining the color and countdown of a traffic light provided in this application;

[0046] Figure 3b A flow chart of the method for determining the traffic light color and countdown provided in this application;

[0047] Figure 4 A flowchart of Example 3 of the method for determining the color and countdown of a traffic light provided in this application;

[0048] Figure 5 This is a schematic diagram of the structure of an embodiment of the device for determining the green light color and countdown provided by this application;

[0049] Figure 6 This is a schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0050] To make the purpose, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments made by ordinary technicians in this field based on the inspiration of these embodiments fall within the scope of protection of this application.

[0051] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the numbers used in this way are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0052] With the development of science and technology, autonomous driving technology has emerged, improving vehicle driving safety. In autonomous driving technology, it is necessary to obtain the color and countdown of traffic lights so that vehicles can drive according to the color and countdown.

[0053] In existing technologies, traffic light color and countdown are typically determined by parsing data from traffic light indicators. If traffic light data is unavailable, the color and countdown of the traffic light cannot be determined. Therefore, a method for determining the color and countdown of a traffic light is urgently needed, capable of determining the color and countdown of a traffic light even in situations where traffic light data is unavailable.

[0054] In response to the problems existing in the prior art, the inventors discovered during their research on methods for determining traffic light color and countdown that, in order to determine the color and countdown of traffic lights in scenarios where data from traffic lights cannot be obtained, cameras can be installed at locations such as streetlight poles and road sign poles, and the cameras can capture images including traffic lights. Based on the current frame of traffic light image captured by the camera and the shooting time, image features and time features can be determined and input into a classification model to obtain the light color. The countdown can then be determined based on the shooting time, the light color corresponding to the current frame of traffic light image, and whether the light color corresponding to the previous frame of traffic light image was obtained. Based on the above-mentioned inventive concept, the traffic light color and countdown determination scheme of this application was designed.

[0055] In this application, the execution entity of the method for determining the color and countdown of a traffic light can be a server, or a camera, a computer, etc. This application does not limit it. The following description will be made using a server as an example.

[0056] For example, Figure 1 Schematic diagram of the application scenario of the method for determining the color and countdown of traffic lights provided in this application, such as Figure 1 As shown, the application scenario may include: a server 11, a camera 12 and an autonomous driving vehicle 13.

[0057] exist Figure 1 In the application scenario shown, the camera 12 is installed on a street light pole and can capture images of traffic lights.

[0058] Each time the camera 12 captures a frame of traffic light image, it sends the captured current frame of traffic light image and the capturing time to the server 11 .

[0059] The server 11 determines the image features and time features based on the acquired current frame traffic light image and the shooting time; and then inputs the image features and time features into the classification model to obtain the light color corresponding to the current frame traffic light image.

[0060] The server 11 determines the countdown of the light color corresponding to the current frame of traffic light image according to the shooting time, the light color corresponding to the current frame of traffic light image, and whether the light color corresponding to the previous frame of traffic light image before the current frame of traffic light image is obtained.

[0061] When the autonomous driving vehicle 13 drives to the location of the traffic light, the server 11 sends the determined traffic light color and countdown to the autonomous driving vehicle 13, and the autonomous driving vehicle 13 drives according to the light color and countdown.

[0062] It should be noted that Figure 1 This is only a schematic diagram of an application scenario provided by the embodiment of the present application. Figure 1 The actual form of the various devices included in the Figure 1 The interaction mode between devices is limited, and in the specific application of the solution, it can be set according to actual needs.

[0063] The technical solution of the present application is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0064] Figure 2 This is a flow chart of the first embodiment of the method for determining the color and countdown of a traffic light provided by this application. This embodiment of the application describes how the server determines the color and countdown corresponding to the current frame of traffic light image based on the current frame of traffic light image, the shooting time of the current frame of traffic light image, and whether the color of the previous frame of traffic light image before the current frame of traffic light image is obtained. The method in this embodiment can be implemented by software, hardware, or a combination of software and hardware. Figure 2 As shown, the method for determining the traffic light color and countdown specifically includes the following steps:

[0065] S201: Determine image features and time features based on the acquired current frame traffic light image and the shooting time of the current frame traffic light image.

[0066] The camera is installed at locations such as street light poles and road sign poles to capture images including traffic lights. Each frame of traffic light image captured will record the corresponding shooting time. The camera will send the traffic light image and shooting time to the server.

[0067] In this step, after the server obtains the current frame traffic light image and the shooting time of the current frame traffic light image sent by the camera, it determines the image features and time features according to the current frame traffic light image and the shooting time.

[0068] S202: Input the image features and time features into the classification model to obtain the light color corresponding to the traffic light image of the current frame.

[0069] In this step, the server determines the image and time features and then feeds them into a classification model to determine the light color corresponding to the current frame of the traffic light image. The classification model is a pre-trained machine learning model that determines the light color based on image and time features.

[0070] It should be noted that the classification model can be a support vector machine (SVM) model, a convolutional neural network (CNN) model, a K-nearest neighbor (KNN) model, etc. The embodiment of the present application does not limit the classification model and can be determined according to actual conditions.

[0071] S203: Determine the countdown of the light color corresponding to the current frame of traffic light image according to the shooting time, the light color corresponding to the current frame of traffic light image, and whether the light color corresponding to the previous frame of traffic light image before the current frame of traffic light image is obtained.

[0072] In this step, after the server obtains the light color corresponding to the current frame of traffic light image, it determines the countdown of the light color corresponding to the current frame of traffic light image based on the shooting time, the light color corresponding to the current frame of traffic light image, and whether the light color corresponding to the previous frame of traffic light image of the current frame of traffic light image is obtained.

[0073] Specifically, if the light color corresponding to the previous frame of traffic light image of the current frame of traffic light image is obtained, it means that the current frame of traffic light image is not the first frame of image. Then, based on the shooting time, whether the light color corresponding to the current frame of traffic light image is the same as that of the previous frame of traffic light image, the countdown is determined.

[0074] If the light color corresponding to the current traffic light image is the same as that corresponding to the previous traffic light image, indicating that the light color has not changed, the start time and light color cycle corresponding to the stored traffic light image of the current traffic light frame are obtained. Since the server stores the start time and light color cycle corresponding to each light color, the start time and light color cycle corresponding to the current traffic light image can be obtained.

[0075] Then, the countdown is determined based on the start time, light color cycle and shooting time. That is, the shooting time is subtracted from the start time to obtain the duration of the light color, and then the light color cycle is subtracted from the duration of the light color to obtain the countdown of the light color corresponding to the current frame of the traffic light image.

[0076] If the light color corresponding to the current frame of traffic light image is different from that of the previous frame of traffic light image, it means that the light color has changed. Then, according to the shooting time, the starting time of the light color corresponding to the stored current frame of traffic light image is updated; that is, the starting time of the light color corresponding to the current frame of traffic light image is updated to the shooting time.

[0077] The countdown is then determined based on the updated start time, the color cycle of the light color corresponding to the current frame of the traffic light image, and the capture time. Specifically, the updated start time is subtracted from the capture time to obtain the duration of the light color. The color cycle is then subtracted from the duration of the light color to obtain the countdown of the light color corresponding to the current frame of the traffic light image.

[0078] It should be noted that if the light color corresponding to the current frame traffic light image is different from that of the previous frame traffic light image, in order to more accurately determine the light color and countdown later, the light color cycle of other light colors needs to be updated according to the light color corresponding to the current frame traffic light image.

[0079] If the light color corresponding to the current frame traffic light image is green, update the stored light color cycle corresponding to red according to the shooting time and the stored start time corresponding to red; that is, update the light color cycle corresponding to red to the difference between the shooting time and the start time corresponding to red.

[0080] If the light color corresponding to the current frame traffic light image is yellow, update the stored light color cycle corresponding to green according to the shooting time and the stored start time corresponding to green; that is, update the light color cycle corresponding to green to the difference between the shooting time and the start time corresponding to green.

[0081] If the light color corresponding to the traffic light image of the current frame is red, the yellow light color cycle will not be updated because the yellow light color cycle will not change and is set to 3 seconds.

[0082] If the light color corresponding to the previous frame of traffic light image of the current frame of traffic light image is not obtained, it means that the current frame of traffic light image is the first frame of image. When this scheme is executed for the first time, the start time of the light color corresponding to the stored current frame of traffic light image is updated according to the shooting time, that is, the start time of the light color corresponding to the current frame of traffic light image is updated to the shooting time.

[0083] The countdown is then determined based on the updated start time, the color cycle of the light color corresponding to the current frame of the traffic light image, and the capture time. Specifically, the updated start time is subtracted from the capture time to obtain the duration of the light color. The duration of the light color is then subtracted from the color cycle to obtain the countdown of the light color corresponding to the current frame of the traffic light image.

[0084] It should be noted that if the light color corresponding to the previous frame of traffic light image of the current frame of traffic light image is not obtained, in order to more accurately determine the light color and countdown later, it is also necessary to update the light color cycle of other light colors according to the light color corresponding to the current frame of traffic light image.

[0085] If the light color corresponding to the current frame traffic light image is green, update the stored light color cycle corresponding to red according to the shooting time and the stored start time corresponding to red; that is, update the light color cycle corresponding to red to the difference between the shooting time and the start time corresponding to red.

[0086] If the light color corresponding to the current frame traffic light image is yellow, update the stored light color cycle corresponding to green according to the shooting time and the stored start time corresponding to green; that is, update the light color cycle corresponding to green to the difference between the shooting time and the start time corresponding to green.

[0087] If the light color corresponding to the traffic light image of the current frame is red, the yellow light color cycle will not be updated because the yellow light color cycle will not change and is set to 3 seconds.

[0088] It should be noted that when this solution is first executed, the stored start time and light color cycle corresponding to each light color are the default values. For example, the start times corresponding to red, green, and yellow are 00:00:00 on January 1, 2024, 12:00:00 on February 1, 2024, and 16:30:10 on March 1, 2024. The light color cycles corresponding to red and green are 30 seconds, 60 seconds, 120 seconds, 180 seconds, and so on. The light color cycle corresponding to yellow is 3 seconds. This embodiment of the application does not limit the default values ​​of the start time and light color cycle corresponding to each light color, and can be determined according to actual conditions.

[0089] The method for determining the traffic light color and countdown provided in this embodiment determines image features and time features based on the current frame of traffic light image and the shooting time of the current frame of traffic light image, and then inputs the features and time features into a classification model to obtain the light color corresponding to the frame of traffic light image. The countdown of the light color corresponding to the current frame of traffic light image is then determined based on the shooting time, the light color corresponding to the current frame of traffic light image, and whether the light color corresponding to the previous frame of traffic light image before the current frame of traffic light image was obtained. This solution determines the light color based on image features and time features, and determines the countdown based on the determined light color, the shooting time, and whether the light color corresponding to the previous frame of traffic light image was obtained. This not only allows the determination of the traffic light color and countdown in scenarios where data from the traffic light cannot be obtained, but also improves the accuracy of the determined light color and countdown.

[0090] Figure 3aThis is a flow chart of the second embodiment of the method for determining the color and countdown of a traffic light provided by this application. Based on the above embodiment, this embodiment of the application describes how the server determines the image features and time features based on the current frame traffic light image and the shooting time. Figure 3a As shown, the method for determining the traffic light color and countdown specifically includes the following steps:

[0091] S301: Use the target detection model to detect the position of the traffic light in the current frame of the traffic light image to determine the position of the traffic light.

[0092] In this step, after the server obtains the current frame of traffic light image, it uses the object detection model to detect the traffic light position in the current frame of traffic light image to determine the traffic light position. The traffic light position refers to the location area of ​​the traffic light in the current frame of traffic light image.

[0093] It should be noted that the target detection model is a pre-trained neural network model that determines the location of traffic lights based on traffic light images. The target detection model can be a YOLO (You Only Look Once) model, a Region-Convolutional Neural Network (R-CNN) model, a Feature Pyramid Network (FPN) model, etc. This embodiment of the application does not limit the target detection model and can be determined based on actual conditions.

[0094] It should be noted that the representation method of the traffic light position can be the coordinates of the upper left corner and the lower right corner of a detection frame, or the coordinates, width and length of the center point of a detection frame, or the coordinates of the four corner points of a detection frame. The embodiment of this application does not take into account the representation method of the green light position, and it can be determined according to actual conditions.

[0095] S302: Determine image features based on the traffic light image and traffic light position in the current frame.

[0096] In this step, after determining the location of the traffic light, the server determines image features based on the current frame of the traffic light image and the location of the traffic light. The image features include a first image feature and a second image feature.

[0097] Specifically, according to the position of the traffic light, a traffic light area image is cut out from the traffic light image of the current frame.

[0098] It should be noted that Gaussian filtering can also be performed on the traffic light area image to reduce image noise and improve the accuracy of determining the light color.

[0099] According to the traffic light area image, first image features for characterizing red, green, blue, hue, saturation, brightness, red-green ratio, and green-blue ratio are determined.

[0100] The first image feature includes the first feature to the fourteenth feature. The first feature is the average value of the red component of each pixel in the traffic light area image, the second feature is the average value of the green component of each pixel in the traffic light area image, the third feature is the average value of the blue component of each pixel in the traffic light area image, the fourth feature is the standard deviation of the red component of each pixel in the traffic light area image, the fifth feature is the standard deviation of the green component of each pixel in the traffic light area image, and the sixth feature is the standard deviation of the blue component of each pixel in the traffic light area image.

[0101] The seventh feature is the average value of the hue of each pixel in the traffic light area image, the eighth feature is the average value of the saturation of each pixel in the traffic light area image, the ninth feature is the average value of the brightness of each pixel in the traffic light area image, the tenth feature is the standard deviation of the hue of each pixel in the traffic light area image, the eleventh feature is the standard deviation of the saturation of each pixel in the traffic light area image, and the twelfth feature is the standard deviation of the brightness of each pixel in the traffic light area image.

[0102] The thirteenth feature is the ratio of the first feature to the second feature. In order to ensure that the denominator is not 0 in the process of calculating the thirteenth feature, the second feature can be added with a preset value. The preset value can be 1e -7 、1e -6 、1e -5 wait.

[0103] The fourteenth feature is the ratio of the second feature to the third feature. In order to ensure that the denominator is not 0 in the process of calculating the fourteenth feature, the third feature can be added with a preset value, which can be 1e -7 、1e -6 、1e -5 wait.

[0104] According to the traffic light image of the current frame and the traffic light area image, a second image feature for characterizing the saturation difference and the brightness difference between the traffic light area and the overall image is determined.

[0105] The second image feature includes a fifteenth feature and a sixteenth feature.

[0106] The fifteenth feature is the difference between the ninth feature and the average value of the brightness of each pixel in the traffic light image of the current frame.

[0107] The sixteenth feature is the difference between the eighth feature and the average value of the saturation of each pixel in the traffic light image of the current frame.

[0108] It should be noted that if the current frame traffic light image or traffic light area image is a red-green-blue (RGB) encoded image, it is converted into a hue-saturation-value (HSV) encoded image, and the image features are determined based on the two encoded images. If the current frame traffic light image or traffic light area image is an HSV encoded image, it is converted into an RGB encoded image, and the image features are determined based on the two encoded images. If the current frame traffic light image or traffic light area image is not an RGB or HSV encoded image, it is converted into an RGB or HSV encoded image, and the image features are determined based on the two encoded images. Since HSV-encoded images have more obvious color distinctions, the accuracy and robustness of the light color can be improved by using the features related to hue, saturation, and brightness.

[0109] S303: Determine the time feature according to the shooting time.

[0110] In this step, after obtaining the shooting time, the server determines the time feature according to the shooting time.

[0111] The time features include features for representing the month, week, and hour periods. The time features include the seventeenth feature to the twenty-first feature.

[0112] The seventeenth feature is the encoded value of the month at the time of capture, the eighteenth feature is the encoded value of the day of the week at the time of capture, and the nineteenth feature is the encoded value of the hour at the time of capture. The encoding method for the month, day of the week, and hour can be one-hot encoding, label encoding, binary encoding, etc.

[0113] For example, if the day of the week at the time of capture is Tuesday, a 7-dimensional vector is generated, with the second element of this vector set to 1 and the remaining elements set to 0, resulting in the eighteenth feature being [0, 1, 0, 0, 0, 0, 0]. If the hour of the hour at the time of capture is the 13th hour, a 24-dimensional vector is generated, with the 13th element of this vector set to 1 and the remaining elements set to 0, resulting in the nineteenth feature being [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0].

[0114] The embodiment of the present application does not limit the encoding method of the month, week, and hour, which can be determined according to actual conditions.

[0115] The twentieth feature is the cosine value of the hourly period value, and the twenty-first feature is the sine value of the hourly period value.

[0116] The hourly period value can be expressed using the formula: time_of_day = (hour + minute / 60) / 24 * 2π, where time_of_day represents the hourly period value, hour represents the hourly period of the shooting time, and minute represents the number of minutes in the hourly period of the shooting time.

[0117] In order to more accurately capture the periodic features using the 20th and 21st features, the time period is encoded as sine and cosine values, that is, the discrete variables are converted into continuous periodic variables, which adapts to the periodic changes of variables such as light over time.

[0118] Since traffic lights have different color values ​​under different weather, lighting and other conditions, and weather, lighting and time are related, using time characteristics to determine the light color can improve the accuracy of the light color.

[0119] It should be noted that the execution order of steps S301-step S302 and step S303 can be: first execute steps S301-step S302, then execute step S303. Alternatively, it can be: first execute step S303, then execute steps S301-step S302. Alternatively, it can be: steps S301-step S302 and step S303 are executed simultaneously. This embodiment of the application does not limit the execution order of steps S301-step S302 and step S303, and can be determined based on actual circumstances.

[0120] For example, Figure 3b A flow chart of the method for determining the color and countdown of a traffic light provided in this application is shown as follows: Figure 3b As shown, a target detection model is used to detect the traffic light position in the current traffic light image. After determining the traffic light position, a traffic light area image is cut out from the current traffic light image based on the traffic light position. Image features are determined based on the current traffic light image and the traffic light area image, and time features are determined based on the capture time. The image features and time features are then input into the classification model to obtain the light color corresponding to the current traffic light image. Based on the light color corresponding to the current traffic light image and whether the light color corresponding to the previous traffic light image is obtained, the start time and light color cycle of the light color corresponding to the current traffic light image are determined. Furthermore, the countdown of the light color corresponding to the current traffic light image is determined based on the capture time.

[0121] The method for determining traffic light color and countdown provided in this embodiment first determines the location of the traffic light and then determines the image features. This allows accurate image features to be obtained even when the camera vibrates due to external forces, improving the accuracy of the light color and facilitating maintenance and upgrades. By determining time features that mark the month, day of the week, and hour, the accuracy of the time features is improved. By using both image and time features to determine the light color, the accuracy of the light color can be improved, adapting to different lighting and weather conditions.

[0122] Figure 4 This is a flow chart of the third embodiment of the method for determining the color and countdown of a traffic light provided by this application. Based on the above embodiment, this embodiment of the application illustrates the situation where the server trains the initial machine learning model to obtain a classification model. Figure 4 As shown, the method for determining the traffic light color and countdown specifically includes the following steps:

[0123] S401: Acquire multiple sets of training data.

[0124] In this step, in order to train a classification model, multiple sets of training data are first obtained, each set of training data includes image features and time features.

[0125] S402: Select a set of target training data from multiple sets of training data.

[0126] S403: Input the target training data into the initial machine learning model to obtain the predicted light color.

[0127] In the above steps, after the server obtains multiple sets of training data, it selects a set of target training data from the multiple sets of training data, and then inputs the target training data into the initial machine learning model to obtain the predicted light color.

[0128] S404: Update the initial machine learning model according to the predicted light color to obtain a trained machine learning model.

[0129] In this step, after the server obtains the predicted light color, it calculates the loss value based on the predicted light color and the target light color corresponding to the target training data, updates the initial machine learning model based on the loss value, and obtains the trained machine learning model.

[0130] S405: Update the number of training times.

[0131] In this step, after the server updates the initial machine learning model, it also needs to update the number of training times, that is, increase the number of training times by one.

[0132] S406: Determine whether the updated number of training times is equal to the preset number; if the updated number of training times is equal to the preset number, execute step S407; if the updated number of training times is less than the preset number, execute step S408.

[0133] In this step, after the server updates the number of training times, in order to determine whether it is necessary to continue model training, it is determined whether the updated number of training times is equal to the preset number.

[0134] It should be noted that the number of training times can be 500, 1000, 10000, 500000, etc. The embodiment of the present application does not limit the number of training times and can be determined according to actual conditions.

[0135] S407: Using the trained machine learning model as a classification model.

[0136] In this step, if the server determines that the updated number of training times is equal to the preset number of times, it means that there is no need to continue model training, and the trained machine learning model is used as the classification model.

[0137] S408: Use the trained machine learning model as a new initial machine learning model and execute step S402.

[0138] In this step, if the server determines that the updated training times are less than the preset times, it means that the model training needs to be continued, and the trained machine learning model is used as the new initial machine learning model to execute step S402.

[0139] That is, the trained machine learning model is used as the new initial machine learning model. A set of target training data is reselected from multiple sets of training data, and the target training data is input into the new initial machine learning model to obtain a new predicted light color. The new initial machine learning model is updated according to the predicted light color to obtain a new trained machine learning model. The number of training times is updated again. If the updated number of training times is less than the preset number, this process is repeated until the updated number of training times equals the preset number, and the trained machine learning model is used as the classification model.

[0140] The method for determining the color and countdown of a traffic light provided in this embodiment trains an initial machine learning model using training data including image features and time features to obtain a classification model, thereby effectively improving the accuracy of the classification model in obtaining the light color.

[0141] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0142] Figure 5 This is a structural diagram of an embodiment of the device for determining the green light color and countdown provided in this application.

[0143] like Figure 5 As shown, the green light color and countdown determining device 50 includes:

[0144] The processing module 51 is used to determine the image characteristics and time characteristics based on the acquired current frame traffic light image and the shooting time of the current frame traffic light image;

[0145] A classification module 52 is configured to input the image features and the time features into a classification model to obtain the light color corresponding to the traffic light image in the current frame; the classification model is a pre-trained machine learning model that determines the light color based on the image features and the time features;

[0146] The processing module 51 is further used to determine the countdown of the light color corresponding to the current frame of traffic light image based on the shooting time, the light color corresponding to the current frame of traffic light image, and whether the light color corresponding to the previous frame of traffic light image of the current frame of traffic light image is obtained.

[0147] Furthermore, the processing module 51 is specifically configured to:

[0148] Using a target detection model to detect the position of the traffic light in the current frame of the traffic light image to determine the position of the traffic light; the target detection model is a pre-trained neural network model that determines the position of the traffic light based on the traffic light image;

[0149] Determining the image feature according to the current frame traffic light image and the traffic light position;

[0150] The time feature is determined according to the shooting moment.

[0151] Furthermore, the image feature includes a first image feature and a second image feature, and the processing module 51 is specifically configured to:

[0152] Cutting out a traffic light area image from the current frame traffic light image according to the traffic light position;

[0153] Determining, based on the traffic light area image, the first image features for characterizing red, green, blue, hue, saturation, lightness, red-green ratio, and green-blue ratio;

[0154] The second image feature for characterizing the saturation difference and the brightness difference between the traffic light area and the overall image is determined based on the current frame traffic light image and the traffic light area image.

[0155] Furthermore, the time features include features for characterizing month, week, and hour periods.

[0156] Furthermore, the processing module 51 is specifically configured to:

[0157] If a light color corresponding to a previous traffic light image frame of the current traffic light image frame is obtained, then determining the countdown time based on whether the light colors corresponding to the current traffic light image frame and the previous traffic light image frame are the same according to the shooting time;

[0158] If the light color corresponding to the previous frame of traffic light image of the current frame of traffic light image is not obtained, the start time of the light color corresponding to the stored current frame of traffic light image is updated according to the shooting time, and the countdown is determined according to the updated start time, the light color cycle of the light color corresponding to the stored current frame of traffic light image, and the shooting time.

[0159] Furthermore, the processing module 51 is specifically configured to:

[0160] If the light color corresponding to the current frame of traffic light image is the same as that corresponding to the previous frame of traffic light image, obtaining the start time and light color cycle of the light color corresponding to the stored current frame of traffic light image;

[0161] The countdown is determined according to the start time, the light color cycle and the shooting time.

[0162] Furthermore, the processing module 51 is specifically configured to:

[0163] If the light color corresponding to the current frame of traffic light image is different from that corresponding to the previous frame of traffic light image, updating the start time of the light color corresponding to the stored current frame of traffic light image according to the shooting time;

[0164] The countdown is determined according to the updated start time, the light color cycle of the light color corresponding to the stored current frame traffic light image, and the shooting time.

[0165] Furthermore, the processing module 51 is specifically configured to:

[0166] If the light color corresponding to the traffic light image of the current frame is green, updating the stored light color cycle corresponding to red according to the shooting time and the stored start time corresponding to red;

[0167] If the light color corresponding to the traffic light image of the current frame is yellow, the stored light color cycle corresponding to green is updated according to the shooting time and the stored start time corresponding to green.

[0168] The device for determining the green light color and countdown provided in this embodiment is used to execute the technical solution in any of the aforementioned method embodiments. Its implementation principles and technical effects are similar and will not be repeated here.

[0169] Figure 6 This is a schematic diagram of the structure of an electronic device provided by this application. Figure 6As shown, the electronic device 60 includes:

[0170] Processor 61, memory 62, and communication interface 63;

[0171] The memory 62 is used to store executable instructions of the processor 61;

[0172] The processor 61 is configured to execute the technical solution in any of the aforementioned method embodiments by executing the executable instructions.

[0173] Optionally, the memory 62 can be independent or integrated with the processor 61.

[0174] Optionally, when the memory 62 is a device independent of the processor 61, the electronic device 60 may further include:

[0175] The bus 64 , the memory 62 and the communication interface 63 are connected to the processor 61 via the bus 64 and communicate with each other. The communication interface 63 is used to communicate with other devices.

[0176] Optionally, the communication interface 63 may be implemented using a transceiver. The communication interface is used to enable communication between the database access device and other devices (e.g., a client, a read-write library, and a read-only library). The memory may include random access memory (RAM) and may also include non-volatile memory (non-volatile memory), such as at least one disk storage device.

[0177] Bus 64 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the figure shows only one thick line, but this does not imply that there is only one bus or only one type of bus.

[0178] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0179] The electronic device is used to execute the technical solution in any of the aforementioned method embodiments, and its implementation principles and technical effects are similar and will not be repeated here.

[0180] An embodiment of the present application further provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the technical solution provided by any of the aforementioned method embodiments.

[0181] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the technical solution provided by any of the aforementioned method embodiments.

[0182] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0183] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for determining the color and countdown of a traffic light, characterized in that: include: Determine image features and time features according to the acquired current frame traffic light image and the shooting time of the current frame traffic light image; Input the image features and the time features into a classification model to obtain the light color corresponding to the traffic light image of the current frame; the classification model is a pre-trained machine learning model that determines the light color based on the image features and the time features; The countdown of the light color corresponding to the current frame of traffic light image is determined according to the shooting time, the light color corresponding to the current frame of traffic light image, and whether the light color corresponding to the previous frame of traffic light image of the current frame of traffic light image is obtained.

2. The method according to claim 1, characterized in that The determining of image features and time features according to the acquired current frame traffic light image and the shooting time of the current frame traffic light image includes: Using a target detection model to detect the position of a traffic light on the current frame traffic light image to determine the position of the traffic light; the target detection model is a pre-trained neural network model for determining the position of a traffic light based on the traffic light image; Determining the image feature according to the current frame traffic light image and the traffic light position; The time feature is determined according to the shooting moment.

3. The method according to claim 2, characterized in that The image feature includes a first image feature and a second image feature, and determining the image feature according to the current frame traffic light image and the traffic light position includes: According to the traffic light position, cutting out a traffic light area image from the current frame traffic light image; Determine, according to the traffic light area image, the first image feature for characterizing red, green, blue, hue, saturation, brightness, red-green ratio, and green-blue ratio; According to the current frame traffic light image and the traffic light area image, the second image feature for characterizing the saturation difference and the brightness difference between the traffic light area and the overall image is determined.

4. The method according to claim 2, characterized in that: The time features include features for characterizing month, week, and hour periods.

5. The method according to claim 1, characterized in that The step of determining a countdown of the light color corresponding to the current frame of traffic light image according to the shooting time, the light color corresponding to the current frame of traffic light image, and whether the light color corresponding to the previous frame of traffic light image of the current frame of traffic light image is obtained includes: If the light color corresponding to the previous frame of traffic light image of the current frame of traffic light image is obtained, then according to the shooting time, whether the light colors corresponding to the current frame of traffic light image and the previous frame of traffic light image are the same, the countdown is determined; If the light color corresponding to the previous frame of traffic light image of the current frame of traffic light image is not obtained, the start time of the light color corresponding to the stored current frame of traffic light image is updated according to the shooting time, and the countdown is determined according to the updated start time, the light color cycle of the light color corresponding to the stored current frame of traffic light image, and the shooting time.

6. The method according to claim 5, characterized in that The step of determining the countdown based on whether the light colors corresponding to the current frame of the traffic light image and the previous frame of the traffic light image are the same according to the shooting time includes: If the light color corresponding to the current frame of traffic light image is the same as that corresponding to the previous frame of traffic light image, then obtaining the start time and light color cycle of the light color corresponding to the stored current frame of traffic light image; The countdown is determined according to the start time, the light color cycle and the shooting time.

7. The method according to claim 6, characterized in that The method further comprises: If the light colors corresponding to the current frame of traffic light image and the previous frame of traffic light image are different, updating the start time of the light color corresponding to the stored current frame of traffic light image according to the shooting time; The countdown is determined according to the updated start time, the light color cycle of the light color corresponding to the stored current frame traffic light image, and the shooting time.

8. The method according to claim 5 or 7, characterized in that: The method further comprises: If the light color corresponding to the traffic light image of the current frame is green, updating the stored light color cycle corresponding to red according to the shooting time and the stored start time corresponding to red; If the light color corresponding to the traffic light image of the current frame is yellow, the stored light color cycle corresponding to green is updated according to the shooting time and the stored start time corresponding to green.

9. A device for determining the color and countdown of a green light, characterized in that: include: A processing module, used for determining image features and time features according to the acquired current frame traffic light image and the shooting time of the current frame traffic light image; A classification module, used to input the image features and the time features into a classification model to obtain the light color corresponding to the traffic light image of the current frame; the classification model is a pre-trained machine learning model that determines the light color based on the image features and the time features; The processing module is also used to determine the countdown of the light color corresponding to the current frame of traffic light image based on the shooting time, the light color corresponding to the current frame of traffic light image, and whether the light color corresponding to the previous frame of traffic light image of the current frame of traffic light image is obtained.

10. An electronic device, characterized in that: include: Processor, memory, communication interface; The memory is used to store executable instructions of the processor; Wherein, the processor is configured to execute the traffic light color and countdown determination method as described in any one of claims 1 to 8 by executing the executable instructions.

11. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for determining the color and countdown of a traffic light according to any one of claims 1 to 8 is implemented.

12. A computer program product, characterized in that It includes a computer program, which, when executed by a processor, is used to implement the method for determining the color and countdown of a traffic light as described in any one of claims 1 to 8.

Images