Vehicle red light running detection method, device, electronic equipment and storage medium

Abstract

The present application provides a method, device, electronic device and computer-readable storage medium for detecting vehicles running red lights. The method for detecting vehicles running red lights comprises: obtaining a video from the driving perspective of a vehicle to be detected; obtaining the acquisition time point of the i-th frame image in the video, and the number of zebra crossing pixels contained in the i-th frame image; constructing a change curve of the number of zebra crossing pixels corresponding to the vehicle to be detected based on the number of zebra crossing pixels; determining the target time point when the vehicle to be detected passes through the zebra crossing based on the change curve and the acquisition time point; detecting the indication category of the traffic light in the first target image; and determining whether the vehicle to be detected runs a red light based on the indication category. In the present application, the situation where lane boundaries are missing, blurred, interfered or difficult to identify can be avoided, thereby improving the detection accuracy of vehicles running red lights.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a method, device, electronic device and computer-readable storage medium for detecting a vehicle running a red light. Background Art

[0002] In recent years, with the development of economy and technology, the number of vehicles has also increased significantly. At the same time, various types of vehicle-related violations and illegal cases have also increased year by year. Accurately identifying whether a vehicle has run a red light has positive significance for detecting cases, supervising violations and illegal acts, and ensuring social security.

[0003] In the prior art, generally, whether a vehicle runs a red light is determined based on the identification of lane boundaries. For example, an image of a vehicle on the road can be obtained, and the lane boundary can be extracted from the image of the vehicle to identify whether the vehicle crosses the lane boundary, and then whether the vehicle runs a red light is determined based on the identification result, etc. However, in the process of research and practice of the prior art, the inventor of the present invention found that in some scenarios, the lane boundary may be missing, blurred, interfered or difficult to identify. Therefore, in this case, the detection accuracy of the vehicle running a red light will be affected. Summary of the invention

[0004] The present application provides a method, device, electronic device and computer-readable storage medium for detecting a vehicle running a red light, aiming to solve the problem of low accuracy in detecting a vehicle running a red light when lane boundaries are missing, blurred, interfered with or difficult to identify.

[0005] In a first aspect, the present application provides a method for detecting a vehicle running a red light, the method comprising:

[0006] Acquire a video from the driving perspective of the vehicle to be detected, wherein the vehicle to be detected is equipped with a camera, and the camera is used to collect the video;

[0007] Obtaining the acquisition time point of the i-th frame image in the video and the number of zebra crossing pixels contained in the i-th frame image, wherein the video includes at least N frames of images, i is a positive integer, 1≤i≤N;

[0008] According to the number of pixels of the zebra crossing, a change curve of the number of pixels of the zebra crossing corresponding to the vehicle to be detected is constructed;

[0009] Determining a target time point at which the vehicle to be detected passes through the zebra crossing according to the change curve and the acquisition time point;

[0010] Detecting the indication category of the traffic light in the first target image, wherein the first target image refers to an image in the N frames of images whose acquisition time point is the same as the target time point, and the indication category includes allowing passage and prohibiting passage;

[0011] According to the indication category, it is determined whether the vehicle to be detected runs a red light.

[0012] In a second aspect, the present application provides a vehicle red light running detection device, the vehicle red light running detection device comprising:

[0013] An acquisition unit, used to acquire a video from the driving perspective of the vehicle to be detected, wherein the vehicle to be detected is equipped with a camera, and the camera is used to collect the video;

[0014] The acquisition unit is further used to acquire the acquisition time point of the i-th frame image in the video and the number of zebra crossing pixels contained in the i-th frame image, wherein the video includes at least N frames of images, i is a positive integer, 1≤i≤N;

[0015] A construction unit, used for constructing a change curve of the number of zebra crossing pixels corresponding to the vehicle to be detected according to the number of zebra crossing pixels;

[0016] A determination unit, used to determine a target time point at which the vehicle to be detected passes through the zebra crossing according to the change curve and the acquisition time point;

[0017] A first detection unit is used to detect the indication category of the traffic light in a first target image, wherein the first target image refers to an image in the N frames of images whose acquisition time point is the same as the target time point, and the indication category includes allowing passage and prohibiting passage;

[0018] The first detection unit is further used to determine whether the vehicle to be detected runs a red light according to the indication category.

[0019] In some embodiments of the present application, the determining unit is specifically used to:

[0020] According to the change curve and the acquisition time point, determine a first acquisition time point when the number of pixels of the zebra crossing decreases over time until it reaches zero;

[0021] The first acquisition time point is used as the target time point.

[0022] In some embodiments of the present application, the determining unit is specifically used to:

[0023] From the acquisition time points, obtain M consecutive acquisition time points after the first acquisition time point, where M is a positive integer greater than one;

[0024] The first acquisition time point and the M consecutive acquisition time points are used as the target time points.

[0025] In some embodiments of the present application, the determining unit is specifically used to:

[0026] Determine, according to the change curve and the acquisition time point, a second acquisition time point when the number of pixels of the zebra crossing begins to decrease over time;

[0027] The second acquisition time point is used as the target time point.

[0028] In some embodiments of the present application, the vehicle running a red light detection device further includes a second detection unit. After the step of acquiring the video from the driving perspective of the vehicle to be detected, the second detection unit is specifically used to:

[0029] Acquire the three-axis angular velocity of the i-th frame image, wherein the three-axis angular velocity includes an x-axis angular velocity, a y-axis angular velocity, and a z-axis angular velocity, and the x-axis direction is consistent with the driving direction of the vehicle to be detected;

[0030] Determine a first direction curve of the vehicle to be detected according to the y-axis angular velocity, and determine a second direction curve of the vehicle to be detected according to the z-axis angular velocity;

[0031] Determining whether the vehicle to be detected is traveling in a straight line at a collection time point corresponding to the video according to the first direction curve and the second direction curve;

[0032] In some embodiments of the present application, the acquisition unit is specifically used to:

[0033] When it is determined that the vehicle to be detected is traveling in a straight line within the acquisition time point corresponding to the video, the acquisition time point of the i-th frame image in the video and the number of zebra crossing pixels contained in the i-th frame image are obtained.

[0034] In some embodiments of the present application, the first detection unit is specifically used for:

[0035] Performing classification processing on the first target image to obtain a first classification result of the first target image, wherein the first classification result is used to indicate whether there is a traffic light in the first target image and the type of lamp pole of the traffic light;

[0036] When the first classification result is that a traffic light exists in the first target image, the first target image is identified according to the lamp pole type to determine the indication category of the traffic light.

[0037] In some embodiments of the present application, the vehicle running a red light detection device further includes a third detection unit. After the step of acquiring the video from the driving perspective of the vehicle to be detected, the third detection unit is specifically used to:

[0038] Classify each frame image in the video to obtain a second classification result of each frame image, wherein the second classification result is used to indicate whether the vehicle to be detected is near a traffic light intersection;

[0039] Detecting the number of second target images according to the second classification result, wherein the second target image refers to an image in each frame of images in which the classification result shows that the vehicle to be detected is near a traffic light intersection;

[0040] In some embodiments of the present application, the acquisition unit is specifically used to:

[0041] When it is detected that the number of the second target images is greater than a preset number threshold, the acquisition time point of the i-th frame image in the video and the number of zebra crossing pixels contained in the i-th frame image are obtained.

[0042] In a third aspect, the present application further provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, the processor executes the steps of any one of the vehicle red light running detection methods provided in the present application.

[0043] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is loaded by a processor to execute the steps in the method for detecting a vehicle running a red light.

[0044] This application determines the target time point when the vehicle to be detected passes through the zebra crossing based on the change line of the number of pixels of the zebra crossing under the driving perspective of the vehicle to be detected; and detects the indication category of the traffic light at the target time point, thereby determining whether the vehicle to be detected has run a red light according to the indication category. Since there is no need to fit the lane boundary, the problem of low detection accuracy of vehicles running red lights due to missing, blurred, interfered or difficult to identify lane boundaries can be avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1It is a scene schematic diagram of a vehicle red light running detection system provided in an embodiment of the present application;

[0047] Figure 2 This is a flow chart of a method for detecting a vehicle running a red light provided in an embodiment of the present application;

[0048] Figure 3 This is a schematic diagram of an embodiment scenario of a change curve provided in an embodiment of the present application;

[0049] Figure 4 is a schematic diagram of another embodiment scenario of the change curve provided in the embodiments of the present application;

[0050] Figure 5 This is a schematic diagram of a scene of a red light intersection provided in an embodiment of the present application;

[0051] Figure 6 It is a schematic diagram of another embodiment scenario of the change curve provided in the embodiments of the present application;

[0052] Figure 7 is a schematic diagram of another embodiment scenario of the change curve provided in the embodiments of the present application;

[0053] Figure 8 It is a schematic diagram of the structure of an embodiment of a detection device for a vehicle running a red light provided in an embodiment of the present application;

[0054] Fig. 9 It is a schematic diagram of the structure of an embodiment of an electronic device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0056] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0057] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present application can also be implemented without using these specific details. In other examples, the known process will not be elaborated in detail to avoid unnecessary details that make the description of the present application embodiment obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the present application.

[0058] The embodiments of the present application provide a method, device, electronic device and computer-readable storage medium for detecting a vehicle running a red light, wherein the vehicle running a red light detection device can be integrated in an electronic device, which can be a server, a terminal or other device.

[0059] First of all, before introducing the embodiments of the present application, the relevant content about the application background of the embodiments of the present application is introduced.

[0060] In recent years, with the explosive development of the express delivery and fast food industries, the number of couriers and food delivery boys needed for terminal delivery has increased rapidly. The means of transportation they need are mostly two-wheeled or three-wheeled trams, without hardware safety protection measures. In addition, due to the high complexity of urban traffic in my country, the time of terminal delivery is relatively concentrated and the timeliness requirements are high; this has greatly affected the safety of terminal delivery personnel.

[0061] According to statistics, in the first half of 2019 in Shanghai alone, there were 325 safety accidents related to terminal delivery personnel, resulting in 5 deaths and 324 injuries. Therefore, in order to reduce the chance of injury in the event of a safety accident, the development of a red light running monitoring function is urgent.

[0062] Based on the above-mentioned defects of the existing related technologies, the embodiments of the present application provide a method for detecting vehicles running red lights, which at least overcomes the defects of the existing related technologies to a certain extent.

[0063] The executor of the vehicle red light running detection method in the embodiment of the present application may be the vehicle red light running detection device provided in the embodiment of the present application, or different types of electronic devices such as a server device, a physical host or a user equipment (UE) integrated with the vehicle red light running detection device, wherein the vehicle red light running detection device may be implemented in hardware or software, and the UE may specifically be a terminal device such as a smart phone, a tablet computer, a laptop computer, a PDA, a desktop computer or a personal digital assistant (PDA).

[0064] The electronic device can adopt a single-operation working mode, or can also adopt a device cluster working mode, and by applying the vehicle red light running detection method provided in the embodiment of the present application, obtain a video from the driving perspective of the vehicle to be detected; obtain the acquisition time point of the i-th frame image in the video, and the number of zebra crossing pixels contained in the i-th frame image; according to the zebra crossing pixel points, construct a change curve of the zebra crossing pixel points corresponding to the vehicle to be detected; according to the change curve and the acquisition time point, determine the target time point for the vehicle to be detected to pass through the zebra crossing; detect the indication category of the traffic light in the first target image; and determine whether the vehicle to be detected has run a red light according to the indication category.

[0065] See also Figure 1 , Figure 1 1 is a schematic diagram of a scene of a vehicle red light running detection system provided in an embodiment of the present application. The vehicle red light running detection system may include an electronic device 100, and a vehicle red light running detection device is integrated in the electronic device 100. For example, the electronic device can avoid the situation where the lane boundary is missing, blurred, interfered or difficult to identify, thereby improving the detection accuracy of the vehicle red light running.

[0066] In addition, if Figure 1 As shown, the vehicle red light running detection system may further include a memory 200 for storing data, such as image data and video data.

[0067] It should be noted that Figure 1 The scenario diagram of the vehicle red light running detection system shown is merely an example. The vehicle red light running detection system and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the vehicle red light running detection system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is equally applicable to similar technical problems.

[0068] Next, we will introduce the method for detecting a vehicle running a red light provided in an embodiment of the present application. In the embodiment of the present application, an electronic device is used as an execution subject. For the sake of simplicity and ease of description, the execution subject will be omitted in the subsequent method embodiments.

[0069] Reference Figure 2 , Figure 2 It is a flow chart of a method for detecting a vehicle running a red light provided by an embodiment of the present application. It should be noted that although a logical sequence is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than that shown here. The method for detecting a vehicle running a red light includes steps S10 to S60, wherein:

[0070] S10: Obtain a video from the driving perspective of the vehicle to be detected.

[0071] The vehicle to be detected is equipped with a camera, and the camera is used to collect the video. The video from the driving perspective of the vehicle to be detected is referred to as video in the following for simplicity of description; unless otherwise specified, the video is the video from the driving perspective of the detection vehicle.

[0072] For example, in some embodiments of the present application, a camera is installed in front of the vehicle to be detected (such as at the highest point of the windshield) to collect video from the driving perspective of the vehicle to be detected. In order to allow as many traffic lights and zebra crossing lights on the road where the vehicle to be detected is located as possible to fall into the video, a camera with a field of view as large as possible can be selected, for example, a camera with a field of view of 170 degrees vertical field of view can be selected.

[0073] Specifically, in actual applications, an electronic device that uses the method for detecting a vehicle running a red light provided in an embodiment of the present application may directly include a camera on the vehicle to be detected in terms of hardware (the camera is mainly used to collect images of the vehicle to be detected from the driving perspective), and store the images taken by the camera locally, which can be directly read inside the electronic device; alternatively, the electronic device may also establish a network connection with the camera, and obtain the images obtained by the camera online from the camera based on the network connection; alternatively, the electronic device may also read the images obtained by the camera from a related storage medium that stores the images obtained by the camera, and the specific acquisition method is not limited here.

[0074] The camera can capture images according to a preset shooting method, such as setting a shooting height, shooting direction or shooting distance. The specific shooting method can be adjusted according to the camera itself, and is not limited here. The multiple frames of images captured by the camera can be combined into a video through a timeline.

[0075] S20, obtaining the acquisition time point of the i-th frame image in the video, and the number of zebra crossing pixels contained in the i-th frame image.

[0076] The video includes at least N frames of images, i is a positive integer, 1≤i≤N.

[0077] In some embodiments, the number of image frames in the video is N. That is, for each of the N frames of the video, the acquisition time point of the frame image (recorded as Ti) and the number of zebra crossing pixels contained in the frame image (recorded as Xi) are obtained.

[0078] In some embodiments, the number of image frames in the video is greater than N. That is, firstly, the image in the video is intercepted to obtain N frames of images (for example, for video a, one frame of image is intercepted every 3 frames to obtain N frames of image); then, for each frame in the N frames of image, the acquisition time point of the frame of image (denoted as Ti) and the number of zebra crossing pixels contained in the frame of image (denoted as Xi) are obtained respectively.

[0079] For example, if N is 100, then for N frames of images in the video, the acquisition time points (respectively recorded as: T1, T2, T3, ..., T100) of the i-th frame image (i is 1, 2, 3, ..., 100) are obtained respectively. And the number of zebra crossing pixels contained in the i-th frame image (i is 1, 2, 3, ..., 100) is obtained respectively (respectively recorded as: X1, X2, X3, ..., X100).

[0080] Among them, "obtaining the number of zebra crossing pixels contained in the i-th frame image" can specifically include: first, extracting features of the i-th frame image through the trained zebra crossing detection network to obtain the image features of the i-th frame image. And predicting the zebra crossing area in the i-th frame image based on the image features of the i-th frame image through the zebra crossing detection network. Then, counting the number of pixels in the zebra crossing area in the i-th frame image, thereby obtaining the number of zebra crossing pixels Xi contained in the i-th frame image.

[0081] The above zebra crossing detection network can be trained through the following steps:

[0082] 1. Build a preliminary zebra crossing detection network.

[0083] For example, an open source network (such as a YOLOv network) with model parameters as default values ​​(which can be used for detection tasks) can be used as a preset zebra crossing detection network, and the preliminary zebra crossing detection network can include a feature extraction module and a prediction module. Among them, the feature extraction module is used to extract features from the sample image to obtain the image features of the sample image, and the prediction module is used to predict based on the image features of the sample image to obtain the zebra crossing area contained in the sample image.

[0084] 2. Get the training dataset.

[0085] The training data set includes multiple sample images, a portion of the sample images may be images containing zebra crossings, and a portion of the sample images may be images not containing zebra crossings.

[0086] 3. Use the training data set to train the preliminary zebra crossing detection network until the preliminary zebra crossing detection network converges to obtain the trained zebra crossing detection network.

[0087] Among them, the trained zebra crossing detection network can fully learn the relationship between the zebra crossing area and the image features, so as to accurately detect the detection frame of the zebra crossing area in the image.

[0088] Among them, the training process of the zebra crossing detection network is similar to the training process of the existing network model. For the training process that is not described in detail, the training method of the existing network model can be referred to, and will not be repeated here.

[0089] S30. Constructing a variation curve of the number of zebra crossing pixels corresponding to the vehicle to be detected according to the number of zebra crossing pixels.

[0090] Please refer to Figure 3 and Figure 4 , Figure 3 is a schematic diagram of an embodiment scenario of a change curve provided in an embodiment of the present application, Figure 4 It is a schematic diagram of another embodiment scenario of the change curve provided in the embodiments of the present application.

[0091] The change curve refers to the change curve of the number of zebra crossing pixels in the video. The change curve is used to reflect the traffic status (including stopped traffic and continued traffic) of the vehicle to be detected on the zebra crossing before and after crossing the stop line at the zebra crossing.

[0092] Specifically, in some embodiments, the acquisition time point Ti of the i-th frame image is used as the independent variable and the number of zebra crossing pixels Xi is used as the dependent variable to construct a change curve of the number of zebra crossing pixels corresponding to the vehicle to be detected, such as Figure 3 In order to facilitate the direct determination of the acquisition time point Ti, in the following text, the change curve constructed by the construction method of "taking the acquisition time point Ti of the i-th frame image as the independent variable and the number of zebra crossing pixels Xi as the dependent variable" is taken as an example.

[0093] In some embodiments, the image sequence number "i" of the i-th frame image is used as the independent variable and the number of zebra crossing pixels Xi is used as the dependent variable to construct a change curve of the number of zebra crossing pixels corresponding to the vehicle to be detected, such as Figure 4 It can be understood that if the change curve is constructed in the manner of "taking the image sequence number "i" of the i-th frame image as the independent variable and the number of zebra crossing pixels Xi as the dependent variable", the independent variable i can be matched with the acquisition time point Ti.

[0094] In practical applications, the above two implementations may not be limited to constructing a change curve of the number of zebra crossing pixels corresponding to the vehicle to be detected, as long as the constructed change curve can highlight the change of the number of zebra crossing pixels.

[0095] In order to better understand the change curve in the embodiment of the present application, three specific examples are provided below for illustration.

[0096] Please refer to Figure 3 、 Figure 4 and Figure 5 , Figure 5 which is a schematic diagram of a scenario at a red light intersection provided in an embodiment of the present application. In a specific example, the vehicle to be detected first gradually approaches the zebra crossing intersection within a period of time, then stops passing in front of the stop line at the zebra crossing intersection for a period of time, and finally continues to pass through the zebra crossing stop line. At this time, the change curve sequentially reflects the following 3 situations (i.e., the change curve is a 3-segment curve):

[0097] 1. When the vehicle to be detected gradually approaches the zebra crossing, in the image captured by the camera, the ratio between the zebra crossing area and the image gradually increases. That is, the number of zebra crossing pixel points Xi gradually increases, as shown in the part of Ti < Tl in Figure 3 or as shown in the part of i < il in Figure 4 .

[0098] 2. When the vehicle to be detected stops passing in front of the stop line at the zebra crossing intersection, in the image captured by the camera, the ratio between the zebra crossing area and the image remains unchanged. That is, the number of zebra crossing pixel points Xi remains unchanged, as shown in the part of Tl ≤ Ti ≤ Tr in Figure 3 or as shown in the part of il ≤ i ≤ ir in Figure 4 .

[0099] 3. When the vehicle to be detected is passing through the zebra crossing, in the image captured by the camera, the ratio between the zebra crossing area and the image gradually decreases. That is, the number of zebra crossing pixel points Xi gradually decreases until it becomes 0, as shown in the part of Ti < Tr in Figure 3 or as shown in the part of i < ir in Figure 4 .

[0100] Please refer to Figure 5 and Figure 6 , Figure 6 which is another schematic diagram of an embodiment scenario of the change curve provided in an embodiment of the present application. In a specific example, the vehicle to be detected first gradually approaches the zebra crossing intersection within a period of time, crosses the stop line at the zebra crossing intersection and continues to pass for a period of time, (after crossing the stop line at the zebra crossing intersection) stops passing on the zebra crossing for a period of time, and finally continues to pass. At this time, the change curve sequentially reflects the following 4 situations (i.e., the change curve is a 4-segment curve):

[0101] 1. When the vehicle to be detected gradually approaches the zebra crossing, in the image captured by the camera, the ratio between the zebra crossing area and the image gradually increases. That is, the number of zebra crossing pixel points Xi gradually increases, as shown in the part of Ti < Tl in Figure 6 .

[0102] 2. When the vehicle to be detected continuously passes after crossing the stop line of the zebra crossing intersection, in the image captured by the camera, the ratio between the zebra crossing area and the image gradually decreases. That is, the number of zebra crossing pixel points Xi gradually decreases, as shown in the part of Figure 6 where Tl ≤ Ti ≤ Tm. Figure 6 as shown in the part of Tl ≤ Ti ≤ Tm in Figure 6 .

[0103] 3. When the vehicle to be detected crosses the stop line of the zebra crossing intersection and stops passing on the zebra crossing, in the image captured by the camera, the ratio between the zebra crossing area and the image remains unchanged. That is, the number of zebra crossing pixel points Xi remains unchanged, as shown in the part of Figure 6 where Tm ≤ Ti ≤ Tr. Figure 6 as shown in the part of Tm ≤ Ti ≤ Tr in Figure 6 .

[0104] 4. When the vehicle to be detected continues to cross the zebra crossing, in the image captured by the camera, the ratio between the zebra crossing area and the image gradually decreases. That is, when the number of zebra crossing pixel points Xi gradually decreases until it becomes 0, as shown in the part of Figure 6 where Ti < Tr. Figure 6 as shown in the part of Ti < Tr in Figure 6 .

[0105] Please refer to Figure 5 and Figure 7 , Figure 7 Figure 7 is another schematic diagram of an embodiment scenario of the change curve provided in the embodiment of the present application. In a specific example, when the vehicle to be detected continuously passes from approaching the zebra crossing intersection, crossing the stop line of the zebra crossing intersection to crossing the zebra crossing, the change curve sequentially reflects the following two situations at this time (that is, the change curve is a two-segment curve):

[0106] 1. When the vehicle to be detected gradually approaches the zebra crossing until before crossing the stop line, in the image captured by the camera, the ratio between the zebra crossing area and the image gradually increases. That is, the number of zebra crossing pixel points Xi gradually increases, as shown in the part of Figure 7 where Ti < Tlr. Figure 7 as shown in the part of Ti < Tlr in Figure 7 .

[0107] 2. When the vehicle to be detected continuously passes after crossing the stop line of the zebra crossing intersection, in the image captured by the camera, the ratio between the zebra crossing area and the image gradually decreases. That is, the number of zebra crossing pixel points Xi gradually decreases, as shown in the part of Figure 7 where Tlr ≤ Ti. Figure 7 as shown in the part of Tlr ≤ Ti in Figure 7 .

[0108] It can be understood that in addition to the above three specific examples, the change curve can also reflect more situations of the passing state (including stopping and continuous passing) of the vehicle to be detected before and after passing the zebra crossing (that is, the traffic light intersection).

[0109] S40. Determine the target time point for the vehicle to be detected to pass the zebra crossing according to the change curve and the acquisition time point.

[0110] Among them, the target time point refers to the time point when the vehicle to be detected passes the zebra crossing.

[0111] In the embodiment of the present application, for the characteristics of "during the whole process of the vehicle to be detected passing through the zebra crossing, the number of zebra crossing pixel points Xi of the change curve increases from 0, and after Xi reaches the maximum value, Xi gradually decreases from the maximum value until Xi is 0", and "the time point when Xi gradually decreases from the maximum value proves that the vehicle to be detected has just passed the stop line of the zebra crossing", the target time point when the vehicle to be detected passes through the stop line of the zebra crossing is determined according to the change curve and the acquisition time point Ti. The method for determining the target time point is described in detail later, and will not be repeated here.

[0112] S50: Detect the indication category of the traffic light in the first target image.

[0113] The first target image refers to an image in the N frames of images whose acquisition time point is the same as the target time point. The indication category is used to indicate whether the traffic light is red at the target time point, and the indication category may include allowing passage and prohibiting passage.

[0114] For example, if the traffic light in the first target image is a red light, the indication category is no passage; that is, at the target time point, the traffic light indicates no passage.

[0115] For another example, if the traffic light in the first target image is green, then the indication category is passage is allowed; that is, at the target time point, the traffic light indicates passage is allowed.

[0116] Among them, “detecting the indication category of the traffic light in the first target image” is introduced in detail and exemplified in the following text, and will not be repeated here for the sake of simplicity.

[0117] S60: Determine whether the vehicle to be detected runs a red light according to the indication category.

[0118] Specifically, if the indication category is that passage is allowed, it is determined that the vehicle to be detected has not run a red light. If the indication category is that passage is prohibited, it is determined that the vehicle to be detected has run a red light.

[0119] From the above, it can be seen that the target time point when the vehicle to be detected passes through the zebra crossing is determined based on the change line of the number of zebra crossing pixels under the driving perspective of the vehicle to be detected; and the indication category of the traffic light at the target time point is detected, so as to determine whether the vehicle to be detected runs a red light according to the indication category. Since there is no need to fit the lane boundary, the problem of low detection accuracy of vehicles running red lights due to missing, blurred, interfering or difficult to identify lane boundaries can be avoided.

[0120] In some embodiments, the target time point determined in the above step S40 is a time point, that is, the target time point refers to a time point when the vehicle to be detected passes through the stop line of the zebra crossing. Specifically, it can be the time point when the number of zebra crossing pixels Xi begins to gradually decrease (recorded as T0), which is used as the target time point; it can also be a time point after T0 (recorded as T0'), which is used as the target time point. Figure 3 As shown in FIG. 1 , since the number of zebra crossing pixel points Xi gradually decreases after the moment Tr, it proves that the vehicle to be detected is crossing the stop line of the zebra crossing at the moment Tr, so the moment Tr can be defined as the target time point when the vehicle to be detected passes through the zebra crossing. Alternatively, the next moment after Tr (such as Tr+1) can be defined as the target time point when the vehicle to be detected passes through the zebra crossing.

[0121] In a specific example, the acquisition time point of the image corresponding to when the number of zebra crossing pixels Xi decreases with time until it reaches zero is used as the target time point. That is, at this time, step S40 may specifically include: determining the first acquisition time point when the number of zebra crossing pixels decreases with time until it reaches zero according to the change curve and the acquisition time point; and using the first acquisition time point as the target time point.

[0122] The first acquisition time point refers to the acquisition time point of the corresponding image when the number of zebra crossing pixels in the video decreases over time until it reaches zero.

[0123] For example, the change curve is Figure 3 As shown, according to the change curve and the acquisition time point Ti, the first acquisition time point when the number of zebra crossing pixels Xi decreases with time until it reaches zero is determined to be Tp, and Tp is used as the target time point.

[0124] For example, the change curve is Figure 4 As shown, according to the change curve, the image corresponding to when the number of zebra crossing pixels Xi decreases with time until it reaches zero is determined to be ip. Then the acquisition time point Tp (ie, the first acquisition time point) of the image ip is used as the target time point.

[0125] For example, the change curve is Figure 6 As shown, according to the change curve and the acquisition time point Ti, the first acquisition time point when the number of zebra crossing pixels Xi decreases with time until it reaches zero is determined to be Tp, and Tp is used as the target time point.

[0126] From the above content, it can be seen that since the number of pixels of the zebra crossing decreases over time until it reaches zero, the corresponding image acquisition time point just reflects the time point when the vehicle to be detected crosses the zebra crossing. By using the first acquisition time point when the number of pixels of the zebra crossing decreases over time until it reaches zero as the target time point, the subsequent judgment of the indication category of the traffic light at the target time point can more accurately reflect the indication category of the traffic light when the vehicle to be detected crosses the zebra crossing, thereby improving the accuracy of judging whether the vehicle runs a red light.

[0127] In a specific example, the acquisition time point of the image corresponding to when the number of zebra crossing pixels Xi starts to decrease over time is used as the target time point. That is, step S40 may specifically include: determining a second acquisition time point when the number of zebra crossing pixels starts to decrease over time according to the change curve and the acquisition time point; and using the second acquisition time point as the target time point.

[0128] The second acquisition time point refers to the acquisition time point of the corresponding image when the number of zebra crossing pixels in the video begins to decrease over time.

[0129] For example, the change curve is Figure 3 As shown, according to the change curve and the acquisition time point Ti, the second acquisition time point when the number of zebra crossing pixels Xi begins to decrease over time is determined to be Tr, and Tr is used as the target time point.

[0130] For example, the change curve is Figure 4 As shown, according to the change curve, the image corresponding to when the number of zebra crossing pixels Xi starts to decrease with time is determined to be ir. Then the acquisition time point Tr (ie, the second acquisition time point) of the image ir is used as the target time point.

[0131] From the above content, it can be seen that the acquisition time point of the corresponding image when the number of pixels of the zebra crossing begins to decrease over time also reflects the time point when the vehicle to be detected crosses the zebra crossing. By taking the second acquisition time point when the number of pixels of the zebra crossing begins to decrease over time as the target time point, the subsequent determination of the indication category of the traffic light at the target time point can more accurately reflect the indication category of the traffic light when the vehicle to be detected crosses the zebra crossing, thereby improving the accuracy of determining whether the vehicle has run a red light.

[0132] On the one hand, the indication category of the traffic light may jump between two adjacent time points; on the other hand, it takes a certain amount of time for the vehicle to pass through the zebra crossing, not just a moment; in order to improve the accuracy of detecting whether the vehicle has run a red light, the indication category of the traffic light must be "allowed to pass" for a certain period of time to determine that the vehicle to be detected has not run a red light. For example, since the indication category of the traffic light at the second acquisition time point Tr is "allowed to pass" and the indication category of the traffic light at the next time point after the second acquisition time point Tr is "no passage", the vehicle to be detected still runs a red light when passing through the zebra crossing.

[0133] To this end, in some embodiments, the target time point determined in the above step S40 includes multiple time points, that is, the target time point refers to a certain time period when the vehicle to be detected passes through the zebra crossing. Specifically, it can be the time point when the number of zebra crossing pixel points Xi begins to gradually decrease (recorded as T0), and multiple consecutive time points after T0, as the target time point; it can also be a certain time point after T0 (recorded as T0'), and multiple consecutive time points after T0' as the target time point. Figure 3 As shown, since the number of zebra crossing pixel points Xi begins to gradually decrease after time Tr, it proves that the vehicle to be detected is crossing the stop line of the zebra crossing at time Tr. Therefore, the time period Tr to (Tr+n) can be defined as the target time point for the vehicle to be detected to pass through the zebra crossing.

[0134] That is, further, in a specific example, the above “taking the first acquisition time point as the target time point” may specifically include: obtaining M consecutive acquisition time points after the first acquisition time point from the acquisition time points; taking the first acquisition time point and the M consecutive acquisition time points as the target time point. Wherein, M is a positive integer greater than one.

[0135] For example, the change curve is Figure 3 As shown, M consecutive acquisition time points after the first acquisition time point Tp can be obtained from the acquisition time point Ti; and the first acquisition time point Tp and the M consecutive acquisition time points after the first acquisition time point Tp, that is, the time period Tp to (Tp+M), are used as the target time point.

[0136] Further, in a specific example, the above-mentioned “taking the second collection time point as the target time point” may specifically include: obtaining M consecutive collection time points after the second collection time point from the collection time points; and taking the second collection time point and the M consecutive collection time points as the target time point. Wherein, M is a positive integer greater than one.

[0137] For example, the change curve is Figure 6As shown, M consecutive acquisition time points after the second acquisition time point Tr can be obtained from the acquisition time point Ti; and the second acquisition time point Tr and the M consecutive acquisition time points after the second acquisition time point Tr, that is, the time period of Tr to (Tr+M), are used as the target time point.

[0138] From the above content, it can be seen that, on the one hand, due to the certain distance between the zebra crossings, it takes a certain amount of time for the vehicle to be detected to cross the zebra crossing; on the other hand, since the indication category of the traffic light may jump between the first time point and the second time point; therefore, by obtaining the first acquisition time point and M consecutive acquisition time points after the first acquisition time point as the target time point, the indication category of the traffic light within a period of time can be determined, which can improve the detection accuracy of whether the vehicle runs a red light.

[0139] Since the above-mentioned vehicle running red light detection method can only accurately detect whether a vehicle has run a red light under the premise that the vehicle to be detected is in a straight-moving state. To this end, the vehicle running red light detection method provided in the embodiments of the present application also provides a scheme for detecting whether the vehicle is in a straight-moving state. That is, in some embodiments of the present application, the vehicle running red light detection method also includes the following steps a1 to a3, wherein:

[0140] a1. Obtain the three-axis angular velocity of the i-th frame image.

[0141] The three-axis angular velocity of the i-th frame image refers to the three-axis angular velocity of the vehicle to be detected at the acquisition time point Ti of the i-th frame image. The three-axis angular velocity of the i-th frame image specifically includes: the x-axis angular velocity, y-axis angular velocity and z-axis angular velocity of the vehicle to be detected at the acquisition time point Ti of the i-th frame image (relative to the three-dimensional coordinate axis of the reference). The x-axis direction is consistent with the driving direction of the vehicle to be detected.

[0142] a2. Determine a first direction curve of the vehicle to be detected according to the y-axis angular velocity, and determine a second direction curve of the vehicle to be detected according to the z-axis angular velocity.

[0143] The first direction curve is used to indicate the displacement of the vehicle to be detected in the y-axis direction, and the second direction curve is used to indicate the displacement of the vehicle to be detected in the z-axis direction.

[0144] Specifically, in some embodiments, the acquisition time point Ti of the i-th frame image is used as the independent variable, and the y-axis angular velocity of the vehicle to be detected (at the acquisition time point Ti of the i-th frame image) is used as the dependent variable to construct the first direction curve of the vehicle to be detected. The first direction curve can be a multi-order polynomial, for example, the first direction curve is a third-order polynomial, expressed as y=a1*Ti 3 +b1*Ti2 +c1*Ti+d1, where a1, b1, c1, and d1 are all constants.

[0145] In some embodiments, the image sequence number "i" of the i-th frame image is used as the independent variable and the z-axis angular velocity of the vehicle to be detected (at the acquisition time point Ti of the i-th frame image) is used as the dependent variable to construct a second direction curve of the vehicle to be detected. The second direction curve can be a multi-order polynomial, for example, the second direction curve is a third-order polynomial, expressed as y=a2*Ti 3 +b2*Ti 2 +c2*Ti+d2, where a2, b2, c2, and d2 are all constants.

[0146] It is understandable that the x-axis, y-axis, and z-axis are only used for the purpose of achieving a clear description, and are not used to limit specific direction axes. In actual application, the x-axis, y-axis, and z-axis can be interchanged. For example, the y-axis direction can be made consistent with the driving direction of the vehicle to be detected, and the first direction curve of the vehicle to be detected is determined according to the x-axis angular velocity, and the second direction curve of the vehicle to be detected is determined according to the z-axis angular velocity. For another example, the z-axis direction can be made consistent with the driving direction of the vehicle to be detected, and the first direction curve of the vehicle to be detected is determined according to the x-axis angular velocity, and the second direction curve of the vehicle to be detected is determined according to the y-axis angular velocity.

[0147] a3. Determine whether the vehicle to be detected is traveling in a straight line at a collection time point corresponding to the video according to the first direction curve and the second direction curve.

[0148] Specifically, it is detected whether the first directional curve and the second directional curve are both close to a straight line. For example, if the constants of the high-order terms of the first directional curve (such as the constants a1 and b1 in the first directional curve example of step a2 above) are close to 0, and if the constants of the high-order terms of the first directional curve are all less than the preset value 0.001, it can be determined that the first directional curve is close to a straight line. For another example, if the constants of the high-order terms of the second directional curve (such as the constants a2 and b2 in the second directional curve example of step a2 above) are both close to 0, and if the constants of the high-order terms of the second directional curve are all less than the preset value 0.002, it can be determined that the second directional curve is close to a straight line.

[0149] If it is detected that both the first direction curve and the second direction curve are close to straight lines, it proves that the vehicle to be detected has no deflection in the y-axis direction and the z-axis direction, that is, the vehicle to be detected maintains straight driving within the acquisition time point corresponding to the video.

[0150] If it is detected that the first direction curve is not close to a straight line, or it is detected that the second direction curve is not close to a straight line, it proves that the vehicle to be detected has deflection in the y-axis direction or the z-axis direction, that is, the vehicle to be detected is not traveling in a straight line at the acquisition time point corresponding to the video.

[0151] When it is determined that the vehicle to be detected is driving in a straight line at the acquisition time point corresponding to the video, step S20 is continued; otherwise, step S20 is not continued. That is, step S20 may specifically include: when it is determined that the vehicle to be detected is driving in a straight line at the acquisition time point corresponding to the video, the acquisition time point of the i-th frame image in the video and the number of zebra crossing pixels contained in the i-th frame image are obtained. By limiting the case of driving in a straight line and continuing to process the video further, on the one hand, the accuracy of distinguishing vehicles running red lights can be improved; on the other hand, unnecessary data processing can be reduced.

[0152] In some embodiments of the present application, the above step S50 may specifically include the following steps b1 to b2, wherein:

[0153] b1. Classify the first target image to obtain a first classification result of the first target image.

[0154] The first classification result is used to indicate whether there is a traffic light in the first target image and the type of lamp pole of the traffic light.

[0155] The light pole types may specifically include a vertical pole with only one indicator light, an inverted L-shaped pole with only one indicator light, an inverted L-shaped pole with multiple indicator lights, and the like.

[0156] In some implementations, the trained indicator light detection network may be used to perform classification processing on the first target image to obtain a first classification result of the first target image.

[0157] Specifically, first, the trained indicator light detection network is called to perform feature extraction on the first target image to obtain the first image feature of the first target image.

[0158] Then, the trained indicator light detection network is called to perform prediction processing according to the first image feature to determine whether there is a traffic light in the first target image and the lamp pole type of the traffic light in the first target image, and obtain a first classification result of the first target image.

[0159] The above indicator light detection network can be trained through the following steps:

[0160] 1. Build a preliminary indicator light detection network.

[0161] For example, an open source network (such as a MobileNet network) with default model parameters (which can be used for detection and classification tasks) can be used as a preset indicator light detection network. The preliminary indicator light detection network can include a feature extraction module and a prediction module. The feature extraction module is used to extract features from the sample image to obtain the image features of the sample image, and the prediction module is used to predict based on the image features of the sample image to determine whether there is a traffic light in the sample image and the type of the light pole of the traffic light in the sample image.

[0162] 2. Get the training dataset.

[0163] The training data set includes multiple sample images, a portion of the sample images may be images containing traffic lights, and a portion of the sample images may be images not containing traffic lights.

[0164] 3. Use the training data set to train the preliminary indicator light detection network until the preliminary indicator light detection network converges to obtain the trained indicator light detection network.

[0165] Among them, the trained traffic light detection network can fully learn the image features of zebra crossing traffic lights and the relationship between the light pole type of the traffic light and the image features, so that it can accurately identify whether there is a traffic light in the image (including the detection frame of the traffic light) and the light pole type of the traffic light.

[0166] Among them, the training process of the indicator light detection network is similar to the training process of the existing network model. For the training process that is not described in detail, the training method of the existing network model can be referred to, and will not be repeated here.

[0167] b2. When the first classification result is that a traffic light exists in the first target image, the first target image is identified according to the lamp pole type to determine the indication category of the traffic light.

[0168] Specifically, the indicator light detection network may further include an indicator light classification module. The indicator light classification module may further include multiple parallel classification branches, each of which may extract the image features of the detection frame area of ​​the traffic light from the first target image for each different type of light pole; and classify the traffic light according to the image features of the detection frame area of ​​the traffic light (e.g., the traffic light may be classified into "red light" or "green light"), thereby obtaining the indication category of the traffic light.

[0169] For example, the first classification result is that there is a traffic light in the first target image. When the light pole type is "inverted L-shaped pole with only one light", based on the detection frame of the traffic light in the first target image determined in the above step b1, a classification branch of the light classification module in the light detection network (for "inverted L-shaped pole with only one light") is further called to extract and identify the indication category of the traffic light based on the image features of the detection frame area of ​​the traffic light (for example, the traffic light can be classified as "red light" or "green light" to determine whether the indication category of the traffic light is to allow passage or prohibit passage).

[0170] From the above content, it can be seen that since there are multiple types of lamp poles for traffic lights, and the number and position of traffic lights of each type of lamp pole may be different; by identifying and determining the indication category of the traffic light in the driving direction of the vehicle to be detected (i.e., the straight direction in the embodiment of the present application) based on whether there is a traffic light in the first target image and the lamp pole type of the traffic light, the misjudgment rate of whether the vehicle to be detected is allowed to pass can be reduced, thereby improving the detection accuracy of vehicles running red lights.

[0171] Since the vehicle to be detected will always shoot video, when the vehicle to be detected is not at a traffic light intersection, there is no possibility that the vehicle to be detected will run a red light. Therefore, the video shot by the vehicle to be detected needs to be filtered out to avoid unnecessary data processing.

[0172] To this end, in some embodiments of the present application, the method for detecting a vehicle running a red light may further include the following steps c1 to c2, wherein:

[0173] c1. Classify each frame image in the video to obtain a second classification result of each frame image.

[0174] The second classification result is used to indicate whether the vehicle to be detected is near a traffic light intersection.

[0175] Specifically, first, the trained video classification network is called to extract features of each frame image in the video to obtain the second image features of each frame image in the video.

[0176] Then, the trained video classification network is called to perform prediction processing according to the second image feature to determine whether the vehicle to be detected in each frame of the video is near the traffic light intersection, and the second classification result of each frame of the video is obtained.

[0177] Among them, the training process of the above-mentioned video classification network is similar to the training process of the above-mentioned indicator light detection network. For details, please refer to the above-mentioned indicator light detection network training process. To simplify the description, it will not be repeated here.

[0178] c2. Detect the number of second target images according to the second classification result.

[0179] The second target image refers to an image in each frame of the video, in which the classification result is that the vehicle to be detected is near a traffic light intersection.

[0180] For example, if there are 100 images of the vehicle to be detected near a traffic light intersection in each frame image in the video, then the number of the second target images is determined to be 100.

[0181] When it is determined that the number of the second target images is greater than the preset number threshold, step S20 is continued; otherwise, step S20 is not continued. That is, at this time, step S20 may specifically include: when it is detected that the number of the second target images is greater than the preset number threshold, the acquisition time point of the i-th frame image in the video and the number of zebra crossing pixels contained in the i-th frame image are obtained. By limiting the case where the vehicle reaches the traffic light intersection and continuing to process the video further, on the one hand, the accuracy of distinguishing vehicles running red lights can be improved; on the other hand, the amount of unnecessary data processing can be reduced.

[0182] In order to better implement the vehicle red light running detection method in the embodiment of the present application, based on the vehicle red light running detection method, the embodiment of the present application also provides a vehicle red light running detection device, such as Figure 8 FIG. 8 is a schematic diagram of a structure of a vehicle running a red light detection device according to an embodiment of the present application. The vehicle running a red light detection device 800 includes:

[0183] The acquisition unit 801 is used to acquire a video from the driving perspective of the vehicle to be detected, wherein the vehicle to be detected is equipped with a camera, and the camera is used to collect the video;

[0184] The acquisition unit 801 is further configured to acquire a capture time point of an i-th frame image in the video and the number of zebra crossing pixels contained in the i-th frame image, wherein the video includes at least N frames of images, i is a positive integer, 1≤i≤N;

[0185] A construction unit 802 is used to construct a change curve of the number of zebra crossing pixels corresponding to the vehicle to be detected according to the number of zebra crossing pixels;

[0186] A determination unit 803 is used to determine a target time point at which the vehicle to be detected passes through the zebra crossing according to the change curve and the acquisition time point;

[0187] A first detection unit 804 is used to detect the indication category of the traffic light in a first target image, wherein the first target image refers to an image in the N frames of images whose acquisition time point is the same as the target time point, and the indication category includes allowing passage and prohibiting passage;

[0188] The first detection unit 804 is further configured to determine whether the vehicle to be detected has run a red light according to the indication category.

[0189] In some embodiments of the present application, the determining unit 803 is specifically used to:

[0190] According to the change curve and the acquisition time point, determine a first acquisition time point when the number of pixels of the zebra crossing decreases over time until it reaches zero;

[0191] The first acquisition time point is used as the target time point.

[0192] In some embodiments of the present application, the determining unit 803 is specifically used to:

[0193] From the acquisition time points, obtain M consecutive acquisition time points after the first acquisition time point, where M is a positive integer greater than one;

[0194] The first acquisition time point and the M consecutive acquisition time points are used as the target time points.

[0195] In some embodiments of the present application, the determining unit 803 is specifically used to:

[0196] Determine, according to the change curve and the acquisition time point, a second acquisition time point when the number of pixels of the zebra crossing begins to decrease over time;

[0197] The second acquisition time point is used as the target time point.

[0198] In some embodiments of the present application, the vehicle red light running detection device 800 further includes a second detection unit (not shown in the figure). After the step of acquiring the video from the driving perspective of the vehicle to be detected, the second detection unit is specifically used to:

[0199] Acquire the three-axis angular velocity of the i-th frame image, wherein the three-axis angular velocity includes an x-axis angular velocity, a y-axis angular velocity, and a z-axis angular velocity, and the x-axis direction is consistent with the driving direction of the vehicle to be detected;

[0200] Determine a first direction curve of the vehicle to be detected according to the y-axis angular velocity, and determine a second direction curve of the vehicle to be detected according to the z-axis angular velocity;

[0201] Determining whether the vehicle to be detected is traveling in a straight line at a collection time point corresponding to the video according to the first direction curve and the second direction curve;

[0202] In some embodiments of the present application, the acquisition unit 801 is specifically used to:

[0203] When it is determined that the vehicle to be detected is traveling in a straight line within the acquisition time point corresponding to the video, the acquisition time point of the i-th frame image in the video and the number of zebra crossing pixels contained in the i-th frame image are obtained.

[0204] In some embodiments of the present application, the first detection unit 804 is specifically used to:

[0205] Performing classification processing on the first target image to obtain a first classification result of the first target image, wherein the first classification result is used to indicate whether there is a traffic light in the first target image and the type of lamp pole of the traffic light;

[0206] When the first classification result is that a traffic light exists in the first target image, the first target image is identified according to the lamp pole type to determine the indication category of the traffic light.

[0207] In some embodiments of the present application, the vehicle running a red light detection device 800 further includes a third detection unit (not shown in the figure). After the step of acquiring the video from the driving perspective of the vehicle to be detected, the third detection unit is specifically used to:

[0208] Classify each frame image in the video to obtain a second classification result of each frame image, wherein the second classification result is used to indicate whether the vehicle to be detected is near a traffic light intersection;

[0209] Detecting the number of second target images according to the second classification result, wherein the second target image refers to an image in each frame of images in which the classification result shows that the vehicle to be detected is near a traffic light intersection;

[0210] In some embodiments of the present application, the acquisition unit 801 is specifically used to:

[0211] When it is detected that the number of the second target images is greater than a preset number threshold, the acquisition time point of the i-th frame image in the video and the number of zebra crossing pixels contained in the i-th frame image are obtained.

[0212] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous method embodiments, which will not be repeated here.

[0213] Since the vehicle red light running detection device can execute the present application as follows Figures 1 to 7 Corresponding to the steps in the method for detecting a vehicle running a red light in any embodiment, the present application can be implemented as follows Figures 1 to 7 The beneficial effects that can be achieved by the method for detecting a vehicle running a red light in any embodiment are detailed in the previous description and will not be repeated here.

[0214] In addition, in order to better implement the method for detecting a vehicle running a red light in the embodiment of the present application, on the basis of the method for detecting a vehicle running a red light, the embodiment of the present application further provides an electronic device, referring to Fig. 9 , Fig. 9 A schematic diagram of the structure of an electronic device according to an embodiment of the present application is shown. Specifically, the electronic device provided by the embodiment of the present application includes a processor 901, and the processor 901 is used to execute a computer program stored in a memory 902 to implement the following Figures 1 to 7 Corresponding to the steps of the method for detecting a vehicle running a red light in any embodiment; or, the processor 901 is used to execute the computer program stored in the memory 902 to implement the following Figure 8 The functions of each unit in the corresponding embodiment.

[0215] Exemplarily, the computer program may be divided into one or more modules / units, one or more modules / units are stored in the memory 902, and executed by the processor 901 to complete the embodiment of the present application. One or more modules / units may be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the computer device.

[0216] The electronic device may include, but is not limited to, a processor 901 and a memory 902. Those skilled in the art will appreciate that the illustration is merely an example of an electronic device and does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the illustration, or a combination of certain components, or different components. For example, the electronic device may also include an input / output device, a network access device, a bus, etc., and the processor 901, the memory 902, the input / output device, and the network access device are connected via a bus.

[0217] The processor 901 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device.

[0218] The memory 902 can be used to store computer programs and / or modules. The processor 901 implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 902 and calling the data stored in the memory 902. The memory 902 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the electronic device (such as audio data, video data, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0219] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described vehicle red light running detection device, electronic device and its corresponding units can refer to the following. Figures 1 to 7 The description of the method for detecting a vehicle running a red light in any embodiment will not be repeated here in detail.

[0220] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0221] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the present application as follows: Figures 1 to 7 Corresponding to the steps in the method for detecting a vehicle running a red light in any embodiment, the specific operations can be referred to as follows Figures 1 to 7 The description of the method for detecting a vehicle running a red light in any embodiment will not be repeated here.

[0222] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0223] Due to the instructions stored in the computer-readable storage medium, the present application can be executed. Figures 1 to 7 Corresponding to the steps in the method for detecting a vehicle running a red light in any embodiment, the present application can be implemented as follows Figures 1 to 7 The beneficial effects that can be achieved by the method for detecting a vehicle running a red light in any embodiment are detailed in the previous description and will not be repeated here.

[0224] The above is a detailed introduction to a vehicle red light running detection method, device, electronic device and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for detecting a vehicle running a red light, characterized in that: The method comprises: Acquire a video from the driving perspective of the vehicle to be detected, wherein the vehicle to be detected is equipped with a camera, and the camera is used to collect the video; Obtaining the acquisition time point of the i-th frame image in the video and the number of zebra crossing pixels contained in the i-th frame image, wherein the video includes at least N frames of images, i is a positive integer, 1≤i≤N; According to the number of pixels of the zebra crossing, a change curve of the number of pixels of the zebra crossing corresponding to the vehicle to be detected is constructed; Determining a target time point at which the vehicle to be detected passes through the zebra crossing according to the change curve and the acquisition time point; Detecting the indication category of the traffic light in the first target image, wherein the first target image refers to an image in the N frames of images whose acquisition time point is the same as the target time point, and the indication category includes allowing passage and prohibiting passage; Determining whether the vehicle to be detected runs a red light according to the indication category; Determining the target time point at which the vehicle to be detected passes through the zebra crossing according to the change curve and the acquisition time point includes: According to the change curve and the acquisition time point, determine a first acquisition time point when the number of pixels of the zebra crossing decreases over time until it reaches zero; The first acquisition time point is used as the target time point.

2. The method for detecting a vehicle running a red light according to claim 1, characterized in that: The taking the first acquisition time point as the target time point includes: From the acquisition time points, obtain M consecutive acquisition time points after the first acquisition time point, where M is a positive integer greater than one; The first acquisition time point and the M consecutive acquisition time points are used as the target time points.

3. The method for detecting a vehicle running a red light according to claim 1, characterized in that: Determining the target time point at which the vehicle to be detected passes through the zebra crossing according to the change curve and the acquisition time point includes: Determine, according to the change curve and the acquisition time point, a second acquisition time point when the number of pixels of the zebra crossing begins to decrease over time; The second acquisition time point is used as the target time point.

4. The method for detecting a vehicle running a red light according to claim 1, characterized in that: The step of obtaining the video from the driving perspective of the vehicle to be detected further includes: Acquire the three-axis angular velocity of the i-th frame image, wherein the three-axis angular velocity includes an x-axis angular velocity, a y-axis angular velocity, and a z-axis angular velocity, and the x-axis direction is consistent with the driving direction of the vehicle to be detected; Determine a first direction curve of the vehicle to be detected according to the y-axis angular velocity, and determine a second direction curve of the vehicle to be detected according to the z-axis angular velocity; Determining whether the vehicle to be detected is traveling in a straight line at a collection time point corresponding to the video according to the first direction curve and the second direction curve; The acquiring the acquisition time point of the i-th frame image in the video and the number of zebra crossing pixels contained in the i-th frame image includes: When it is determined that the vehicle to be detected is traveling in a straight line within the acquisition time point corresponding to the video, the acquisition time point of the i-th frame image in the video and the number of zebra crossing pixels contained in the i-th frame image are obtained.

5. The method for detecting a vehicle running a red light according to claim 1, characterized in that: The detecting the indication category of the traffic light in the first target image includes: Performing classification processing on the first target image to obtain a first classification result of the first target image, wherein the first classification result is used to indicate whether there is a traffic light in the first target image and the type of lamp pole of the traffic light; When the first classification result is that a traffic light exists in the first target image, the first target image is identified according to the lamp pole type to determine the indication category of the traffic light.

6. The method for detecting a vehicle running a red light according to any one of claims 1 to 5, characterized in that: The step of obtaining the video from the driving perspective of the vehicle to be detected further includes: Classify each frame image in the video to obtain a second classification result of each frame image, wherein the second classification result is used to indicate whether the vehicle to be detected is near a traffic light intersection; Detecting the number of second target images according to the second classification result, wherein the second target image refers to an image in each frame of images in which the classification result shows that the vehicle to be detected is near a traffic light intersection; The acquiring the acquisition time point of the i-th frame image in the video and the number of pixels of the i-th frame image containing the zebra crossing includes: When it is detected that the number of the second target images is greater than a preset number threshold, the acquisition time point of the i-th frame image in the video and the number of zebra crossing pixels contained in the i-th frame image are obtained.

7. A vehicle red light running detection device, characterized in that: The vehicle red light running detection device comprises: An acquisition unit, used to acquire a video from the driving perspective of the vehicle to be detected, wherein the vehicle to be detected is equipped with a camera, and the camera is used to collect the video; The acquisition unit is further used to acquire the acquisition time point of the i-th frame image in the video and the number of zebra crossing pixels contained in the i-th frame image, wherein the video includes at least N frames of images, i is a positive integer, 1≤i≤N; A construction unit, used for constructing a change curve of the number of zebra crossing pixels corresponding to the vehicle to be detected according to the number of zebra crossing pixels; A determination unit, used to determine a target time point at which the vehicle to be detected passes through the zebra crossing according to the change curve and the acquisition time point; A first detection unit is used to detect the indication category of the traffic light in a first target image, wherein the first target image refers to an image in the N frames of images whose acquisition time point is the same as the target time point, and the indication category includes allowing passage and prohibiting passage; The first detection unit is further used to determine whether the vehicle to be detected runs a red light according to the indication category; The determining unit is further configured to: According to the change curve and the acquisition time point, determine a first acquisition time point when the number of pixels of the zebra crossing decreases over time until it reaches zero; The first acquisition time point is used as the target time point.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the method for detecting a vehicle running a red light as claimed in any one of claims 1 to 6 is executed.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the method for detecting a vehicle running a red light as described in any one of claims 1 to 6.

Images