Vehicle red light running behavior judgment method and system based on vehicle-mounted video

Abstract

The invention relates to a vehicle red light running behavior judgment method and system based on a vehicle-mounted video. The method comprises the following steps: based on a vehicle-mounted video, judging whether a zebra crossing is detected or not; if the zebra crossing is detected, traffic lights in the vehicle-mounted video are detected based on the optimized YOLOv8 network, and the location, the number and the color of the traffic lights are obtained; matching the traffic lights with the corresponding lanes according to the positioning and number of the traffic lights; determining the driving direction of the current vehicle based on the gray projection curves of the front and rear multi-frame images of the vehicle-mounted video; and judging whether the vehicle runs the red light or not based on the traffic light color corresponding to the current vehicle driving lane and the current vehicle driving direction. According to the method, vehicle red light running judgment can be carried out only through the vehicle-mounted video, the problem of binding of the traffic light and the vehicle is converted into the problem of binding of the traffic light and the driving direction of the vehicle by seeing the steering direction of the vehicle and then judging whether passing is permitted in the direction, and vehicle red light running behavior judgment is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle red light running behavior judgment, in particular to a vehicle red light running behavior judgment method and system based on vehicle video. BACKGROUND

[0002] With the development of society, in order to manage taxis or online taxis, taxi management platforms generally install vehicle videos or vehicle recorders in the vehicles. By analyzing and processing vehicle video data (vehicle recorder data), it is determined whether there is a vehicle violation behavior in the video, so as to punish the violation behavior and strengthen the supervision of vehicle violation behavior, and create a good traffic environment.

[0003] With the rapid development and popularization of deep learning and computer vision, video understanding has achieved good results in video action recognition, time sequence action positioning and other fields. At the same time, target detection, image classification and other technologies have become mature, which makes it possible to detect vehicle violations based on vehicle video and vehicle recorder. This method has the characteristics of full automation, high coverage, high efficiency, all-weather, low cost. As for the selection of monitoring video and vehicle recorder, monitoring video needs to install fixed monitoring equipment on the road, and is connected to the traffic management system, which is high in cost and difficult to effectively expand. SUMMARY

[0004] Therefore, it is necessary to provide a vehicle red light running behavior judgment method and system based on vehicle video in view of the above technical problems.

[0005] The first aspect of the present application provides a vehicle red light running behavior judgment method based on vehicle video, comprising the following steps: Based on the vehicle video, it is determined whether a zebra crossing is detected; If the zebra crossing is detected, the red light in the vehicle video is detected based on the optimized YOLOv8 network to obtain the positioning, quantity and color of the red light; According to the positioning and quantity of the red light, the red light is matched with the corresponding lane; Based on the gray scale projection curve of the front and rear frames of images of the vehicle video, the driving direction of the current vehicle is determined; Based on the color of the red light corresponding to the driving lane of the current vehicle and the driving direction of the current vehicle, it is determined whether the vehicle has a red light running behavior.

[0006] Further, the optimized YOLOv8 network comprises: The first two C2f modules in the backbone network of the YOLOv8 network are replaced with Swin-transformer modules, and a residual connection is reserved, each Swin-transformer module includes n serial Linear Embedding operations and Swin-Transformer Blocks; A new up-sampling block and a new down-sampling block are added in the neck of the YOLOv8 network, which are used to fuse the output of the first Swin-transformer module with the output of the high layer. A small target detection head is added in the head of the YOLOv8 network, and the feature map obtained by fusing the output of the first Swin-transformer module with the output of the high layer is input into the small target detection head.

[0007] Further, according to the positioning and quantity of the traffic lights, the traffic lights are matched with the corresponding lanes, including: When the number of traffic lights is 0, the traffic lights do not need to be matched with the corresponding lanes; When the number of traffic lights is 1, the traffic lights are matched with all lanes; When the number of traffic lights is 2, the traffic light located on the left side of the image is matched with the left-turn lane, and the traffic light located on the right side of the image is matched with the straight / right-turn lane; When the number of traffic lights is 3, the traffic light located on the left side of the image is matched with the left-turn lane, the traffic light located in the middle of the image is matched with the straight lane, and the traffic light located on the right side of the image is matched with the right-turn lane.

[0008] Further, the gray projection curve of the front and rear multiple images of the vehicle-mounted video includes: When a zebra crossing is detected, the vertical projection calculation is performed on each frame of image of the vehicle-mounted video, and the pixel brightness average is calculated along the X-axis direction to obtain the gray projection curve.

[0009] Further, the gray projection curve of the front and rear multiple images of the vehicle-mounted video further includes: The gray projection curve is divided into left and right independent analysis areas along the image central axis to obtain left and right gray projection curves; The left and right gray projection curves are subjected to adaptive threshold segmentation and binarization processing to obtain left and right projection histograms.

[0010] Further, the driving direction of the current vehicle is determined, including: According to the left and right projection histograms of each frame of image of the vehicle-mounted video, the displacement difference of the center of the brightness region on the X-axis is determined and ; When any of the following conditions is met, it is determined that the driving direction of the current vehicle is right turn:

[0011]

[0012] When any of the following conditions is met, it is determined that the driving direction of the current vehicle is left turn:

[0013]

[0014] When all the above conditions are not met, it is determined that the driving direction of the current vehicle is straight ahead; wherein, represents a minimum displacement difference threshold, represents a maximum displacement difference threshold.

[0015] Further, determining the driving direction of the current vehicle further comprises a steering activation condition, a state maintenance condition and a straight ahead recovery condition, wherein: The steering activation condition is: When the steering determination condition is continuously met for a first number of frames, a state transition is triggered; The state maintenance condition is: The single frame condition is not met, but must be recovered to meet the condition within a subsequent second number of frames, otherwise a state rollback is triggered; The straight ahead recovery condition is: The straight ahead determination condition is met for a third number of consecutive frames.

[0016] The second aspect of the application provides a vehicle red light running behavior judgment system based on vehicle-mounted video, comprising: A first judgment module, the first judgment module judges whether a zebra crossing is detected based on the vehicle-mounted video; A first detection module, the first detection module detects a red light in the vehicle-mounted video based on the optimized YOLOv8 network if the zebra crossing is detected, to obtain the positioning, number and color of the red light; A first matching module, the first matching module matches the red light with the corresponding lane according to the positioning and number of the red light; A first determination module, the first determination module determines the driving direction of the current vehicle based on the gray projection curves of the front and rear multiple images of the vehicle-mounted video; A second judgment module, the second judgment module judges whether the vehicle has a red light running behavior based on the color of the red light corresponding to the driving lane of the current vehicle and the driving direction of the current vehicle.

[0017] The third aspect of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements the steps of the vehicle red-light-running behavior judgment method based on vehicle-mounted video.

[0018] The fourth aspect of the present application provides a computer storage medium, wherein the computer storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps of the vehicle red-light-running behavior judgment method based on vehicle-mounted video.

[0019] Compared with the prior art, the present application has the following advantages: The vehicle red-light-running behavior judgment method based on vehicle-mounted video of the present application first judges whether a zebra crossing is detected based on vehicle-mounted video; if a zebra crossing is detected, the red light is detected in the vehicle-mounted video based on the optimized YOLOv8 network to obtain the positioning, quantity and color of the red light; the red light is matched with the corresponding lane according to the positioning and quantity of the red light; the driving direction of the current vehicle is determined based on the gray projection curve of the front and rear multiple images of the vehicle-mounted video; and whether the vehicle has a red-light-running behavior is judged based on the color of the red light corresponding to the driving lane of the current vehicle and the driving direction of the current vehicle. The present application only relies on vehicle-mounted video to judge the red-light-running behavior of the vehicle, that is, a pure visual method is used to solve the problem of binding the vehicle with the red light and the judgment of the color of the red light. The detection of small-sized red light is realized through the optimized YOLOv8 network to obtain the state of the red light. Then, by looking at the turning direction of the vehicle first and then judging whether the direction is allowed to pass, the problem of binding the vehicle with the red light is converted into the problem of binding the vehicle with the driving direction of the red light to realize the judgment of the red-light-running behavior of the vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A flowchart of a vehicle red-light-running behavior judgment method based on vehicle-mounted video provided for an embodiment of the present application; Figure 2 A network framework diagram of an optimized YOLOv8 network provided for an embodiment of the present application; Figure 3 A loss curve diagram of an optimized YOLOv8 network provided for an embodiment of the present application; Figure 4 A P-R curve diagram of a YOLOv8 network provided for an embodiment of the present application; Figure 5 A P-R curve diagram of an optimized YOLOv8 network provided for an embodiment of the present application; Figure 6 An example diagram of recognizing a red light using an optimized YOLOv8 network provided for an embodiment of the present application; Figure 7 One of the vehicle-mounted video image examples in vehicle driving provided by the embodiment of the present application; Figure 8 The corresponding gray scale projection curve schematic diagram provided by the embodiment of the present application; Figure 7 The corresponding gray scale projection curve schematic diagram provided by the embodiment of the present application; Figure 9 The second vehicle-mounted video image example in vehicle driving provided by the embodiment of the present application; Figure 10 The corresponding gray scale projection curve schematic diagram provided by the embodiment of the present application; Figure 9 The corresponding gray scale projection curve schematic diagram provided by the embodiment of the present application; Figure 11 The third vehicle-mounted video image example in vehicle driving provided by the embodiment of the present application; Figure 12 The corresponding gray scale projection curve schematic diagram provided by the embodiment of the present application; Figure 11 The corresponding gray scale projection curve schematic diagram provided by the embodiment of the present application; Figure 13 The fourth vehicle-mounted video image example in vehicle driving provided by the embodiment of the present application; Figure 14 The corresponding gray scale projection curve schematic diagram provided by the embodiment of the present application; Figure 13 The corresponding gray scale projection curve schematic diagram provided by the embodiment of the present application; Figure 15 The fifth vehicle-mounted video image example in vehicle driving provided by the embodiment of the present application; Figure 16 The corresponding gray scale projection curve schematic diagram provided by the embodiment of the present application; Figure 15 The corresponding gray scale projection curve schematic diagram provided by the embodiment of the present application; Figure 17 The module schematic diagram of a vehicle red light running behavior judgment system based on vehicle-mounted video provided by the embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0022] The basic judgment logic for the current vehicle red light running behavior is that the vehicle drives into the intersection under the red light state, so two aspects of information need to be detected: one is the binding of the vehicle and the red light, that is, there are multiple red lights at the intersection, and the vehicle needs to see which red light; the other is the color state of the red light bound with the vehicle, if the color is red, then the vehicle red light running behavior is established. The vehicle red light running judgment of the embodiment of the present application only relies on the vehicle video, that is, the problem of binding the vehicle and the red light and the judgment of the color of the red light are solved by using a pure visual method, and the key lies in how to solve the problem of binding the vehicle and the red light. The embodiment of the present application uses reverse thinking to solve the above problem, which is transformed from the normal "first see the light and then judge the permission to pass the state and direction" to "first see the vehicle turning direction, and then judge whether the direction is permitted to pass", so the problem of binding the red light and the vehicle is transformed into the problem of binding the red light and the driving direction of the vehicle.

[0023] The embodiment of the present application provides a vehicle red light running behavior judgment method based on a vehicle video, as shown in the formula (I): Figure 1 The embodiment of the present application provides a vehicle red light running behavior judgment method based on a vehicle video, as shown in the formula (I): Based on the vehicle video, whether the zebra crossing is detected is judged; If the zebra crossing is detected, the red light in the vehicle video is detected based on the optimized YOLOv8 network to obtain the positioning, quantity and color of the red light; According to the positioning and quantity of the red light, the red light is matched with the corresponding lane; Based on the gray projection curve of the front and rear multiple images of the vehicle video, the driving direction of the current vehicle is determined; Based on the color of the red light corresponding to the driving lane of the current vehicle and the driving direction of the current vehicle, whether the vehicle has a red light running behavior is judged.

[0024] Firstly, based on the vehicle video, whether the zebra crossing is detected is judged; if the zebra crossing is detected, the red light in the vehicle video is detected based on the optimized YOLOv8 network to obtain the positioning, quantity and color of the red light; according to the positioning and quantity of the red light, the red light is matched with the corresponding lane; based on the gray projection curve of the front and rear multiple images of the vehicle video, the driving direction of the current vehicle is determined; based on the color of the red light corresponding to the driving lane of the current vehicle and the driving direction of the current vehicle, whether the vehicle has a red light running behavior is judged. The vehicle red light running judgment of the present application only relies on the vehicle video, that is, the problem of binding the vehicle and the red light and the judgment of the color of the red light are solved by using a pure visual method, through the optimized YOLOv8 network, the detection of the small size red light is realized, the state of the red light is obtained, then by first seeing the vehicle turning direction and then judging whether the direction is permitted to pass, the problem of binding the red light and the vehicle is transformed into the problem of binding the red light and the driving direction of the vehicle, and the vehicle red light running behavior judgment is realized.

[0025] In a further embodiment, the corresponding vehicle-mounted video (driving recorder video) is obtained from the system terminal, since the video is usually as long as several hours, it is unrealistic to input all the videos at one time, and at the same time, based on the actual purpose of the embodiment of the application, in order to save computing resources, the application divides the video into n short video segments, and frames the video at fixed intervals, and finally inputs to the network to extract the traffic light and vehicle driving direction information, through random sampling of operating vehicles, random sampling of operating time periods, and interval frame extraction, the detection rate is guaranteed to the maximum extent while reducing the memory and computing resource overhead.

[0026] In a further embodiment, the embodiment of the application uses an optimized YOLOv8 network for traffic light positioning and color discrimination, YOLOv8 has three default detection heads, when the input image size is 640*640, the three detection heads are used to detect targets above 8*8, 16*16, 32*32 respectively, and the detection accuracy for small targets such as traffic lights is relatively low, to solve this problem, the embodiment of the application first adds a small target detection head in the lower layer of YOLOv8, and replaces the C2f module in the lower layer of the backbone network with a Swin-transformer; as shown in Figure 2 The optimized YOLOv8 network includes: In order to enhance the network's ability to detect small targets, the C2f module in the p2 and p3 of the backbone network of the YOLOv8 network is replaced with a Swin-transformer module, and a residual connection is retained, each Swin-transformer module includes n serial Linear Embedding operations and Swin-Transformer Blocks; The original network structure does not participate in the fusion of feature maps at p2, the network only takes P4, P5, P6 three size feature maps, which are 80*80, 40*40, 20*20, which correspond to 8*8, 16*16, 32*32 three size target detection heads. Since the small target (traffic light) in the application is usually below 8*8, in order to better detect such small targets, a new up-sampling block and a down-sampling block are added in the neck of the YOLOv8 network, which are used to fuse the output of the first Swin-transformer module with the output of the high layer, so that the output (P2) of the low layer can also have the feature representation of the high layer; A small target detection head is added in the head of the YOLOv8 network, and the output of the first Swin-transformer module is input into the small target detection head after feature fusion with the output of the high layer to obtain the output of the classification task and the regression task.

[0027] In specific embodiments, the small target detection result of the optimized YOLOv8 network is analyzed, specifically: The embodiments of the present application analyze the results of the YOLOv8 network and the optimized YOLOv8 network on the open-source traffic light dataset Traffic Lights Domestic Dataset. The results show that the optimized YOLOv8 network is significantly improved, with an mAP50 of 5.9% higher than that of the YOLOv8 network, and an mAP50 of 9.25% higher than that of the YOLOv8 network for classes with sufficient samples. In specific experiments, in order to ensure fairness, the YOLOv8 network and the optimized YOLOv8 network use the same parameters on the traffic light dataset, and are run for 100 rounds on the training set, and then the validation set is used for testing. The experimental parameters are shown in Table 1: Table 1 Experimental parameters

[0028] Figure 3 is the loss curve of the YOLOv8 network. As can be seen, with the increase of the training rounds, the regression frame loss and the classification loss are gradually decreasing, and gradually converge at the 100th round, reaching a stable state.

[0029] Figure 4 and Figure 5 respectively represent the P-R curves of the YOLOv8 network and the optimized YOLOv8 network. As can be seen from the figure, the mAP50 of all classes of the two networks is 0.547 and 0.606 respectively, and the optimized YOLOv8 network is 5.9% higher than the YOLOv8 network, with a significant improvement effect. It is not difficult to see that due to the imbalance of the classes in the data and the large data demand of the Swin-transformer, the value of mAP50 is actually lowered by False and Yellow, and in the case of sufficient data, i.e. Red and Green, the AP of the optimized YOLOv8 network is 8.5% and 10% higher than that of the YOLOv8 network respectively, with an average of 9.25%, with a significant improvement.

[0030] In further embodiments, after the optimized YOLOv8 network identifies the traffic light information, a post-processing step is performed on the traffic light information, including: The traffic light information is the main basis for judging whether the vehicle appears "running a red light", and the traffic light related information mainly has two, one is the color information of the traffic light, and the other is the position information of the traffic light. The traffic light data structure template is designed, and the detected traffic light data is put into the template for subsequent processing, and the structure template is shown in Table 2.

[0031] Table 2 Traffic light data structured template

[0032] In a further embodiment, the traffic light is matched with the corresponding lane according to the positioning and number of the traffic light, comprising: When the number of traffic lights is 0, the traffic light does not need to be matched with the corresponding lane; When the number of traffic lights is 1, the traffic light is matched with all lanes; When the number of traffic lights is 2, the traffic light located on the left side of the image is matched with the left turn lane, and the traffic light located on the right side of the image is matched with the straight / right turn lane; When the number of traffic lights is 3, the traffic light located on the left side of the image is matched with the left turn lane, the traffic light located in the middle of the image is matched with the straight lane, and the traffic light located on the right side of the image is matched with the right turn lane.

[0033] In a specific embodiment, the application adopts the method of first judging the turning direction of the vehicle, and then judging the state of the traffic light in the direction to judge whether the vehicle has the behavior of "running a red light", based on this idea, in the post-processing part, the relative position of the traffic light is calculated based on the relative position of the traffic light in this frame and the number of traffic lights, so as to judge the passing direction of the corresponding lane of the traffic light, and the traffic light is indirectly matched with the lane, as shown in Table 3.

[0034] Table 3 Permitted passing direction calculation logic

[0035] In a further embodiment, an improved YOLOv8 model is used for real-time signal light detection, and the target position, color state and confidence are output, a cross-frame backtracking matching mechanism is proposed: based on IOU similarity calculation (threshold value ≥0.4), the history frame backtracking search (maximum backtracking depth n=5 frames) is performed on the unmatched target, and a unique identification ID is dynamically allocated.

[0036] In a further embodiment, a signal light group intelligent clustering algorithm is proposed, specifically: Frequency filtering: eliminating interference targets with a total appearance frequency of less than 30% and a relative frequency of less than 60%; A Manhattan distance matrix is constructed, and a hierarchical clustering algorithm (threshold value δ=50 pixels) is used to generate a light group cluster; Introducing the central axis priority strategy: selecting the light group closest to the geometric center of the image vertical central axis as the master control signal light group.

[0037] In further embodiments, a state verification and completion mechanism is proposed, specifically: The sliding window interpolation method is used for the missed detection frame to perform spatial compensation on the historical position mean value. Establish a weighted voting model: give time decay weight to the detection results of consecutive k frames, and select the highest weighted vote number as the final output.

[0038] Figure 6 The example diagram of the traffic light set and the traffic light group recognition result is shown, wherein the light group 1 is closest to the geometric center of the picture, so it is used as the master control light group.

[0039] The next embodiment detects the driving direction of the vehicle through the gray projection curve of the front and rear images of the vehicle-mounted video. The specific principle is: from the vehicle record video, it can be seen that the scene around the vehicle is continuously moving during driving, so the gray level of the image is also continuously changing. However, the gray level changes of straight vehicles, left-turn vehicles and right-turn vehicles are different, and the physical phenomenon basis is: When the vehicle turns, the camera view angle changes due to the deflection of the vehicle body, and the distribution of the background high brightness area (such as the sky and the roof) in the image presents a regular shift, as shown in Figures 7 to 10 .

[0040] As shown in Figure 8 and Figure 10 , the black and white parts are the projection of the brightness in the x-axis direction. As can be seen from the figure, the part with a high proportion of sky has a higher peak value, and the dark part usually serves as a valley value. When the vehicle moves, the projection curve shifts regularly, which is specifically manifested as: dividing the image into left and right parts, when the vehicle is driving straight, the peak points and valley points of the left and right parts shift to the left and right respectively; when the vehicle turns left, the peak points and valley points of the left and right parts shift to the right respectively; when the vehicle turns right, the peak points and valley points of the left and right parts shift to the left respectively.

[0041] In further embodiments, the gray level projection curve of the front and rear images of the vehicle-mounted video includes: When a zebra crossing is detected, the vertical projection calculation is performed on each frame of the front and rear images of the vehicle-mounted video, the pixel brightness mean value is calculated column by column along the X-axis direction, and the gray level projection curve is obtained.

[0042] In further embodiments, the gray level projection curve of the front and rear images of the vehicle-mounted video further includes: The gray level projection curve is divided into left and right independent analysis areas along the image central axis, and left and right gray level projection curves are obtained. Adaptive threshold segmentation and binarization are performed on the left and right grayscale projection curves to obtain the left and right projection histograms.

[0043] In a further embodiment, the adaptive threshold segmentation specifically refers to: ,

[0044] In the formula, This represents the 75th percentile of 15 historical projected data frames. , The dynamic thresholds for the left and right parts are respectively, and the image after adaptive thresholding is as follows: Figure 11 and 12 As shown; Taking the left analysis region as an example, the binarization process is specifically as follows:

[0045] In the formula, This represents the grayscale projection curve of the left analysis region. The image after binarization of the projection histogram of the left analysis region is shown below. Figure 13 and Figure 14 As shown, binarization can preserve high-brightness areas such as the sky and lights, while eliminating interference signals such as road surface reflections. As can be seen from the figure, the high-brightness areas are segmented by the low-brightness areas. After binarization, it is convenient to track the movement of the high-brightness areas on the x-axis, thereby analyzing the vehicle's turning direction.

[0046] In a further embodiment, a continuous frame signal association algorithm is proposed, A. performing temporal superposition analysis of 15 consecutive frames of binarized signals, specifically as follows: Define a continuous non-zero region for a signal segment and calculate the spatial overlap between adjacent frame signal segments:

[0047] In the formula, The spatial overlap of adjacent frame signal segments. for Binarized projection histogram at time points; Construct the longest consecutive associated signal chain, such that the signal chain satisfies:

[0048] Only the longest signal chain (n≥5) that meets the conditions is retained as a valid analysis object, such as... Figure 15 , 16 The image shown is a superimposed diagram of 15 consecutive frames of histogram signals. Figure 16 From top to bottom, the frames are t to t+15. As can be seen from the figure, when the vehicle turns right, the brightness areas on both sides tend to shift to the left.

[0049] In further embodiments, determining the driving direction of the current vehicle comprises: determining the displacement difference of the center of the brightness region on the X-axis according to the left and right projection histograms of each frame of the front and rear images of the vehicle-mounted video and ; when any of the following conditions is met, determining that the driving direction of the current vehicle is right turn:

[0050]

[0051] when any of the following conditions is met, determining that the driving direction of the current vehicle is left turn:

[0052]

[0053] when all the above conditions are not met, determining that the driving direction of the current vehicle is straight ahead; wherein, represents the minimum displacement difference threshold, represents the maximum displacement difference threshold.

[0054] In this embodiment, a three-level conditional judgment system is established:

[0055] In the table, and can be calibrated by the following method: Minimum displacement difference threshold : determined by experiment, taking 3 times the standard deviation of the displacement difference fluctuation range in the straight ahead state, ; Maximum displacement difference threshold : based on extreme scene calibration, taking =3 =150px; In specific embodiments, the displacement difference can be obtained by the following steps: Define the displacement difference of the t-th frame in the continuous frame sequence as:

[0056]

[0057] In the formula, is the left region signal center displacement difference, is the right region signal center displacement difference; Construct a window cumulative feature quantity:

[0058]

[0059] In a further embodiment, the determining the driving direction of the current vehicle further comprises a steering activation condition, a state maintaining condition and a straight driving recovery condition, wherein: The steering activation condition is: When the steering determination condition is continuously satisfied for 5 frames (about 0.2 seconds @ 25fps), a state transition is triggered; The state maintaining condition is: The single frame condition is not satisfied, but must be recovered to satisfy the condition within the next 2 frames, otherwise the state rollback is triggered; The straight driving recovery condition is: The straight driving determination condition is satisfied for 7 consecutive frames (about 0.28 seconds @ 25fps) to prevent false positives caused by temporary lane deviation.

[0060] The second embodiment of the present application provides a vehicle red light running behavior judgment system based on vehicle-mounted video, as shown in Figure 17 The second embodiment of the present application provides a vehicle red light running behavior judgment system based on vehicle-mounted video, as shown in The first determination module determines the driving direction of the current vehicle based on the gray projection curves of the front and rear multiple images of the vehicle-mounted video. The first detection module detects the zebra crossing based on the vehicle-mounted video, and detects the red light based on the optimized YOLOv8 network to obtain the positioning, quantity and color of the red light. The first matching module matches the red light with the corresponding lane according to the positioning and quantity of the red light. The first determination module determines the driving direction of the current vehicle based on the gray projection curves of the front and rear multiple images of the vehicle-mounted video. The second determination module determines whether the vehicle has a red light running behavior based on the color of the red light corresponding to the driving lane of the current vehicle and the driving direction of the current vehicle.

[0061] The third embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program, and the computer program is executed by the processor to realize the steps of the vehicle red light running behavior judgment method based on vehicle-mounted video.

[0062] The fourth embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a computer program, and the computer program is executed by a processor to realize the steps of the vehicle red light running behavior judgment method based on vehicle-mounted video.

[0063] The same or similar reference numerals mean the same or similar components; The terms describing the positional relationship in the drawings are used only for illustrative purposes and should not be construed as limiting the present patent; Obviously, the above-mentioned embodiments of the present application are merely exemplary and are not intended to limit the implementation of the present application. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, it is not necessary and impossible to exhaust all the implementations. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the claims of the present application.

Claims

1. A vehicle red-light running behavior judgment method based on vehicle-mounted video, characterized in that, The method comprises the following steps: Based on the vehicle-mounted video, it is judged whether a zebra crossing is detected; If a zebra crossing is detected, the red and green lights in the vehicle-mounted video are detected based on the optimized YOLOv8 network to obtain the positioning, number and color of the red and green lights; According to the positioning and number of the red and green lights, the red and green lights are matched with the corresponding lanes; Based on the gray-scale projection curves of the front and rear multiple images of the vehicle-mounted video, the driving direction of the current vehicle is determined; Based on the color of the red and green lights corresponding to the driving lane of the current vehicle and the driving direction of the current vehicle, it is judged whether the vehicle has a red light running behavior.

2. The vehicle red-light running behavior judgment method based on vehicle video according to claim 1, characterized in that, The optimized YOLOv8 network comprises: The first two C2f modules in the backbone network of the YOLOv8 network are replaced with Swin-transformer modules, and residual connections are reserved, each Swin-transformer module comprising n serial Linear Embedding operations and Swin-Transformer Blocks; New up-sampling blocks and down-sampling blocks are added to the neck of the YOLOv8 network for feature fusion of the output of the first Swin-transformer module and the output of the high layer; A small target detection head is added to the head of the YOLOv8 network, and the feature map obtained by feature fusion of the output of the first Swin-transformer module and the output of the high layer is input into the small target detection head.

3. The vehicle red-light running behavior judgment method based on vehicle video according to claim 1, characterized in that, According to the positioning and number of the red and green lights, the red and green lights are matched with the corresponding lanes, comprising: When the number of red and green lights is 0, the red and green lights do not need to be matched with the corresponding lanes; When the number of red and green lights is 1, the red and green lights are matched with all lanes; When the number of red and green lights is 2, the red and green light located on the left side of the image is matched with the left-turn lane, and the red and green light located on the right side of the image is matched with the straight / right-turn lane; When the number of red and green lights is 3, the red and green light located on the left side of the image is matched with the left-turn lane, the red and green light located in the middle of the image is matched with the straight lane, and the red and green light located on the right side of the image is matched with the right-turn lane.

4. The vehicle red-light running behavior judgment method based on vehicle video according to claim 1, characterized in that, The gray-scale projection curves of the front and rear multiple images of the vehicle-mounted video comprise: When a zebra crossing is detected, vertical projection calculation is performed on each frame of image of the vehicle-mounted video, the pixel brightness mean value is calculated column by column along the X-axis direction to obtain the gray-scale projection curve.

5. The vehicle red-light-running behavior judgment method based on in-vehicle video according to claim 4, characterized in that, The gray-scale projection curves of the front and rear multiple images of the vehicle-mounted video further comprise: The gray-scale projection curves are divided into left and right independent analysis areas along the image central axis to obtain left and right gray-scale projection curves; The left and right gray-scale projection curves are subjected to adaptive threshold segmentation and binarization processing to obtain left and right projection histograms.

6. The vehicle red-light-running behavior judgment method based on in-vehicle video according to claim 5, characterized in that, Determining the driving direction of the current vehicle comprises: According to the left and right projection histograms of each frame image of the front and back of the vehicle-mounted video, the displacement difference of the center of the brightness region on the X axis is determined and ; When any of the following conditions is met, it is determined that the driving direction of the current vehicle is right turn: When any of the following conditions is met, it is determined that the driving direction of the current vehicle is left turn: When all the above conditions are not met, it is determined that the driving direction of the current vehicle is straight; wherein, represents a minimum displacement difference threshold value, represents a maximum displacement difference threshold value.

7. The vehicle red-light running behavior judgment method based on vehicle video according to claim 5, characterized in that, Determining the driving direction of the current vehicle further comprises a turning activation condition, a state maintaining condition and a straight recovery condition, wherein: The turning activation condition is: When it is detected that the turning determination condition is continuously satisfied for a first number of frame counts, a state transition is triggered; The state maintaining condition is: The single-frame condition is not satisfied, but must be recovered to satisfy the condition within a subsequent second number of frame counts, otherwise a state rollback is triggered; The straight-ahead recovery condition is: The straight-ahead determination condition is satisfied for a continuous third number of frame counts.

8. A vehicle red-light running behavior judgment system based on vehicle-mounted video, characterized in that, Comprise: A first determination module determines whether a zebra crossing is detected based on the vehicle-mounted video; A first detection module detects the position, number and color of the traffic light in the vehicle-mounted video based on the optimized YOLOv8 network if the zebra crossing is detected; A first matching module matches the traffic light with the corresponding lane according to the position and number of the traffic light; A first determination module determines the driving direction of the current vehicle based on the gray projection curves of the front and rear images of the vehicle-mounted video; A second determination module determines whether the vehicle has a red light running behavior based on the color of the traffic light corresponding to the driving lane of the current vehicle and the driving direction of the current vehicle.

9. An electronic device, comprising: The computer program is stored in the computer storage medium, and when the computer program is executed by the processor, the steps of the vehicle red light running behavior judgment method based on the vehicle-mounted video in any one of claims 1 to 7 are realized.

10. A computer storage medium, characterized in that, The computer program is stored in the computer storage medium, and when the computer program is executed by the processor, the steps of the vehicle red light running behavior judgment method based on the vehicle-mounted video in any one of claims 1 to 7 are realized.

Images