Vehicle congestion detection method and device, equipment and storage medium

Abstract

The invention provides a vehicle congestion detection method and device, equipment and a storage medium. Comprising the steps of obtaining a current frame detection frame picture based on a current frame monitoring picture; the current frame detection frame picture comprises a plurality of target detection frames; performing clustering analysis on the plurality of target detection frames by adopting a preset distance clustering strategy to obtain a preprocessing clustering result; the preset distance clustering strategy is a strategy for clustering distances between adjacent vehicles; the vehicle corresponding to the pre-processing detection frame identifier may be a congested vehicle; tracking the vehicles in the possible congestion state to obtain a next frame of detection frame picture corresponding to the vehicles in the possible congestion state, and determining a secondary clustering result of a moment corresponding to the next frame of detection frame picture based on a preset position and direction clustering strategy and a preset distance clustering strategy; the preset position and direction clustering strategy is a strategy for clustering the driving position and direction of each vehicle; and determining that the vehicles on the trunk road are in a congested state based on a secondary clustering result and a preset time length.

Description

Technical Field

[0001] The present application relates to data processing technology, and in particular to a method, device, equipment and storage medium for detecting vehicle congestion. Background Art

[0002] For the convenience of society and life, vehicles have become a necessary means of transportation for users to travel. A large number of vehicles may cause congestion when driving on the trunk roads, so it is necessary to detect congestion of vehicles driving on the trunk roads to facilitate traffic evacuation by traffic sorting personnel.

[0003] In the prior art, after the monitoring system collects the monitoring video, the monitoring system determines the target vehicle based on the monitoring video, as well as the front and rear monitoring images of the target vehicle, thereby calculating the movement value to determine that the target vehicle is a slow-moving vehicle, and further determines that the target vehicle is in a congested state.

[0004] However, the monitoring system determines the congestion state by using the movement value of the target vehicle, which does not conform to the actual congestion situation. At the same time, the target vehicle is determined to be in a congestion state if it is determined to be a slow-moving vehicle, but there is also the possibility that the target vehicle itself is moving slowly on the trunk road, so the accuracy is low. Summary of the invention

[0005] The present application provides a vehicle congestion detection method, device, equipment and storage medium, which are used to solve the problems of not conforming to the actual congestion situation and low accuracy.

[0006] In a first aspect, the present application provides a method for detecting vehicle congestion, the method comprising:

[0007] receiving a current frame monitoring picture on a trunk road in real time, and obtaining a current frame detection frame picture based on the current frame monitoring picture; the current frame detection frame picture includes a plurality of target detection frames; the target detection frame surrounds the vehicle in the current frame monitoring picture; the detection frame is a rectangular frame;

[0008] A preset distance clustering strategy is used to perform cluster analysis on the multiple target detection frames to obtain a preprocessing clustering result; the preprocessing clustering result includes a preprocessing detection frame identifier that meets the preset preprocessing clustering condition; the preset distance clustering strategy is a strategy for clustering the distance between adjacent vehicles; the vehicle corresponding to the preprocessing detection frame identifier may be a vehicle in a congested state; the distance between adjacent vehicles is the straight-line distance between the center points of the two target detection frames corresponding to the adjacent vehicles;

[0009] Track the vehicle that may be in a congested state to obtain a next frame detection frame image corresponding to the vehicle that may be in a congested state, and determine a secondary clustering result of the next frame detection frame image at a corresponding moment based on a preset position and direction clustering strategy and a preset distance clustering strategy; the preset position and direction clustering strategy is a strategy for clustering the driving position and direction of each vehicle;

[0010] Based on the secondary clustering result and the preset duration, it is determined that the vehicles on the trunk road are in a congested state.

[0011] In one embodiment, obtaining a current frame detection frame picture based on the current frame monitoring picture includes:

[0012] The current frame monitoring picture is input into the vehicle detection model trained to convergence to obtain the current frame detection frame picture; the vehicle detection model trained to convergence is obtained based on the training sample set; the training sample set includes an original training sample picture set and an expanded training sample picture set; the original training sample picture set includes at least one training sample picture; the expanded training sample picture set includes at least one expanded training sample picture; the training sample picture includes a labeled training detection frame; the labeled training detection frame surrounds the training vehicle in the training sample picture; the labeled training detection frame is a rectangular frame; the expanded training sample picture is a sample picture generated based on the original training sample picture and taking into account multiple preset vehicle occlusion methods.

[0013] In one embodiment, the clustering analysis of the plurality of target detection frames using a preset distance clustering strategy to obtain a preprocessing clustering result includes:

[0014] When executing the preset distance clustering strategy, determining the target detection frame with the largest area from multiple target detection frames, and obtaining the target detection frame width and target detection frame height corresponding to the target detection frame with the largest area;

[0015] The maximum value among the values ​​corresponding to the target detection frame width and the target detection frame height is determined as the initial distance threshold;

[0016] A preprocessing clustering result is obtained based on the initial distance threshold.

[0017] In one embodiment, obtaining a preprocessing clustering result based on the initial distance threshold includes:

[0018] Determine each target detection frame as a central detection frame, and determine its adjacent target adjacent detection frame based on the central detection frame, and make the straight-line distance between the center point of the target adjacent detection frame and the center point of the central detection frame less than or equal to the initial distance threshold;

[0019] Determine a cluster set to be selected; the cluster set to be selected includes a center detection box identifier and its adjacent target adjacent detection box identifiers;

[0020] Continue to determine each of the target adjacent detection frames as a central detection frame, and determine target adjacent detection frames adjacent to the central detection frame based on the central detection frame;

[0021] Adding the newly determined target adjacent detection box identifier to the to-be-selected cluster set, and continuing to perform the steps of determining each target adjacent detection box as a central detection box to adding the newly determined target adjacent detection box identifier to the to-be-selected cluster set, until there is no newly determined target adjacent detection box, so as to obtain a final to-be-selected cluster set corresponding to each target detection box as a central detection box;

[0022] A preprocessing clustering result is determined based on the final set of clusters to be selected.

[0023] In one embodiment, determining the preprocessing clustering result based on the final set of clusters to be selected includes:

[0024] Determining the target detection frame corresponding to each target detection frame identifier from the final to-be-selected cluster set as a reference detection frame;

[0025] Determine a distance range detection frame within a preset distance range of the reference detection frame, and determine a near detection frame width or near detection frame height of the nearest distance range detection frame as an update distance threshold; the distance range detection frame is at least one; the update distance threshold is the maximum value of corresponding values ​​of the near detection frame width and the near detection frame height;

[0026] If the straight-line distances between the reference detection frame and each of the distance range detection frames are greater than the preset multiple of the update distance threshold, the reference detection frame identifier is removed from the final to-be-selected cluster set to obtain a preprocessed cluster set;

[0027] If the straight-line distances between the reference detection frame and each of the distance range detection frames are less than or equal to the preset multiple of the update distance threshold, retain the reference detection frame to obtain a preprocessing cluster set;

[0028] In response to the preprocessing cluster set satisfying a preset preprocessing cluster condition, all target detection box identifiers included in the preprocessing cluster set are determined as preprocessing detection box identifiers, and the preprocessing cluster set is determined as a preprocessing cluster result;

[0029] The method further comprises:

[0030] In response to the preprocessing cluster set not satisfying a preset preprocessing cluster condition, it is determined that the vehicles corresponding to the preprocessing cluster set are not in a congested state.

[0031] In one embodiment, the step of determining the adjacent target detection frame based on the central detection frame includes:

[0032] Based on the central detection frame, determine all adjacent adjacent detection frames to be selected;

[0033] Calculate the straight-line distance between the center point of the central detection frame and the center points of each adjacent detection frame to be selected respectively;

[0034] If the straight-line distance is less than or equal to the initial distance threshold, determining the candidate adjacent detection frame corresponding to the straight-line distance as the target adjacent detection frame;

[0035] The method further comprises:

[0036] If the straight-line distance is greater than the initial distance threshold, it is determined that the to-be-selected adjacent detection frame corresponding to the straight-line distance is not the target adjacent detection frame.

[0037] In one embodiment, the method of determining the secondary clustering result at the corresponding moment of the next frame detection frame image based on the preset position and direction clustering strategy and the preset distance clustering strategy includes:

[0038] When executing the preset position and direction clustering strategy, the tracking trajectory of each vehicle that may be in a congested state is determined based on the next frame detection frame image; the tracking trajectory reflects the position change and direction change of the vehicle that may be in a congested state;

[0039] Calculate the vehicle driving directions corresponding to all the vehicles that may be in a congested state based on the tracking trajectory;

[0040] If any vehicle driving direction includes vehicles that may be in a congested state with a number greater than or equal to a preset number of congested vehicles, the vehicles that may be in a congested state with a number greater than or equal to the preset number of congested vehicles are determined as direction clustering vehicles;

[0041] Determining a direction clustering result corresponding to the vehicle driving direction based on the direction clustering vehicles; the direction clustering result includes a direction clustering vehicle identifier;

[0042] In response to obtaining the directional clustering result, executing the preset distance strategy, and using the preset distance clustering strategy to perform cluster analysis on the target detection frame corresponding to the directional clustering vehicle identification to determine a secondary clustering result at a corresponding moment; the secondary clustering result includes a final detection frame identification;

[0043] The method further comprises:

[0044] If any vehicle driving direction includes vehicles that may be in a congested state and the number of vehicles is less than a preset congestion number, the vehicle driving direction is determined to be a non-congested direction.

[0045] In one embodiment, the method further comprises:

[0046] Get the original training sample images;

[0047] Randomly selecting at least one labeled training detection frame and its corresponding training vehicle from the original training sample image, and generating a new training sample image to be used;

[0048] Adding occlusion marks to the training vehicles in the stand-by training sample pictures according to a preset vehicle occlusion mode set to generate an expanded training sample picture set; the preset vehicle occlusion mode set includes multiple preset occlusion modes; the expanded training sample picture set includes multiple preset occlusion modes corresponding to expanded training sample pictures with the at least one labeled training detection frame, and the number of expanded training sample pictures corresponding to each preset occlusion mode satisfies a preset number ratio.

[0049] In one embodiment, the method further comprises:

[0050] Obtain a training sample set;

[0051] Using the training sample set to train an initial vehicle detection model;

[0052] In response to satisfying a preset training convergence condition, the vehicle detection model satisfying the preset training convergence condition is determined as the vehicle detection model trained to convergence.

[0053] In one embodiment, the determining that vehicles on the trunk road are in a congested state based on the secondary clustering result and a preset duration includes:

[0054] Determine the corresponding final vehicle based on the final detection box identifier in the secondary clustering result;

[0055] Determine at least one secondary clustering result corresponding to all moments included in the preset duration;

[0056] If there is a secondary clustering result at all times that includes a final detection frame identifier corresponding to the final vehicle, it is determined that the final vehicle on the trunk road is in a congested state.

[0057] In a second aspect, the present application provides a vehicle congestion detection device, the device comprising:

[0058] A receiving module, used for receiving a current frame monitoring picture on a trunk road in real time, and obtaining a current frame detection frame picture based on the current frame monitoring picture; the current frame detection frame picture includes a plurality of target detection frames; the target detection frame surrounds the vehicle in the current frame monitoring picture; the detection frame is a rectangular frame;

[0059] A clustering module, configured to perform cluster analysis on the plurality of target detection frames using a preset distance clustering strategy to obtain a preprocessing clustering result; the preprocessing clustering result includes a preprocessing detection frame identifier that satisfies a preset preprocessing clustering condition; the preset distance clustering strategy is a strategy for clustering the distances between adjacent vehicles; the vehicle corresponding to the preprocessing detection frame identifier may be a vehicle in a congested state; the distance between adjacent vehicles is a straight-line distance between the center points of the two target detection frames corresponding to the adjacent vehicles;

[0060] A tracking module is used to obtain the next frame detection frame image corresponding to the vehicle that may be in a congested state. A clustering determination module is used to determine the secondary clustering result of the next frame detection frame image at the corresponding time based on a preset position and direction clustering strategy and a preset distance clustering strategy; the preset position and direction clustering strategy is a strategy for clustering the driving position and direction of each vehicle;

[0061] The congestion determination module is used to determine that the vehicles on the trunk road are in a congested state based on the secondary clustering result and a preset duration.

[0062] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0063] The memory stores computer-executable instructions;

[0064] The processor executes the computer-executable instructions stored in the memory to implement the method as described in the first aspect or any one of the above-mentioned methods.

[0065] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method as described in the first aspect or any one of the methods.

[0066] The present application provides a method, device, equipment and storage medium for detecting vehicle congestion. The detection device of the present application receives a current frame monitoring picture on a trunk road in real time, and then obtains a current frame detection frame picture based on the current frame monitoring picture. The current detection frame picture includes multiple target detection frames, which can surround the vehicle in the current frame monitoring. The target detection frame is a rectangular frame. Then the detection device uses a preset distance clustering strategy to cluster the multiple target detection frames, thereby obtaining a preprocessing clustering result including a preprocessing detection frame identifier that meets the preset preprocessing clustering condition. The preset distance clustering strategy is a strategy for clustering the distances between adjacent vehicles. It can be seen that the detection device takes into account the distances between the vehicles and performs clustering analysis based on the distances between them, so that there is also a distance relationship between the to-be-selected detection frame identifiers in the preprocessing clustering result. Among them, the vehicle corresponding to the preprocessing detection frame identifier may be a vehicle in a congested state, indicating that the preset distance clustering strategy performs clustering analysis based on the distance, implements the clustering operation for the first time, and determines that the vehicle may be in a congested state. For vehicles that may be in a congested state, the detection device then obtains the next frame detection frame image corresponding to the vehicle that may be in a congested state, and determines the secondary clustering result at the corresponding time of the next frame detection frame image based on the preset position and direction clustering strategy and the preset distance clustering strategy. In the present application, after obtaining the preprocessing clustering result corresponding to the current frame monitoring image, that is, after the first clustering operation is realized, two clustering operations are realized based on the preset position and direction clustering strategy and the preset distance strategy, and then the secondary clustering result at the corresponding time of the next frame detection frame image is obtained. It can be seen that multiple clustering is performed in the present application, thereby realizing multiple screening, and finally determining that the vehicle on the trunk road is in a congested state. The distance between adjacent vehicles is taken into consideration in the present application, so it is in line with the actual situation in real life. Multiple clustering is combined in the present application to improve the accuracy of the vehicle being in a congested state. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0068] Figure 1 An application scenario diagram of a vehicle congestion detection method provided in this application;

[0069] Figure 2 A schematic flow chart of a method for detecting vehicle congestion provided in Example 1;

[0070] Figure 3 A flow chart of a method for detecting vehicle congestion provided in Example 3;

[0071] Figure 4 A flow chart of a method for detecting vehicle congestion provided in Example 4;

[0072] Figure 5 A schematic diagram of a current frame detection frame image provided in Embodiment 4;

[0073] Figure 6 A schematic flow chart of a method for detecting vehicle congestion provided in Example 5;

[0074] Figure 7 A flow chart of a method for detecting vehicle congestion provided in Example 6;

[0075] Figure 8 A flow chart of a method for detecting vehicle congestion provided in Example 7;

[0076] Fig. 9 A schematic diagram of a next frame detection frame image provided in Example 7;

[0077] Fig.10 A flow chart of a method for detecting vehicle congestion provided in Embodiment 8;

[0078] Fig.11 A schematic diagram of a method for dividing training vehicles provided in Example 8;

[0079] Fig.12 A schematic flow chart of a method for detecting vehicle congestion provided in Embodiment 9;

[0080] Fig.13 A flow chart of a method for detecting vehicle congestion provided in Example 10;

[0081] Fig.14 A schematic diagram of the structure of a vehicle congestion detection device provided in Example 11;

[0082] Fig.15 A schematic diagram of the structure of an electronic device provided in Example 12.

[0083] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0084] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0085] In the prior art, after the monitoring system collects the monitoring video, the monitoring system determines the target vehicle based on the monitoring video, as well as the front and rear monitoring images of the target vehicle, thereby calculating the movement value to determine that the target vehicle is a slow-moving vehicle, and further determines that the target vehicle is in a congested state.

[0086] However, the monitoring system determines the congestion state by using the movement value of the target vehicle, which does not conform to the actual congestion situation. At the same time, the target vehicle is determined to be in a congestion state if it is determined to be a slow-moving vehicle, but there is also the possibility that the target vehicle itself is moving slowly on the trunk road, so the accuracy is low.

[0087] In order to solve the defects of the prior art, the inventor of this solution has designed a new solution after creative research. The present invention provides a method for detecting vehicle congestion. In order to solve the problem of not conforming to the actual congestion situation and having low accuracy, after receiving the current frame monitoring picture on the trunk road, the detection device of the present invention obtains the current frame detection frame picture based on the current frame monitoring picture. The current frame detection frame picture includes multiple target detection frames. Then the detection device uses a preset distance clustering strategy to cluster the multiple target detection frames to obtain a preprocessing clustering result. The preset distance clustering strategy in the present invention is a strategy for clustering the distances between adjacent vehicles. Therefore, the first screening can screen out the preprocessing detection frame identifiers of the same cluster with a short distance, and the vehicles corresponding to the preprocessing detection frame identifiers are vehicles that may be in a congested state, so the vehicles that may be in a congested state are retained for continued detection. In real life, when vehicles are driving on the trunk road, congestion is caused by the distances between the vehicles being too close. Therefore, in the present invention, the distances between adjacent vehicles are first considered through the preset distance clustering strategy, so it can conform to the actual situation. Then, the next frame detection is determined based on the preset position and direction clustering strategy and the preset distance strategy. The secondary clustering result of the corresponding moment of the frame picture is obtained, so the position change of the vehicle during the movement is further considered, and the distance clustering is performed based on the distance between the vehicles at the corresponding moment of the next frame detection frame picture, so it is more in line with the actual situation. In the prior art, the monitoring system determines the intersection-and-combination ratio and the movement value of the target vehicle in the front and rear pictures to determine that the target vehicle is in a congested state. In fact, the distance between the target vehicles is not considered, and only the movement of the target vehicle itself is considered. If the target vehicle is driving slowly on the trunk road in the real scene, and there are no other vehicles around, it is impossible to judge that the target vehicle is in a congested state at this time, so the prior art does not conform to the actual situation. This scheme considers the distance between the vehicles through the current frame detection frame picture, and then first determines the vehicles that may be in a congested state, and then considers the position of the vehicle after moving and the distance between the vehicles at the corresponding moment of the next frame detection frame picture based on the next frame detection frame picture, and then determines whether the vehicle on the trunk road is in a congested state, so this scheme conforms to the actual situation, and through multiple clustering, the accuracy can be improved.

[0088] The following is an introduction to the application scenarios of a vehicle congestion detection method, device, equipment and storage medium provided by the present application.

[0089] Figure 1 This is an application scenario diagram of a vehicle congestion detection method provided in this application. Figure 1 As shown, the application scenario diagram includes an electronic device 101.

[0090] The electronic device 101 can process the surveillance video to detect vehicle congestion on the trunk road.

[0091] The electronic device 101 may be a computer or a server, etc., which is not limited here.

[0092] The electronic device 101 includes a vehicle congestion detection device.

[0093] It should be noted that deploying monitoring equipment (which may be cameras, etc.) on trunk roads can monitor vehicles and generate monitoring videos.

[0094] There may be at least one vehicle 1 on the trunk road, and the types of vehicles may be various, such as a van, a bus, a truck or a family car, etc., which are not limited here. It should be noted that the trunk road may also include small objects such as people.

[0095] In this scenario, there are N vehicles traveling on the trunk road, and the electronic device 101 monitors the trunk road and uses a camera to perform monitoring and shooting, thereby obtaining a monitoring video.

[0096] Furthermore, the electronic device 101 receives the monitoring video sent by the monitoring device, and parses the monitoring video to obtain a current frame monitoring picture.

[0097] Further, the electronic device 101 obtains the current frame detection frame image based on the current frame monitoring image. The electronic device uses a preset distance clustering strategy to perform cluster analysis on multiple target detection frames to obtain a preprocessing clustering result. The preprocessing clustering result includes a preprocessing detection frame identifier that meets the preset preprocessing clustering condition, and the vehicle corresponding to the preprocessing detection frame identifier may be a congested vehicle.

[0098] Furthermore, the electronic device 101 obtains the next frame detection frame image corresponding to the vehicle in the possible congested state, and determines the secondary clustering result of the next frame detection frame image at the corresponding moment based on the preset position and direction clustering strategy and the preset distance clustering strategy.

[0099] Furthermore, the electronic device 101 determines that the vehicles on the trunk road are in a congested state based on the secondary clustering result and the preset duration.

[0100] The present application provides a method for detecting vehicle congestion, aiming to solve the above technical problems in the prior art.

[0101] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0102] Embodiment 1

[0103] The executing entity of this application is a vehicle congestion detection device, which is located in an electronic device.

[0104] Figure 2 A flow chart of a method for detecting vehicle congestion provided in Example 1. Figure 2 As shown, the specific steps include the following steps.

[0105] S201, receiving a current frame monitoring picture on a trunk road in real time, and obtaining a current frame detection frame picture based on the current frame monitoring picture; the current frame detection frame picture includes multiple target detection frames; the target detection frame surrounds the vehicle in the current frame monitoring picture; the detection frame is a rectangular frame.

[0106] The trunk road refers to an area of ​​interest that can be captured by the monitoring device. It should be noted that the monitoring device monitors the trunk road to obtain monitoring videos, and sends the monitoring videos to the electronic device to prompt the electronic device to detect vehicle congestion.

[0107] Among them, the current frame monitoring picture is a picture obtained by analyzing the monitoring video.

[0108] Among them, the current frame detection frame picture includes multiple target detection frames, which are rectangular frames. The target detection frames include vehicles in the current monitoring picture, so the target detection frames surround the vehicles in the current frame monitoring picture.

[0109] It should be noted that the size of the target detection box varies depending on the size of the vehicle.

[0110] It should be noted that if the detection device can identify more and more accurately the vehicles included in the current frame monitoring image, then the solution of the present application will be more accurate.

[0111] S202, cluster analysis is performed on multiple target detection frames using a preset distance clustering strategy to obtain a preprocessing clustering result; the preprocessing clustering result includes a preprocessing detection frame identifier that meets the preset preprocessing clustering condition; the preset distance clustering strategy is a strategy for clustering the distances between adjacent vehicles; the vehicle corresponding to the preprocessing detection frame identifier may be a vehicle in a congested state; the distance between adjacent vehicles is the straight-line distance between the center points of the two target detection frames corresponding to the adjacent vehicles.

[0112] Among them, the preset distance clustering strategy is a strategy for clustering the distances between adjacent vehicles, wherein the distances between adjacent vehicles are clustered. For example, if the current frame monitoring picture includes vehicle 1, vehicle 2, vehicle 3, and vehicle 4. Among them, vehicle 1 is adjacent to vehicle 2, vehicle 2 is adjacent to vehicle 3, and vehicle 3 is adjacent to vehicle 4. Although vehicle 1 is not directly adjacent to vehicle 3 and vehicle 4, the preset distance clustering strategy is adopted to consider the distances between adjacent vehicles, so the detection device will consider the distances between vehicle 2 and vehicle 3 and vehicle 3 and vehicle 4 for clustering.

[0113] It should be noted that the preset distance strategy can cluster vehicles that are close together. According to the above example, vehicle 1 is adjacent to vehicle 2, vehicle 2 is adjacent to vehicle 3, and vehicle 3 is adjacent to vehicle 4. The distance between each vehicle satisfies the distance condition, so that vehicle 1, vehicle 2, vehicle 3 and vehicle 4 are finally clustered in the same cluster. If vehicle 3 and vehicle 4 are adjacent, they may not meet the distance condition, so at this time, vehicle 1, vehicle 2, and vehicle 3 are finally clustered together. Among them, satisfying the distance condition means that the distance between adjacent vehicles is less than or equal to the initial distance threshold or satisfies the preset multiple of the updated distance threshold, wherein the description of the updated distance threshold request embodiment five is not repeated here.

[0114] The distance between adjacent vehicles is the straight-line distance between the center points of two target detection frames corresponding to the adjacent vehicles. That is, the center points of the two target detection frames are determined, and then the straight-line distance between the two center points is determined as the distance between adjacent vehicles.

[0115] The preprocessing clustering result includes a preprocessing detection frame identifier that meets a preset preprocessing clustering condition, and the preprocessing detection frame identifier is screened out from the target detection frame identifier.

[0116] Among them, the preprocessing detection box identifiers in the preprocessing clustering result are of the same cluster. Since the preprocessing clustering result meets the preset preprocessing conditions, the vehicle corresponding to the preprocessing detection box identifier may be a vehicle in a congested state.

[0117] It should be noted that the first screening (or the preprocessing process) is completed after the preprocessing clustering result is obtained. In the preprocessing process, vehicles that are not in a congested state can be excluded.

[0118] S203, tracking vehicles that may be in a congested state to obtain the next frame detection frame image corresponding to the vehicles that may be in a congested state, and determining the secondary clustering result of the next frame detection frame image at the corresponding moment based on a preset position and direction clustering strategy and a preset distance clustering strategy; the preset position and direction clustering strategy is a strategy for clustering the driving position and direction of each vehicle.

[0119] In one method, the detection device still receives the monitoring picture in real time. After the detection device determines that the vehicle may be in a congested state, it tracks the vehicle that may be in a congested state, so as to obtain the next frame detection frame picture corresponding to the vehicle that may be in a congested state. Specifically, when tracking the vehicle that may be in a congested state, the next frame monitoring picture corresponding to the vehicle that may be in a congested state can be tracked, and then the next frame detection frame picture corresponding to the vehicle that may be in a congested state is obtained based on the next frame monitoring picture corresponding to the vehicle that may be in a congested state.

[0120] It should be noted that the detection device can receive the next frame of monitoring pictures containing all vehicles, track the vehicles that may be in a congested state in the next frame of monitoring pictures, retain the vehicles that may be in a congested state, and generate the next frame of monitoring pictures corresponding to the vehicles that may be in a congested state (in the next frame of monitoring pictures, there may only be vehicles that may be in a congested state corresponding to a certain preprocessing clustering result), and then obtain the next frame of detection frame pictures based on the next frame of monitoring pictures.

[0121] Among them, the preset position and direction clustering strategy is a strategy for clustering the driving position and direction of each vehicle. Since each vehicle that may be in a congested state may move from the corresponding moment of the current frame to the corresponding moment of the next frame, considering the change in position, the position and direction clustering can be performed first at the corresponding moment of the next frame. It should be noted that the change in position of the vehicle may actually be accompanied by a change in direction, that is, the vehicle driving direction of the vehicle can be determined according to the change in position, and the vehicles that may be in a congested state in the same vehicle driving direction can be clustered in the same cluster. For some vehicles that may be in a congested state, they may drive to an idle driving direction, and these vehicles that may be in a congested state in the idle driving direction are not in a congested state. Among them, the idle driving direction is also a vehicle driving direction, but it means that there are fewer vehicles included in the vehicle driving direction, so the vehicle driving direction is relatively idle. For example, it is assumed that there are vehicles 1, 2, 3 and 4, among which, the driving directions of vehicles 1, 2 and 3 from the corresponding moment of the current frame to the corresponding moment of the next frame are all in the same vehicle driving direction, and the driving direction of vehicle 4 is an idle driving direction. Therefore, this application adopts a preset position and direction clustering strategy to perform position and direction clustering.

[0122] Furthermore, the detection device also determines a secondary clustering result of the next frame detection frame image at a corresponding moment based on a preset distance clustering strategy.

[0123] The secondary clustering result refers to performing two clustering operations, that is, performing two clustering operations based on a preset position and direction clustering strategy and a preset distance clustering strategy.

[0124] It should be noted that, based on the preset position and direction clustering strategy, it can only be determined that multiple vehicles are included in the same vehicle driving direction at this time. According to the above exemplary embodiment, it is assumed that vehicle 1, vehicle 2 and vehicle 3 are in the same vehicle driving direction, but due to moving from the current frame to the next frame, although vehicle 1, vehicle 2 and vehicle 3 are in the same vehicle driving direction, some vehicles in the same vehicle driving direction may still be far away, so the preset distance clustering strategy is adopted in this application to cluster the vehicles that meet the distance conditions together. Exemplarily, if vehicle 1 is adjacent to vehicle 2 and vehicle 3 respectively, and the distance conditions are met between the vehicles, it is determined that vehicle 1, vehicle 2 and vehicle 3 are clustered in the same cluster.

[0125] The vehicle driving direction refers to the vehicle driving direction from the current frame to the next corresponding frame.

[0126] S204: Determine that the vehicles on the trunk road are in a congested state based on the secondary clustering result and the preset duration.

[0127] The preset duration refers to a preset duration, which includes multiple future moments, each of which corresponds to a detection frame image at that moment.

[0128] In one embodiment, the detection device obtains the detection frame image corresponding to the last moment based on a preset duration, and determines at least one secondary clustering result corresponding to the detection frame image based on the above method, and determines the at least one secondary clustering result as the secondary clustering result corresponding to the last moment.

[0129] Furthermore, the detection device determines the secondary clustering result at the corresponding moment of the next frame detection frame image and the secondary clustering result at the last moment to determine that the vehicles on the trunk road are in a congested state.

[0130] The present embodiment provides a method for detecting vehicle congestion. The detection device of the present application receives a current frame monitoring picture on a trunk road in real time, and then obtains a current frame detection frame picture based on the current frame monitoring picture. The current detection frame picture includes multiple target detection frames, which can surround the vehicle in the current frame monitoring. The target detection frame is a rectangular frame. Then the detection device uses a preset distance clustering strategy to perform cluster analysis on the multiple target detection frames, thereby obtaining a preprocessing clustering result including a preprocessing detection frame identifier that meets a preset preprocessing clustering condition. The preset distance clustering strategy is a strategy for clustering the distances between adjacent vehicles. It can be seen that the detection device takes into account the distances between the vehicles and performs cluster analysis based on the distances between the vehicles, so that there is also a distance relationship between the to-be-selected detection frame identifiers in the preprocessing clustering result. Among them, the vehicle corresponding to the preprocessing detection frame identifier may be a vehicle in a congested state, indicating that the preset distance clustering strategy performs cluster analysis based on the distance, implements the clustering operation for the first time, and determines that the vehicle may be in a congested state. For vehicles that may be in a congested state, the detection device then obtains the next frame detection frame image corresponding to the vehicle that may be in a congested state, and determines the secondary clustering result at the corresponding time of the next frame detection frame image based on the preset position and direction clustering strategy and the preset distance clustering strategy. In the present application, after obtaining the preprocessing clustering result corresponding to the current frame monitoring image, that is, after the first clustering operation is realized, two clustering operations are realized based on the preset position and direction clustering strategy and the preset distance strategy, and then the secondary clustering result at the corresponding time of the next frame detection frame image is obtained. It can be seen that multiple clustering is performed in the present application, thereby realizing multiple screening, and finally determining that the vehicle on the trunk road is in a congested state. The distance between adjacent vehicles is taken into consideration in the present application, so it is in line with the actual situation in real life. Multiple clustering is combined in the present application to improve the accuracy of the vehicle being in a congested state.

[0131] Embodiment 2

[0132] This embodiment is a further refinement of the above-mentioned embodiment 1. This embodiment is an optional method for obtaining a current frame detection frame image based on a current frame monitoring image.

[0133] The current frame monitoring picture is input into the vehicle detection model trained to convergence to obtain the current frame detection frame picture; the vehicle detection model trained to convergence is obtained based on the training sample set; the training sample set includes the original training sample picture set and the expanded training sample picture set; the original training sample picture set includes at least one training sample picture; the expanded training sample picture set includes at least one expanded training sample picture; the training sample picture includes a labeled training detection frame; the labeled training detection frame surrounds the training vehicle in the training sample picture; the labeled training detection frame is a rectangular frame; the expanded training sample picture is a sample picture generated based on the original training sample picture and taking into account multiple preset vehicle occlusion methods.

[0134] The training sample images include original training sample images and expanded training sample images.

[0135] The training sample images include labeled training detection frames, which are pre-labeled manually or automatically.

[0136] It should be noted that the vehicle detection model trained to convergence in the present application can detect occluded vehicles and small targets in the input image, so that the detection frame in any detection frame image obtained in the present application is relatively complete and the possibility of missed detection is reduced.

[0137] This embodiment provides a method for detecting vehicle congestion. In this application, multiple preset vehicle occlusion modes are considered in the expanded training sample images, so that the initial vehicle detection model can learn multiple preset vehicle occlusion modes in the expanded training sample images, so the obtained vehicle detection model trained to convergence can identify the occluded vehicles in the input image. Even if it is an occluded vehicle, the vehicle detection model trained to convergence in this application can also identify it and mark the detection frame. Therefore, the possibility of missed detection in this embodiment is small, and the accuracy rate can be improved.

[0138] Embodiment 3

[0139] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method of using a preset distance clustering strategy to perform clustering analysis on multiple target detection frames to obtain pre-processed clustering results.

[0140] Figure 3 A flow chart of a method for detecting vehicle congestion provided in Example 3. As shown in 3, the specific steps are as follows.

[0141] S301, when executing a preset distance clustering strategy, determining a target detection frame with the largest area from multiple target detection frames, and obtaining a target detection frame width and a target detection frame height corresponding to the target detection frame with the largest area.

[0142] For example, assume that the current frame detection frame picture includes 5 target detection frames, namely target detection frame 1, target detection frame 2, target detection frame 3, target detection frame 4 and target detection frame 5. Assume that target detection frame 1 has the largest area, and its corresponding target detection frame height is 5 cm and target detection frame width is 6 cm. The unit corresponding to the width or height of the detection frame in this application can be cm (centimeter) or other units of representation methods, which will not be processed here.

[0143] S302: Determine the maximum value of the values ​​corresponding to the target detection frame width and the target detection frame height as the initial distance threshold.

[0144] Furthermore, the detection device determines the maximum value of the corresponding values ​​of the target detection frame width and the target detection frame height, that is, the target detection frame width (6 cm), and uses the target detection frame width as the initial distance threshold.

[0145] S303: Obtain a preprocessing clustering result based on an initial distance threshold.

[0146] This embodiment provides a method for detecting vehicle congestion. In this application, the target detection frame width or target detection frame height corresponding to the target detection frame with the largest area is selected, so that the initial distance threshold is also relatively large, so that the corresponding preprocessing detection frame in the preprocessing clustering result can have a certain inclusiveness.

[0147] Embodiment 4

[0148] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to obtain the pre-processing clustering result based on the initial distance threshold.

[0149] Figure 4 A flow chart of a method for detecting vehicle congestion provided in Example 4. Figure 4 As shown, the specific steps are as follows.

[0150] S401, each target detection frame is determined as a central detection frame, and its adjacent target adjacent detection frame is determined based on the central detection frame; the straight-line distance between the center point of the target adjacent detection frame and the center point of the central detection frame is less than or equal to an initial distance threshold.

[0151] The center detection frame refers to a detection frame that takes a certain target detection frame as the center, which can be used to determine its adjacent detection frames.

[0152] The target adjacent detection frame refers to a frame that is adjacent to the center detection frame and has a straight-line distance between the center point of the target adjacent detection frame and the center point of the center detection frame that is less than or equal to the initial distance threshold.

[0153] Figure 5 A schematic diagram of a current frame detection frame picture provided in Example 4. Figure 5 As shown, it includes target detection frames of different sizes, namely target detection frame 1 to target detection frame 12.

[0154] Specifically, target detection frame 1 is determined as the central detection frame, and its adjacent target adjacent detection frames are determined as target detection frame 2, target detection frame 3, target detection frame 4, target detection frame 5 and target detection frame 11.

[0155] It is understandable that the straight-line distance between target detection frame 1 and target detection frame 2 and between target detection frame 1 and target detection frame 3 is less than or equal to the initial distance threshold. The same is true for target detection frame 1, target detection frame 5 and target detection frame 11, which will not be repeated here.

[0156] like Figure 5 The black circle in the figure represents a vehicle, the number inside the black circle represents the vehicle identification, and the rectangular box outside the black circle is the target detection box corresponding to the vehicle. The numbers around the target detection box represent the identification of the target detection box.

[0157] S402, determining a set of clusters to be selected; the set of clusters to be selected includes a center detection box identifier and its adjacent target adjacent detection box identifiers.

[0158] According to the above exemplary embodiment, the detection device determines that the cluster set to be selected is {1, 2, 3, 4, 5, 11}, which represents target detection box 1, target detection box 2, target detection box 3, target detection box 4, target detection box 5 and target detection box 11.

[0159] Among them, target detection frame 1 is the center detection frame identifier, and the others are target adjacent detection frame identifiers.

[0160] S403: Continue to determine each target adjacent detection frame as a central detection frame, and determine target adjacent detection frames adjacent to the central detection frame based on the central detection frame.

[0161] Further, for the cluster detection task with target detection frame 1 as the center detection frame, the detection device continues to determine each target adjacent detection frame as the center detection frame. It should be noted that if "determining each target detection frame as the center detection frame" in S401 is set as a cluster detection task, then when each target detection frame is used as the center detection frame, it is a cluster detection task. According to the example in this embodiment, there are 12 target detection frames in total, and there are 12 cluster detection tasks.

[0162] Exemplarily, taking the target detection frame 2 as the central detection frame as an example, the target adjacent detection frame adjacent to the target detection frame 2 is then determined based on the target detection frame 2. The determination method is consistent with step S401 and will not be described in detail here.

[0163] like Figure 5 As shown, the target adjacent detection frames adjacent to the target detection frame 2 are target detection frame 1, target detection frame 11 and target detection frame 12.

[0164] It should be noted that target detection frame 3, target detection frame 4, target detection frame 5 and target detection frame 11 are all used as central detection frames respectively, and the method consistent with S401 is executed to determine the target adjacent detection frames adjacent to themselves.

[0165] S404, adding the newly determined target adjacent detection box identifier to the set of clusters to be selected, and continuing to execute the steps of determining each target adjacent detection box as a central detection box to adding the newly determined target adjacent detection box identifier to the set of clusters to be selected, until there are no newly determined target adjacent detection boxes, so as to obtain the final set of clusters to be selected corresponding to each target detection box as the central detection box.

[0166] The newly determined target adjacent detection frame refers to a target detection frame that has not been determined before and meets the distance condition.

[0167] Exemplarily, the adjacent target detection frames of the target detection frame 2 are target detection frame 1, target detection frame 11, and target detection frame 12. Since the corresponding identifiers of target detection frame 1 and target detection frame 11 are in the to-be-selected cluster set, indicating that they are previously determined target adjacent detection frames, the newly determined target adjacent detection frame is target detection frame 11.

[0168] Furthermore, the detection device adds the corresponding identifier of the target detection frame 12 to the cluster set to be selected, and the detection device continues to execute the steps of determining each target adjacent detection frame as the central detection frame to adding the newly determined target adjacent detection frame identifier to the cluster set to be selected until there is no newly determined target adjacent detection frame.

[0169] It should be noted that, in the present application, the newly determined target adjacent detection frame shall continue to be used as the central detection frame, and the method step of S401 shall be executed, and this cycle shall be repeated until there is no newly determined target adjacent detection frame, and the final set of candidate clusters corresponding to the cluster detection task (that is, the cluster detection task triggered by the target detection frame 1 as the central detection frame) is obtained, that is, {1,2,3,4,5,11,12}.

[0170] It can be understood that, using the same method, the cluster detection task triggered by the target detection box 2 as the center detection box also has a corresponding final set of clusters to be selected.

[0171] It should be noted that if the number of target detection frame identifiers included in the corresponding final clustering set with any target detection frame as the central detection frame is less than the preset first candidate number, the final clustering set to be selected is determined to be an invalid set, which indicates that the corresponding vehicles in the final clustering set to be selected are not in a congested state.

[0172] In this embodiment, there are a total of 12 target detection frames, and thus 12 cluster detection tasks, and finally 12 final cluster sets to be selected are obtained.

[0173] As shown in the figure, with target detection frame 6 as the central detection frame, it corresponds to a cluster detection task, and the final cluster set to be selected is {6,7,8,9,10}. If the number of target detection frame identifiers included in the final cluster set to be selected is greater than or equal to the preset first candidate number, it is determined to be a valid set, and S405 is continued to be executed based on the final cluster set to be selected.

[0174] It is understandable that the final to-be-selected cluster sets corresponding to some cluster detection tasks are invalid sets, and the final to-be-selected cluster sets corresponding to some cluster detection tasks are valid sets, ie, they can continue to be used to execute S405.

[0175] S405: Determine a preprocessing clustering result based on the final set of clusters to be selected.

[0176] The present embodiment provides a method for detecting vehicle congestion. In the present embodiment, the detection device determines each target detection frame as a central detection frame, and determines adjacent target adjacent detection frames based on the central detection frame. In the present embodiment, the straight-line distance between the center point of the target adjacent detection frame and the center point of the central detection frame is less than or equal to the initial distance threshold, so that in the present embodiment, the remaining detection frames centered on the target detection frame within the initial distance threshold range are clustered together. Then, in order to find a target detection frame adjacent to the target adjacent detection frame, the present embodiment uses the target adjacent detection frame as the central detection frame, determines the target adjacent detection frames adjacent to it, and adds them to the to-be-selected clustering set, so that the number of detection frame identifiers included in the to-be-selected clustering set will increase. At the same time, the present embodiment can continuously spread outward to find more target adjacent detection frames, so each target detection frame is clustered.

[0177] Embodiment 5

[0178] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to determine the preprocessing clustering result based on the final set of clusters to be selected.

[0179] Figure 6 A flow chart of a method for detecting vehicle congestion provided in Example 5. Figure 6 As shown, the specific steps are as follows.

[0180] S601, determining the target detection box corresponding to each target detection box identifier from the final set of clusters to be selected as a reference detection box.

[0181] The reference detection frame refers to a detection frame used as a reference.

[0182] In one embodiment, a method in which a detection device determines a target detection frame corresponding to each target detection frame identifier from a final set of candidate clusters as a reference detection frame comprises the following steps:

[0183] Specifically, determine the number of target detection box identifiers included in each final cluster set to be selected; if the number of target detection box identifiers is greater than the preset second candidate data, determine that the final cluster set to be selected is a valid set; determine each target detection box identifier from the final cluster set to be selected, and use each target detection box as a benchmark detection box.

[0184] Exemplarily, assuming that the final candidate cluster set is {1, 2, 3, 4, 5, 11, 12}, all target detection boxes included therein are used as reference detection boxes to perform the following steps.

[0185] S602, determine the distance range detection frame within the preset distance range of the reference detection frame, and determine the near detection frame width or near detection frame height of the nearest distance range detection frame as the updated distance threshold; the distance range detection frame is at least one updated distance threshold that is the maximum value of the corresponding values ​​of the near detection frame width and the near detection frame height.

[0186] The preset distance range is pre-set, and within the preset distance range there are one or two target detection frames that are closer to the reference detection frame. The preset distance range may be variable, for example, the preset distance range refers to the range of the target detection frame closest to the reference detection frame, that is, at this time, the distance range detection frame is the target detection frame closest to the reference detection frame. The preset distance range may also refer to the range of the target detection frame closest to the reference detection frame and the second closest target detection frame, that is, at this time, the distance range detection frame is the target detection frame closest to the reference detection frame and the second closest target detection frame.

[0187] The distance range detection frame refers to a target detection frame within a preset distance range.

[0188] For example, Figure 5 As shown, assuming that target detection frame 4 is taken as an example, target detection frame 4 is used as a reference detection frame, and the distance range detection frames within a preset distance range from target detection frame 4 are determined to be target detection frame 1 and target detection frame 3.

[0189] Furthermore, the detection device determines that the target detection frame 3 is closest to the target detection frame 4, and determines the detection frame width corresponding to the target detection 3 as the near-position detection frame width, and determines the detection frame height corresponding to the target detection 3 as the near-position detection frame height. If the value of the near-position detection frame width is greater than or equal to the near-position detection frame height, the near-position detection frame width is determined as the update distance threshold; if the value of the near-position detection frame width is less than the near-position detection frame height, the near-position detection frame height is determined as the update distance threshold.

[0190] The near-position detection frame width refers to the detection frame width corresponding to the distance range detection frame closest to the reference detection frame.

[0191] The near-position detection frame height refers to the detection frame height corresponding to the detection frame in the distance range closest to the reference detection frame.

[0192] S603: If the straight-line distances between the reference detection frame and each distance range detection frame are greater than a preset multiple of the update distance threshold, the reference detection frame identifier is removed from the final candidate cluster set to obtain a preprocessed cluster set.

[0193] The preset multiple may be 1.5 times or 1.3 times, etc., and is not limited here.

[0194] Exemplarily, assuming that the update distance threshold is 5 cm and the preset multiple is 1.5 times, the update distance threshold of the preset multiple may be 1.5*5=7.5 cm.

[0195] Exemplarily, the detection device calculates the straight-line distances between target detection frame 4 and target detection frame 3 and target detection frame 1 respectively. Assuming that the straight-line distance to target detection frame 3 is 8 cm and the straight-line distance to target detection frame 1 is 8.5 cm, it is determined that the straight-line distances are both greater than a preset multiple of the update distance threshold, and the target detection frame 4 identifier (i.e., the benchmark detection frame identifier) ​​is removed from the final candidate clustering set. If the remaining target detection frames, when used as benchmark detection frames, meet step S604 and are retained, i.e., only the target detection frame 4 identifier is removed, thereby obtaining a preprocessed clustering set of {1, 2, 3, 5, 11, 12}.

[0196] It should be noted that the above steps S601 to S603 are performed for each target detection frame in the final candidate cluster set as a reference detection frame, so as to obtain a preprocessed cluster set.

[0197] For example, assuming Figure 5 In the figure, the pre-processed clustering sets obtained after being processed by the above-mentioned embodiment method are {1, 2, 3, 5, 11, 12} and {6, 7, 8, 9, 10} respectively.

[0198] It should be noted that some target detection frames in the final candidate cluster set can be eliminated when used as reference detection frames, and some can be retained, as shown in S604 for details.

[0199] The preprocessed cluster set is a cluster set obtained after removing or retaining each benchmark detection frame.

[0200] S604: If the straight-line distances between the reference detection frame and the detection frames in each distance range are all less than or equal to a preset multiple of the update distance threshold, the reference detection frame is retained to obtain a preprocessed cluster set.

[0201] Exemplarily, assuming that the detection device calculates the straight-line distances between target detection frame 4 and target detection frame 3 and target detection frame 1 respectively, and assuming that the straight-line distance to target detection frame 3 is 5.5 cm, and the straight-line distance to target detection frame 1 is 6 cm, it is determined that the straight-line distances are both less than a preset multiple of the update distance threshold, and the target detection frame 4 identifier (i.e., the benchmark detection frame identifier) ​​is retained. If the remaining target detection frames meet this step when used as benchmark detection frames, they are retained, so the preprocessed clustering set is {1,2,3,5,11,12}.

[0202] S605 , in response to the preprocessing cluster set satisfying a preset preprocessing cluster condition, determining all target detection box identifiers included in the preprocessing cluster set as preprocessing detection box identifiers, and determining the preprocessing cluster set as a preprocessing cluster result.

[0203] Wherein, the preset preprocessing clustering condition can be that if the obtained preprocessing clustering set is one, then the preprocessing clustering set must include preprocessing detection frame identifiers greater than or equal to the preset second candidate number, which means, for example, greater than or equal to 10 vehicles, and the preprocessing clustering set is determined as the preprocessing clustering result; if there are two preprocessing clustering sets, then each preprocessing clustering set must include preprocessing detection frame identifiers greater than or equal to the preset first candidate number, and the distance to the nearest preprocessing detection frame (that is, the nearest adjacent vehicle) must be less than the initial distance threshold of the preset distance multiple. Wherein, the description of the initial distance threshold is shown in Example 3, which is not repeated here. Wherein, the preset second candidate number is greater than the preset first candidate number, wherein the preset first candidate number and the preset second candidate number can be preset according to actual conditions, and for example, the preset second candidate number can be 10, and the preset first candidate number can be 4. Wherein, the preset distance multiple can also be 1.5 times, which is not limited here.

[0204] Exemplarily, assume that the preprocessing clustering result is {1, 2, 3, 5, 11, 12}, and the target detection box identifier included therein is the preprocessing detection box identifier.

[0205] The method of this embodiment also includes:

[0206] In response to the preprocessing cluster set or the retained final to-be-selected cluster set not satisfying the preset preprocessing cluster condition, it is determined that the vehicle corresponding to the preprocessing cluster set is not in a congested state.

[0207] Embodiment 6

[0208] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method for determining adjacent target adjacent detection frames based on the center detection frame.

[0209] Figure 7A flow chart of a method for detecting vehicle congestion provided in Example 6. Figure 6 As shown, the specific steps are as follows.

[0210] S701, based on the central detection frame, determine all adjacent adjacent detection frames to be selected.

[0211] like Figure 5 As shown, assuming that target detection frame 1 is the central detection frame, its adjacent adjacent detection frames to be selected are determined to be target detection frame 2, target detection frame 3, target detection frame 4, target detection frame 5 and target detection frame 11.

[0212] S702, respectively calculating the straight-line distances between the center point of the central detection frame and the center points of each adjacent detection frame to be selected.

[0213] Furthermore, the detection device calculates the straight-line distances between the center point of target detection frame 1 and the center points of target detection frame 2, target detection frame 3, target detection frame 4, and target detection frame 5, respectively, assuming that they are straight-line distance 1-2, straight-line distance 1-3, straight-line distance 1-4, straight-line distance 1-5, and straight-line distance 1-11.

[0214] S703: If the straight-line distance is less than or equal to the initial distance threshold, determine the candidate adjacent detection frame corresponding to the straight-line distance as the target adjacent detection frame.

[0215] Exemplarily, assuming that the straight-line distance 1-2 is less than or equal to the initial distance threshold, the candidate adjacent detection frame corresponding to the straight-line distance 1-2 (ie, the target detection frame 2) is determined as the target adjacent detection frame.

[0216] This embodiment also includes:

[0217] If the straight-line distance is greater than the initial distance threshold, it is determined that the candidate adjacent detection box corresponding to the straight-line distance is not the target adjacent detection box.

[0218] Exemplarily, assuming that the straight-line distance 1-2 is greater than the initial distance threshold, it is determined that the candidate adjacent detection frame corresponding to the straight-line distance 1-2 (ie, the target detection frame 2) is not the target adjacent detection frame.

[0219] The present embodiment provides a method for detecting vehicle congestion. In the present embodiment, the detection device first determines all adjacent detection frames to be selected around the central detection frame, and respectively calculates the straight-line distances between the center point of the central detection frame and the center points of each adjacent detection frame to be selected. The detection device determines that if the straight-line distance is less than or equal to the initial distance threshold, the adjacent detection frame to be selected corresponding to the straight-line distance is determined to be the target adjacent detection frame. It can be seen that the target adjacent detection frame is close to the central detection frame, which can improve the accuracy.

[0220] Embodiment 7

[0221] The present embodiment is a further refinement of any of the above embodiments. The present embodiment is an optional method for determining the secondary clustering result at the corresponding moment of the next frame detection frame image based on a preset position and direction clustering strategy and a preset distance clustering strategy.

[0222] Figure 8 A flow chart of a method for detecting vehicle congestion provided in Example 7. Figure 8 As shown, the specific steps are as follows.

[0223] S801, when executing the preset position and direction clustering strategy, the tracking trajectory of each vehicle that may be in a congested state is determined based on the next frame detection frame image; the tracking trajectory reflects the position change and direction change of the vehicle that may be in a congested state.

[0224] It should be noted that, in this embodiment, the detection device tracks the vehicle that may be in a congested state, so as to obtain the next frame detection frame corresponding to the vehicle that may be in a congested state.

[0225] Furthermore, the detection device determines the tracking trajectory of each possible vehicle in a congested state based on the positions of the center points of the current frame target detection frame and the next frame target detection frame corresponding to each possible vehicle in a congested state.

[0226] The tracking trajectory refers to the trajectory corresponding to the position change of any vehicle that may be in a congested state when it moves from the corresponding time of the current frame to the corresponding time of the next frame. It reflects the position change and direction change of the vehicle that may be in a congested state.

[0227] It can be understood that the driving direction of a vehicle that may be in a congested state can be calculated based on the tracking trajectory.

[0228] It should be noted that each vehicle that may be in a congested state has a corresponding tracking trajectory of its own.

[0229] S802: Calculate the vehicle driving directions corresponding to all vehicles that may be in a congested state based on the tracking trajectory. Fig. 9 A schematic diagram of a vehicle driving direction provided in Example 7. Fig. 9 As shown, for example, taking the preprocessing clustering result {1,2,3,4,5,11,12} as an example, the next frame detection frame picture includes the vehicle corresponding to the preprocessing clustering result and the target detection frame corresponding to each vehicle. Among them, the vehicle corresponding to the target detection frame included in the preprocessing clustering result may be a vehicle in a congested state, that is, vehicle 1 corresponding to target detection frame 1 may be a vehicle in a congested state.

[0230] Furthermore, the 0-360° direction is evenly divided into 8 vehicle driving directions, that is, divided into 8 intervals, such as Fig. 9 As shown, the eight vehicle driving directions are AH. Among them, the vehicle driving direction A is in the range of 0°-44°, the vehicle driving direction B is in the range of 45°-90°, and so on, each vehicle driving direction occupies a range of 45 degrees.

[0231] It is understandable that, since each vehicle that may be in a congested state may move from the current frame to the next frame, and the direction may change during the movement, the detection device needs to determine the vehicle driving direction of each vehicle that may be in a congested state at the corresponding time.

[0232] The vehicle driving direction refers to the vehicle driving from the current frame to the next corresponding driving direction. The vehicle driving direction is calculated based on the tracking trajectory. For example, if the vehicle X moves from (0,0) to (1,0.8), the corresponding tracking trajectory is determined, and then the detection device calculates the driving angle of the vehicle X based on the tracking trajectory. The driving angle is in the range of 0°-45°, and then the detection device determines that the vehicle driving direction of the vehicle X is in the vehicle driving direction A.

[0233] like Fig. 9 As shown, the vehicle 1, the vehicle 2, the vehicle 3, the vehicle 4 and the vehicle 5 are traveling in a direction E, wherein the vehicle 11 is traveling in a direction B, and the vehicle 12 is traveling in a direction D. The black circles represent vehicles.

[0234] S803: If any vehicle driving direction includes vehicles that may be in a congested state whose number is greater than or equal to a preset number, the vehicles that may be in a congested state whose number is greater than or equal to the preset number are determined as direction clustered vehicles.

[0235] The preset congestion number is the number of vehicles that are congested. It is understandable that the preset congestion number can be consistent with the preset first candidate number, and there is no restriction here. Assume that the preset congestion number is 4 vehicles.

[0236] According to the above example, there are a total of 5 vehicles in the vehicle driving direction E, and the vehicle driving direction E includes vehicles that are likely to be in a congested state that are greater than or equal to the preset congestion number (4 vehicles), so the vehicles that are likely to be in a congested state that are greater than or equal to the preset congestion number (i.e., vehicle 1, vehicle 2, vehicle 3, vehicle 4, and vehicle 5) are determined as direction clustered vehicles.

[0237] The directionally clustered vehicles refer to vehicles that are in the same vehicle driving direction and include a number of vehicles that are greater than or equal to a preset number of vehicles that may be in a congested state.

[0238] S804, determining a direction clustering result corresponding to the vehicle driving direction based on the direction clustering vehicles; the direction clustering result includes a direction clustering vehicle identification.

[0239] The direction clustering result refers to the set of corresponding identifications of direction clustered vehicles.

[0240] Furthermore, the detection device determines that the direction clustering result is {1, 2, 3, 4, 5}, where 1 represents the identification of vehicle 1.

[0241] It should be noted that the vehicle 11 is traveling in a direction B. At this time, the vehicle traveling direction B includes vehicles that are less than the preset number of vehicles that may be in a congested state, so that the vehicle traveling direction is an idle vehicle traveling direction, and thus it is determined that the vehicle 11 is not in a congested state. Similarly, the vehicle 12 is not in a congested state. It should be noted that the idle vehicle traveling direction is a non-congested direction.

[0242] S805, in response to obtaining the direction clustering result, executing a preset distance strategy, and using the preset distance clustering strategy to perform cluster analysis on the target detection frame corresponding to the direction clustering vehicle identification to determine the secondary clustering result at the corresponding moment; the secondary clustering result includes the final detection frame identification.

[0243] Furthermore, for the directional clustered vehicles corresponding to the directional clustering results, the detection device continues to use the preset distance clustering strategy to perform clustering analysis on the target detection frame corresponding to the directional clustered vehicle identification in the next frame detection frame image. This method is consistent with the implementation methods of the above-mentioned embodiments three to six to determine the preprocessing clustering results at the corresponding moment, and the preprocessing clustering results at the corresponding moment are determined as secondary clustering results.

[0244] The corresponding time refers to the time corresponding to the processed detection frame picture. If it is the current frame detection frame picture, the corresponding time is the current frame. If it is the next frame detection frame picture, the corresponding time is the next frame.

[0245] Exemplarily, the directional clustering result is {1, 2, 3, 4, 5}, then the corresponding target detection frames are determined to be target detection frame 1, target detection frame 2, target detection frame 3, target detection frame 4, and target detection frame 5, respectively.

[0246] Furthermore, cluster analysis is performed based on the above target detection frames. Assuming that the final pre-processing clustering result is {1, 2, 3, 4, 5}, then {1, 2, 3, 4, 5} is determined as the secondary clustering result.

[0247] It should be noted that the directional clustering result is the identification of the directional clustered vehicles, the preprocessing clustering result and the secondary clustering result are the target detection box identification, and the objects represented by the identification here are different.

[0248] The present embodiment provides a method for detecting vehicle congestion. In the present embodiment, the tracking trajectory of each vehicle in a congested state is determined based on the next frame detection frame image. Due to movement, the vehicle driving direction corresponding to the possible congested vehicle is calculated according to the tracking trajectory, and the possible congested vehicles greater than or equal to the preset congestion number are determined as direction clustered vehicles, which conforms to the actual situation of vehicle movement. Then, a preset distance clustering strategy is used to perform cluster analysis on the target detection frames corresponding to the direction clustered vehicle identification, so as to determine that after the movement, each possible congested vehicle is still in the same cluster due to a relatively close distance, so the accuracy of the present embodiment is improved.

[0249] This embodiment also includes the following contents:

[0250] If any vehicle driving direction includes vehicles that may be in a congested state and the number of vehicles is less than a preset congestion number, the vehicle driving direction is determined to be a non-congested direction.

[0251] Furthermore, the detection device determines that the vehicle's driving direction is an idle driving direction.

[0252] Embodiment 8

[0253] This embodiment is a further refinement of any of the above embodiments, and this embodiment includes the following method.

[0254] Fig.10 A flow chart of a method for detecting vehicle congestion provided in Example 8. Fig.10 As shown, the specific steps are as follows.

[0255] S1001, obtaining original training sample images.

[0256] In one embodiment, the original training sample images are stored locally in advance, so that the detection device obtains the original training sample images locally.

[0257] It should be noted that the original training sample images may include unobstructed training vehicles and / or obstructed training vehicles and their corresponding labeled training detection frames.

[0258] S1002: randomly select at least one labeled training detection frame and its corresponding training vehicle from the original training sample image, and generate a new training sample image to be used.

[0259] Exemplarily, if the original training sample image includes a marked training detection frame and its corresponding training vehicle, the detection device selects the marked training detection frame and its corresponding training vehicle to generate a new training sample image to be used.

[0260] The size of the training sample image to be used is consistent with the size of the original training sample image.

[0261] S1003, adding occlusion marks to the training vehicles in the training sample pictures to be used according to a preset vehicle occlusion mode set to generate an expanded training sample picture set; the preset vehicle occlusion mode set includes multiple preset occlusion modes; the expanded training sample picture set includes expanded training sample pictures with at least one labeled training detection frame corresponding to multiple preset occlusion modes, and the number of expanded training sample pictures corresponding to each preset occlusion mode satisfies a preset number ratio.

[0262] Specifically, the training vehicle in the training sample image to be used is regarded as a rectangle, and then it is evenly divided into 9 grids, with three rows of grids, upper, middle and lower, each row of grids including three grids of equal area. Each grid represents the area of ​​the training vehicle on the training sample image to be used.

[0263] Fig.11 A schematic diagram of a training vehicle division method provided in Example 8. Fig.11 As shown, the 9 grids are 1-9 respectively.

[0264] Among them, the shielding methods include position shielding methods and grid number shielding methods. Among them, the position shielding methods include top shielding methods, bottom shielding methods and left and right shielding methods. Among them, the grid number shielding methods include single-grid shielding methods, double-grid shielding methods and three-grid shielding methods. It should be noted that the position shielding method and the grid number shielding method are three shielding methods, which can be combined in pairs to determine a total of 9 preset shielding methods, that is, to obtain a preset vehicle shielding method set. For example, the preset shielding method can be a top-single-grid shielding method and a bottom-three-grid shielding method. It can be understood that the preset vehicle shielding method set includes 9 preset shielding methods.

[0265] Among them, the three-grid blocking method can be a three-grid blocking method in the horizontal direction, a three-grid blocking method in the vertical direction, and a diagonal blocking method. Among them, the three-grid blocking method in the horizontal direction can be a grid 1-2-3 blocking method (i.e., blocking grids 1, 2, and 3), the three-grid blocking method in the vertical direction can be a grid 1-4-7 blocking method (i.e., blocking grids 1, 4, and 7), and the diagonal blocking method can be a grid 1-2-4 blocking method (i.e., blocking grids 1, 2, and 4). Among them, the top blocking method refers to blocking the top of the training vehicle (for example, blocking at least one of grids 1, 2, and 3), the bottom blocking method (for example, blocking at least one of grids 7, 8, and 9), and the left and right blocking method (for example, blocking at least one of grids 1, 4, and 7).

[0266] Exemplarily, assume that the training sample image to be used is blocked by a top-single-grid blocking method, that is, an occlusion mark is added to the top of the training vehicle in the training sample image to be used to simulate that a grid area on the top of the training vehicle is blocked, thereby generating an expanded training sample image.

[0267] According to the above method steps, multiple expanded training sample images can be obtained. The expanded training sample image set includes expanded training sample images of multiple preset occlusion modes, and the number of expanded training sample images corresponding to each preset occlusion mode meets the preset number ratio.

[0268] Exemplarily, it is assumed that the expanded training sample picture set includes expanded training sample pictures of the top-single-grid occlusion method and expanded training sample pictures of the left-right-three-grid occlusion method, and the ratio of the number of expanded training sample pictures of the top-single-grid occlusion method to the number of expanded training sample pictures of the left-right-three-grid occlusion method satisfies the preset number ratio.

[0269] Table 1 is a schematic table of the quantity probability of each preset occlusion method.

[0270] Position quantity probability Grid number probability Top occlusion mode: 0.1 Single grid occlusion mode: 0.1 Bottom occlusion mode: 0.4 Double grid occlusion mode: 0.4 Left and right occlusion mode: 0.5 Three-grid occlusion mode: 0.5

[0271] As shown in Table 1, if it is a top-single-grid occlusion method, the number probability is 0.1*0.1=0.01, and if it is a left-right-three-grid occlusion method, the number probability is 0.5*0.5=0.25. The number probability is the product of the position occlusion number probability and the grid number probability.

[0272] Further, the preset number ratio is calculated as the number probability of the top-single-grid occlusion mode and the number probability of the left-right-three-grid occlusion mode=1 / 25.

[0273] Exemplarily, assuming that the expanded training sample pictures include 26 expanded training sample pictures, since the preset quantity ratio is met, the number of expanded training sample pictures of the top-single-grid occlusion mode is one, and the number of expanded training sample pictures of the left-right-three-grid occlusion mode is 25.

[0274] This embodiment provides a method for detecting vehicle congestion. In this embodiment, an occlusion mark is added to the training vehicle in the training sample picture to be used according to the preset vehicle occlusion mode set, thereby indicating that the training vehicle is partially occluded. In this embodiment, the number of expanded training sample pictures corresponding to each preset occlusion mode satisfies the preset number ratio, and the preset number ratio can be determined according to the actual situation of vehicle congestion in real life, so that the expanded training sample pictures in this embodiment can be expanded according to the actual situation. In this embodiment, the original training sample pictures are used for expansion, and the training sample pictures can be increased, so as to prepare sufficient data for training the initial vehicle model.

[0275] Embodiment 9

[0276] This embodiment is a further refinement of any of the above embodiments, and includes the following contents.

[0277] Fig.12 A flow chart of a method for detecting vehicle congestion provided in Example 9. Fig.12 As shown, the specific steps are as follows.

[0278] S1201, obtaining a training sample set.

[0279] S1202: Use the training sample set to train the initial vehicle detection model.

[0280] The initial vehicle detection model is a vehicle detection model before training. The initial vehicle detection model corresponds to the initial model parameters at this time.

[0281] S1203: In response to satisfying a preset training convergence condition, determining a vehicle detection model satisfying the preset training convergence condition as a vehicle detection model trained to convergence.

[0282] The preset training convergence condition is a pre-set convergence condition, which may be a setting for a loss value.

[0283] It should be noted that the vehicle detection model trained to convergence in the present application may be a converged Yolov5 model or other models.

[0284] It should be noted that the traditional Yolov5 model includes three detection layers, and corresponds to three sets of initialization detection vehicle aspect ratios. In this embodiment, several operation layers are added, and after the training sample image passes through multiple operation layers, the processed training sample image continues to be upsampled to obtain an enlarged training sample image, and then continues to be enlarged to obtain the final enlarged training image, until the size of the obtained training sample image is 160*160, and the 160*160 training sample image is feature fused with the features in the training sample image obtained in the second layer, and then the enlarged feature image is obtained, so as to perform small target detection based on the enlarged feature image. Therefore, this embodiment can detect small targets.

[0285] Embodiment 10

[0286] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to determine that vehicles on a trunk road are in a congested state based on the secondary clustering result and a preset duration.

[0287] Fig.13 A flow chart of a method for detecting vehicle congestion provided in Example 10. Fig.13 As shown, the specific steps are as follows.

[0288] S1301, determining the corresponding final vehicle based on the final detection box identifier in the secondary clustering result.

[0289] According to the description of the above-mentioned embodiment 7, the secondary clustering result is {1, 2, 3, 4, 5}, so that the target detection frames at the corresponding moments are determined to be target detection frame 1, target detection frame 2, target detection frame 3, target detection frame 4 and target detection frame 5. It should be noted that the target detection frame identifier in the secondary clustering result is the target detection frame identifier in the next frame detection frame picture, not the target detection frame identifier in the current frame detection picture. Since the vehicle does not change between the current frame detection picture and the next frame detection picture, the vehicle and the target detection frame can still correspond.

[0290] Further, the detection device determines the corresponding final vehicles as vehicles 1 to 5 according to target detection frame 1, target detection frame 2, target detection frame 3, target detection frame 4 and target detection frame 5. It should be noted that the target detection identifier in the secondary clustering result is the final detection frame identifier.

[0291] S1302: Determine at least one secondary clustering result corresponding to all moments included in a preset duration.

[0292] According to the description of S204, assuming that the preset duration includes two moments in the future, namely the third moment and the fourth moment, assuming that the current frame corresponds to the first moment and the next frame corresponds to the second moment, the detection device determines at least one secondary clustering result corresponding to the third moment and the fourth moment, respectively. The third moment and the fourth moment are all moments.

[0293] S1303: If there is a secondary clustering result at all times that includes a final detection box identifier corresponding to the final vehicle, it is determined that the final vehicle on the trunk road is in a congested state.

[0294] Exemplarily, it is assumed that the third moment corresponds to secondary clustering result A and secondary clustering result B, and the fourth moment corresponds to secondary clustering result C.

[0295] Further, the detection device determines that the secondary clustering result A includes final detection frame identifiers corresponding to vehicles 1 to 5, thereby determining that the final vehicles 1 to 5 on the trunk road are in a congested state.

[0296] The present embodiment provides a method for detecting vehicle congestion. In the present embodiment, at least one secondary clustering result corresponding to all the included moments is determined according to a preset time length. If the secondary clustering results corresponding to all the moments include the final detection mark of the final vehicle, it means that the final vehicle continues to move slowly on the trunk road, and the distance between each final vehicle is relatively close, and then the final vehicle is determined to be in a congested state.

[0297] Embodiment 11

[0298] The following is an embodiment of the device of the present application. Fig.14 This is a schematic diagram of the structure of a vehicle congestion detection device provided in Example 11. Fig.14 As shown, the detection device 1400 includes the following modules:

[0299] The receiving module 1401 is used to receive the current frame monitoring picture on the trunk road in real time, and the obtaining module 1402 is used to obtain the current frame detection frame picture based on the current frame monitoring picture; the current frame detection frame picture includes multiple target detection frames; the target detection frame surrounds the vehicle in the current frame monitoring picture; the detection frame is a rectangular frame;

[0300] The clustering module 1403 is used to perform cluster analysis on multiple target detection frames using a preset distance clustering strategy to obtain a preprocessing clustering result; the preprocessing clustering result includes a preprocessing detection frame identifier that meets the preset preprocessing clustering condition; the preset distance clustering strategy is a strategy for clustering the distance between each adjacent vehicle; the vehicle corresponding to the preprocessing detection frame identifier may be a vehicle in a congested state; the distance between adjacent vehicles is the straight-line distance between the center points of the two target detection frames corresponding to the adjacent vehicles;

[0301] The tracking module 1404 is used to track the vehicle that may be in a congested state to obtain the next frame detection frame image corresponding to the vehicle that may be in a congested state. The clustering determination module 1405 is used to determine the secondary clustering result of the next frame detection frame image at the corresponding time based on the preset position and direction clustering strategy and the preset distance clustering strategy; the preset position and direction clustering strategy is a strategy for clustering the driving position and direction of each vehicle;

[0302] The congestion determination module 1406 is used to determine that the vehicles on the trunk road are in a congested state based on the secondary clustering result and a preset duration.

[0303] In one embodiment, the obtaining module 1402, when obtaining the current frame detection frame image based on the current frame monitoring image, is specifically used to:

[0304] The current frame monitoring picture is input into the vehicle detection model trained to convergence to obtain the current frame detection frame picture; the vehicle detection model trained to convergence is obtained based on the training sample set; the training sample set includes the original training sample picture set and the expanded training sample picture set; the original training sample picture set includes at least one training sample picture; the expanded training sample picture set includes at least one expanded training sample picture; the training sample picture includes a labeled training detection frame; the labeled training detection frame surrounds the training vehicle in the training sample picture; the labeled training detection frame is a rectangular frame; the expanded training sample picture is a sample picture generated based on the original training sample picture and taking into account a variety of preset vehicle occlusion methods.

[0305] In one embodiment, the clustering module 1403, when performing cluster analysis on multiple target detection frames using a preset distance clustering strategy to obtain a preprocessing clustering result, is specifically used to:

[0306] When executing the preset distance clustering strategy, determine the target detection frame with the largest area from multiple target detection frames, and obtain the target detection frame width and target detection frame height corresponding to the target detection frame with the largest area;

[0307] The largest value between the values ​​corresponding to the target detection frame width and the target detection frame height is determined as the initial distance threshold;

[0308] The preprocessing clustering results are obtained based on the initial distance threshold.

[0309] In one embodiment, the clustering module 1403, when obtaining the pre-processing clustering result based on the initial distance threshold, is specifically used to:

[0310] Determine each target detection frame as a central detection frame, and determine its adjacent target adjacent detection frame based on the central detection frame, and make the straight-line distance between the center point of the target adjacent detection frame and the center point of the central detection frame less than or equal to the initial distance threshold;

[0311] Determine a set of clusters to be selected; the set of clusters to be selected includes a center detection box identifier and its adjacent target adjacent detection box identifiers;

[0312] Continue to determine each target adjacent detection frame as a central detection frame, and determine the target adjacent detection frames adjacent to the central detection frame based on the central detection frame;

[0313] Add the newly determined target adjacent detection box identifier to the to-be-selected cluster set, and continue to perform the steps of determining each target adjacent detection box as the center detection box to adding the newly determined target adjacent detection box identifier to the to-be-selected cluster set until there is no newly determined target adjacent detection box, so as to obtain the final to-be-selected cluster set corresponding to each target detection box as the center detection box;

[0314] The preprocessing clustering result is determined based on the final candidate cluster set.

[0315] In one embodiment, the clustering module 1403, when determining the pre-processing clustering result based on the final candidate cluster set, is specifically used to:

[0316] Determine the target detection frame corresponding to each target detection frame identifier from the final set of clusters to be selected as the reference detection frame;

[0317] Determine a distance range detection frame within a preset distance range of the reference detection frame, and determine a near detection frame width or near detection frame height of the nearest distance range detection frame as an update distance threshold; there is at least one distance range detection frame; the update distance threshold is the maximum value of corresponding values ​​of the near detection frame width and the near detection frame height;

[0318] If the straight-line distances between the reference detection frame and the detection frames in each distance range are greater than a preset multiple of the update distance threshold, the reference detection frame identifier is removed from the final candidate cluster set to obtain a preprocessed cluster set;

[0319] If the straight-line distances between the reference detection frame and the detection frames in each distance range are less than or equal to the preset multiple of the update distance threshold, the reference detection frame is retained to obtain a preprocessed clustering set;

[0320] In response to the preprocessing cluster set satisfying a preset preprocessing cluster condition, all target detection box identifiers included in the preprocessing cluster set are determined as preprocessing detection box identifiers, and the preprocessing cluster set is determined as a preprocessing cluster result;

[0321] This embodiment provides a vehicle congestion detection device, and the clustering module 1403 is further used to determine that the vehicles corresponding to the preprocessing cluster set are not in a congested state in response to the preprocessing cluster set not satisfying the preset preprocessing cluster condition.

[0322] In one embodiment, the clustering module 1403, when determining the adjacent target adjacent detection frames based on the center detection frame, is specifically configured to:

[0323] Based on the central detection frame, determine all adjacent adjacent detection frames to be selected;

[0324] Calculate the straight-line distance between the center point of the central detection frame and the center points of each adjacent detection frame to be selected respectively;

[0325] If the straight-line distance is less than or equal to the initial distance threshold, the candidate adjacent detection frame corresponding to the straight-line distance is determined as the target adjacent detection frame;

[0326] This embodiment provides a vehicle congestion detection device, and the clustering module 1403 is further used to: if the straight-line distance is greater than the initial distance threshold, determine that the candidate adjacent detection box corresponding to the straight-line distance is not the target adjacent detection box.

[0327] In one embodiment, the cluster determination module 1405, when determining the secondary clustering result at the corresponding moment of the next frame of the detection frame image based on the preset position and direction clustering strategy and the preset distance clustering strategy, is specifically used to:

[0328] When executing the preset position and direction clustering strategy, the tracking trajectory of each vehicle that may be in a congested state is determined based on the next frame detection frame image; the tracking trajectory reflects the position change and direction change of the vehicle that may be in a congested state;

[0329] Calculate the driving directions of all vehicles that may be in congestion based on the tracking trajectory;

[0330] If any vehicle driving direction includes vehicles that may be in a congested state with a number greater than or equal to a preset number of congested vehicles, the vehicles that may be in a congested state with a number greater than or equal to the preset number of congested vehicles are determined as direction clustering vehicles;

[0331] Determine the direction clustering result corresponding to the vehicle driving direction based on the direction clustering vehicles; the direction clustering result includes the direction clustering vehicle identification;

[0332] In response to obtaining the direction clustering result, executing a preset distance strategy, and using the preset distance clustering strategy to perform cluster analysis on the target detection frame corresponding to the direction clustering vehicle identification, so as to determine a secondary clustering result at a corresponding moment; the secondary clustering result includes a final detection frame identification;

[0333] This embodiment provides a vehicle congestion detection device, and the cluster determination module 1405 is further used to: if any vehicle driving direction includes vehicles that may be in a congested state less than a preset number of congested vehicles, then determine the vehicle driving direction as a non-congested direction.

[0334] In one approach, this embodiment provides a vehicle congestion detection device, further comprising an acquisition module and a generation module;

[0335] An acquisition module is used to obtain original training sample images;

[0336] A generation module is used to randomly select at least one labeled training detection frame and its corresponding training vehicle in the original training sample image, and generate a new stand-by training sample image; the generation module is also used to add occlusion marks to the training vehicles in the stand-by training sample image according to a preset vehicle occlusion mode set to generate an extended training sample image set; the preset vehicle occlusion mode set includes multiple preset occlusion modes; the extended training sample image set includes multiple preset occlusion modes corresponding to extended training sample images with at least one labeled training detection frame, and the number of extended training sample images corresponding to each preset occlusion mode meets the preset number ratio.

[0337] In one approach, this embodiment provides a vehicle congestion detection device, further comprising a training module and a training determination module;

[0338] The acquisition module is also used to obtain the training sample set;

[0339] A training module uses a training sample set to train the initial vehicle detection model;

[0340] The training determination module is used to determine the vehicle detection model that meets the preset training convergence condition as the vehicle detection model that has been trained to convergence in response to meeting the preset training convergence condition.

[0341] In one embodiment, when the congestion determination module 1406 determines that the vehicles on the trunk road are in a congested state based on the secondary clustering result and the preset duration, it is specifically configured to:

[0342] Determine the corresponding final vehicle based on the final detection box identifier in the secondary clustering result; determine at least one secondary clustering result corresponding to all moments included in the preset time length; if there is a secondary clustering result in all moments that includes the final detection box identifier corresponding to the final vehicle, it is determined that the final vehicle on the trunk road is in a congested state.

[0343] Embodiment 12

[0344] Fig.15 This is a schematic diagram of the structure of an electronic device provided in Example 12. Fig.15 As shown, the electronic device 1500 may include: a processor 1501, and a memory 1502 and a transceiver 1503 that are communicatively connected to the processor 1501. The memory 1502 stores computer-executable instructions; the transceiver 1503 is used to send and receive data; the processor 1501 executes the computer-executable instructions stored in the memory 1502 to implement any one of the method embodiments in the above-mentioned embodiments 1 to 10, and the specific implementation methods and technical effects are similar, which will not be repeated here.

[0345] In this embodiment, the transceiver 1503, the memory 1502 and the processor 1501 are connected via a bus. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0346] Embodiment 13

[0347] The present application provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement any one of the method embodiments of Embodiments 1 to 10 above. The specific implementation methods and technical effects are similar and will not be repeated here.

[0348] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0349] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for detecting vehicle congestion, characterized in that: The method comprises: receiving a current frame monitoring picture on a trunk road in real time, and obtaining a current frame detection frame picture based on the current frame monitoring picture; the current frame detection frame picture includes a plurality of target detection frames; the target detection frame surrounds the vehicle in the current frame monitoring picture; the detection frame is a rectangular frame; A preset distance clustering strategy is used to perform cluster analysis on the multiple target detection frames to obtain a preprocessing clustering result; the preprocessing clustering result includes a preprocessing detection frame identifier that meets the preset preprocessing clustering condition; the preset distance clustering strategy is a strategy for clustering the distance between adjacent vehicles; the vehicle corresponding to the preprocessing detection frame identifier may be a vehicle in a congested state; the distance between adjacent vehicles is the straight-line distance between the center points of the two target detection frames corresponding to the adjacent vehicles; Track the vehicle that may be in a congested state to obtain a next frame detection frame image corresponding to the vehicle that may be in a congested state, and determine a secondary clustering result of the next frame detection frame image at a corresponding moment based on a preset position and direction clustering strategy and a preset distance clustering strategy; the preset position and direction clustering strategy is a strategy for clustering the driving position and direction of each vehicle; Based on the secondary clustering result and the preset duration, it is determined that the vehicles on the trunk road are in a congested state.

2. The method according to claim 1, characterized in that The obtaining of a current frame detection frame picture based on the current frame monitoring picture includes: The current frame monitoring picture is input into the vehicle detection model trained to convergence to obtain the current frame detection frame picture; the vehicle detection model trained to convergence is obtained based on the training sample set; the training sample set includes an original training sample picture set and an expanded training sample picture set; the original training sample picture set includes at least one training sample picture; the expanded training sample picture set includes at least one expanded training sample picture; the training sample picture includes a labeled training detection frame; the labeled training detection frame surrounds the training vehicle in the training sample picture; the labeled training detection frame is a rectangular frame; the expanded training sample picture is a sample picture generated based on the original training sample picture and taking into account multiple preset vehicle occlusion methods.

3. The method according to claim 1, characterized in that The adopting a preset distance clustering strategy to perform clustering analysis on the multiple target detection frames to obtain a preprocessing clustering result includes: When executing the preset distance clustering strategy, determining the target detection frame with the largest area from multiple target detection frames, and obtaining the target detection frame width and target detection frame height corresponding to the target detection frame with the largest area; The maximum value among the values ​​corresponding to the target detection frame width and the target detection frame height is determined as the initial distance threshold; A preprocessing clustering result is obtained based on the initial distance threshold.

4. The method according to claim 3, characterized in that The obtaining of the preprocessing clustering result based on the initial distance threshold comprises: Determine each target detection frame as a central detection frame, and determine its adjacent target adjacent detection frame based on the central detection frame, and make the straight-line distance between the center point of the target adjacent detection frame and the center point of the central detection frame less than or equal to the initial distance threshold; Determine a cluster set to be selected; the cluster set to be selected includes a center detection box identifier and its adjacent target adjacent detection box identifiers; Continue to determine each of the target adjacent detection frames as a central detection frame, and determine target adjacent detection frames adjacent to the central detection frame based on the central detection frame; Adding the newly determined target adjacent detection box identifier to the to-be-selected cluster set, and continuing to perform the steps of determining each target adjacent detection box as a central detection box to adding the newly determined target adjacent detection box identifier to the to-be-selected cluster set, until there is no newly determined target adjacent detection box, so as to obtain a final to-be-selected cluster set corresponding to each target detection box as a central detection box; A preprocessing clustering result is determined based on the final set of clusters to be selected.

5. The method according to claim 4, characterized in that The determining of the preprocessing clustering result based on the final set of clusters to be selected includes: Determining the target detection frame corresponding to each target detection frame identifier from the final to-be-selected cluster set as a reference detection frame; Determine a distance range detection frame within a preset distance range of the reference detection frame, and determine a near detection frame width or near detection frame height of the nearest distance range detection frame as an update distance threshold; the distance range detection frame is at least one; the update distance threshold is the maximum value of corresponding values ​​of the near detection frame width and the near detection frame height; If the straight-line distances between the reference detection frame and each of the distance range detection frames are greater than the preset multiple of the update distance threshold, the reference detection frame identifier is removed from the final to-be-selected cluster set to obtain a preprocessed cluster set; If the straight-line distances between the reference detection frame and each of the distance range detection frames are less than or equal to the preset multiple of the update distance threshold, retain the reference detection frame to obtain a preprocessing cluster set; In response to the preprocessing cluster set satisfying a preset preprocessing cluster condition, all target detection box identifiers included in the preprocessing cluster set are determined as preprocessing detection box identifiers, and the preprocessing cluster set is determined as a preprocessing cluster result; The method further comprises: In response to the preprocessing cluster set not satisfying a preset preprocessing cluster condition, it is determined that the vehicles corresponding to the preprocessing cluster set are not in a congested state.

6. The method according to claim 4, characterized in that The determining of the adjacent target adjacent detection frame based on the central detection frame includes: Based on the central detection frame, determine all adjacent adjacent detection frames to be selected; Calculate the straight-line distance between the center point of the central detection frame and the center points of each adjacent detection frame to be selected respectively; If the straight-line distance is less than or equal to the initial distance threshold, determining the candidate adjacent detection frame corresponding to the straight-line distance as the target adjacent detection frame; The method further comprises: If the straight-line distance is greater than the initial distance threshold, it is determined that the to-be-selected adjacent detection frame corresponding to the straight-line distance is not the target adjacent detection frame.

7. The method according to claim 1, characterized in that The determining of the secondary clustering result of the next frame detection frame picture at the corresponding moment based on the preset position and direction clustering strategy and the preset distance clustering strategy includes: When executing the preset position and direction clustering strategy, the tracking trajectory of each vehicle that may be in a congested state is determined based on the next frame detection frame image; the tracking trajectory reflects the position change and direction change of the vehicle that may be in a congested state; Calculate the vehicle driving directions corresponding to all the vehicles that may be in a congested state based on the tracking trajectory; If any vehicle driving direction includes vehicles that may be in a congested state with a number greater than or equal to a preset number of congested vehicles, the vehicles that may be in a congested state with a number greater than or equal to the preset number of congested vehicles are determined as direction clustering vehicles; Determining a direction clustering result corresponding to the driving direction based on the direction clustering vehicles; the direction clustering result includes a direction clustering vehicle identifier; In response to obtaining the directional clustering result, executing the preset distance strategy, and using the preset distance clustering strategy to perform cluster analysis on the target detection frame corresponding to the directional clustering vehicle identification to determine a secondary clustering result at a corresponding moment; the secondary clustering result includes a final detection frame identification; The method further comprises: If any vehicle driving direction includes vehicles that may be in a congested state and the number of vehicles is less than a preset congestion number, the vehicle driving direction is determined to be a non-congested direction.

8. The method according to claim 2, characterized in that: The method further comprises: Get the original training sample images; Randomly selecting at least one labeled training detection frame and its corresponding training vehicle from the original training sample image, and generating a new training sample image to be used; Adding occlusion marks to the training vehicles in the stand-by training sample pictures according to a preset vehicle occlusion mode set to generate an expanded training sample picture set; the preset vehicle occlusion mode set includes multiple preset occlusion modes; the expanded training sample picture set includes multiple preset occlusion modes corresponding to expanded training sample pictures with the at least one labeled training detection frame, and the number of expanded training sample pictures corresponding to each preset occlusion mode satisfies a preset number ratio.

9. The method according to claim 8, characterized in that The method further comprises: Obtain a training sample set; Using the training sample set to train an initial vehicle detection model; In response to satisfying a preset training convergence condition, the vehicle detection model satisfying the preset training convergence condition is determined as the vehicle detection model trained to convergence.

10. The method according to any one of claims 1 to 9, characterized in that: The determining that the vehicles on the trunk road are in a congested state based on the secondary clustering result and the preset duration includes: Determine the corresponding final vehicle based on the final detection box identifier in the secondary clustering result; Determine at least one secondary clustering result corresponding to all moments included in the preset duration; If there is a secondary clustering result at all times that includes a final detection frame identifier corresponding to the final vehicle, it is determined that the final vehicle on the trunk road is in a congested state.

11. A vehicle congestion detection device, characterized in that: The device comprises: A receiving module is used to receive a current frame monitoring picture on a trunk road in real time, and an obtaining module is used to obtain a current frame detection frame picture based on the current frame monitoring picture; the current frame detection frame picture includes a plurality of target detection frames; the target detection frame surrounds the vehicle in the current frame monitoring picture; the detection frame is a rectangular frame; A clustering module, configured to perform cluster analysis on the plurality of target detection frames using a preset distance clustering strategy to obtain a preprocessing clustering result; the preprocessing clustering result includes a preprocessing detection frame identifier that satisfies a preset preprocessing clustering condition; the preset distance clustering strategy is a strategy for clustering the distances between adjacent vehicles; the vehicle corresponding to the preprocessing detection frame identifier may be a vehicle in a congested state; the distance between adjacent vehicles is a straight-line distance between the center points of the two target detection frames corresponding to the adjacent vehicles; A tracking module is used to track the vehicle that may be in a congested state to obtain a next frame detection frame image corresponding to the vehicle that may be in a congested state. A clustering determination module is used to determine a secondary clustering result of the next frame detection frame image at a corresponding moment based on a preset position and direction clustering strategy and a preset distance clustering strategy; the preset position and direction clustering strategy is a strategy for clustering the driving position and direction of each vehicle; The congestion determination module is used to determine that the vehicles on the trunk road are in a congested state based on the secondary clustering result and a preset duration.

12. An electronic device comprising: A processor, and a memory and a transceiver communicatively connected to the processor; The memory stores computer-executable instructions; the transceiver is used to send and receive data; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 10 when executed by a processor.

Images