Traffic anomaly detection method, device and storage medium

Abstract

The present application discloses a traffic anomaly detection method, device and storage medium, the traffic anomaly detection method comprising: performing anomaly recognition processing and image quality analysis processing on each vehicle image to be detected, obtaining anomaly detection information and image quality information of each vehicle image; determining the detection result of the vehicle to be detected according to the anomaly detection information and image quality information of each vehicle image to be detected; determining whether the vehicle to be detected has a traffic anomaly event according to the detection result. The above scheme can improve the efficiency of traffic anomaly detection.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a traffic anomaly detection method, device and storage medium. Background Art

[0002] In the current security scenario, smart traffic management has always been a focus of attention, among which the identification of abnormal traffic events is a very important part.

[0003] At present, the traffic anomaly detection method mainly uses image recognition technology to identify vehicles in traffic scene images to determine whether there is traffic anomaly, and records the vehicles and / or their drivers with traffic anomalies.

[0004] However, in traditional methods, it is difficult to make a relatively accurate judgment when the image quality is poor or the target object in the traffic scene image is small, which may lead to false alarms and affect the efficiency of traffic anomaly detection. Summary of the invention

[0005] The present application at least provides a traffic anomaly detection method, device, equipment and computer-readable storage medium.

[0006] The first aspect of the present application provides a traffic anomaly detection method, comprising: performing anomaly recognition processing and image quality analysis processing on each vehicle image of a vehicle to be detected, respectively, to obtain anomaly detection information and image quality information of each vehicle image; determining a detection result of the vehicle to be detected based on the anomaly detection information and image quality information of each vehicle image of the vehicle to be detected; and determining whether the vehicle to be detected has a traffic anomaly event based on the detection result.

[0007] In one embodiment, the detection results include normal detection results and abnormal detection results, and the step of determining whether the vehicle to be detected has a traffic abnormality event based on the detection results includes: determining the traffic abnormality rate of the vehicle to be detected based on the normal detection results and the abnormal detection results; in response to the traffic abnormality rate being greater than a target abnormality threshold, determining that the vehicle to be detected has a traffic abnormality event.

[0008] In one embodiment, before the step of determining that a traffic abnormality event exists for the vehicle to be detected in response to the traffic abnormality rate being greater than a target abnormality threshold, the method further includes: determining a target driving distance of the vehicle to be detected based on a vehicle image of the vehicle to be detected; the step of determining that a traffic abnormality event exists for the vehicle to be detected in response to the traffic abnormality rate being greater than a target abnormality threshold includes: determining that a traffic abnormality event exists for the vehicle to be detected in response to the traffic abnormality rate being greater than the target abnormality threshold and the target driving distance being greater than a target distance threshold.

[0009] In one embodiment, the target abnormality threshold and the target driving distance are negatively correlated. Before the step of determining that a traffic abnormality event exists for the vehicle to be detected in response to the traffic abnormality rate being greater than the target abnormality threshold and the target driving distance being greater than the target distance threshold, the method further includes: obtaining a preset abnormality threshold; and performing negative correlation adjustment processing on the preset abnormality threshold according to the target driving distance of the vehicle to be detected to obtain the target abnormality threshold corresponding to the vehicle to be detected.

[0010] In one embodiment, the step of determining the target driving distance of the vehicle to be detected based on the vehicle image of the vehicle to be detected includes: determining the target point positions of the vehicle to be detected during driving based on the vehicle image of the vehicle to be detected; and determining the target driving distance of the vehicle to be detected based on the distance between each target point position.

[0011] In one embodiment, after the step of determining the target point of the vehicle to be detected during driving according to the vehicle image of the vehicle to be detected, the method further includes: determining the point abnormality rate of each target point according to the normal detection results and abnormal detection results of each vehicle to be detected in each target point; and determining the abnormal point according to the point abnormality rate of each target point.

[0012] In one embodiment, the step of performing abnormality recognition processing and image quality analysis processing on each vehicle image of the vehicle to be detected to obtain abnormality detection information and image quality information of each vehicle image includes: inputting each vehicle image of the vehicle to be detected into a pre-trained traffic anomaly recognition model to obtain abnormality detection information corresponding to each vehicle image output by the traffic anomaly recognition model; inputting each vehicle image of the vehicle to be detected into a pre-trained image quality assessment model to obtain image quality information corresponding to each vehicle image output by the image quality assessment model.

[0013] In one embodiment, the step of determining the detection result of the vehicle to be detected based on the abnormality detection information and image quality information of each vehicle image of the vehicle to be detected includes: performing weighted processing based on the abnormality detection information and image quality information of each vehicle image to obtain a weighted result of each vehicle image; and summing the weighted results of each vehicle image of the vehicle to be detected to obtain the detection result of the vehicle to be detected.

[0014] The second aspect of the present application provides a traffic anomaly detection device, including: an image processing module, used to perform anomaly recognition processing and image quality analysis processing on each vehicle image of the vehicle to be detected, so as to obtain anomaly detection information and image quality information of each vehicle image; a detection module, used to determine the detection result of the vehicle to be detected based on the anomaly detection information and image quality information of each vehicle image of the vehicle to be detected; and an anomaly determination module, used to determine whether the vehicle to be detected has a traffic anomaly event based on the detection result.

[0015] A third aspect of the present application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned traffic anomaly detection method.

[0016] A fourth aspect of the present application provides a computer-readable storage medium on which program instructions are stored. When the program instructions are executed by a processor, the above-mentioned traffic anomaly detection method is implemented.

[0017] The above scheme obtains abnormality detection information and image quality information of each vehicle image by performing abnormality recognition processing and image quality analysis processing on each vehicle image to be detected. In the traditional method, it is difficult to make an accurate judgment when the image quality is poor or the detected target is small, which easily leads to false detection problems; while the present application determines the detection result of the vehicle to be detected based on the abnormality detection information and image quality information of each vehicle image of the vehicle to be detected, so that the detection result is more accurate and reliable. It is determined based on the detection result whether the vehicle to be detected has a traffic abnormality event, thereby improving the detection accuracy of traffic abnormalities.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0020] Figure 1 is a flow chart of an exemplary embodiment of the traffic anomaly detection method of the present application;

[0021] Figure 2 is a block diagram of a traffic anomaly detection device shown in an exemplary embodiment of the present application;

[0022] Figure 3 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0023] Figure 4It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0024] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0025] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0026] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0027] It is understandable that the specific implementation of this application involves the acquisition, use and processing of some information and / or data. When the above embodiments of this application are applied to specific products or technologies, it is necessary to obtain the permission or consent of the relevant users and comply with the relevant rules and standards of the relevant regions.

[0028] See also Figure 1 , Figure 1 This is a flow chart of an exemplary embodiment of the traffic anomaly detection method of the present application. Specifically, it may include the following steps:

[0029] Step S110 , performing abnormality recognition processing and image quality analysis processing on each vehicle image to be detected, to obtain abnormality detection information and image quality information of each vehicle image.

[0030] Among them, there may be one or more vehicles to be detected in the present application, which is not limited here. The vehicles to be detected may include but are not limited to motor vehicles and non-motor vehicles. Each vehicle to be detected may correspond to one or more vehicle images. For example, the detection method of the present application can be applied to a mature traffic detection system that has been built. Usually, such a system stores a large number of vehicle images of each vehicle to be detected that can be used for traffic anomaly detection. Taking multiple vehicle images of the same vehicle to be detected as an example, the multiple vehicle images may be collected at the same point and / or collected at different points, which is not limited here.

[0031] It should be noted that in the present application, before the vehicle images are subjected to abnormality recognition processing and image quality analysis processing respectively, the acquired vehicle images can also be aggregated so that multiple vehicle images in the traffic detection system can be aggregated into one or more vehicle image sets. The aggregation processing can refer to the aggregation scheme in the prior art (for example, using the trained image aggregation model, etc.), which will not be described in detail here. Each vehicle to be detected can correspond to at least one vehicle image set, which is equivalent to clustering the acquired vehicle images into the image set of each vehicle to be detected, so as to facilitate traffic anomaly detection for each vehicle to be detected, and then performing abnormality recognition processing and image quality analysis processing on each vehicle image in each vehicle image set, as well as subsequent processing, etc., which will not be described in detail here. Among them, the basis for the aggregation processing of the vehicle image can be based on the vehicle identification in the vehicle image, and the vehicle identification at least includes the vehicle license plate information (such as the license plate number, the license plate color, etc.), and can also be the license plate information and the vehicle brand information, or the license plate information, the vehicle brand information and the vehicle appearance information, etc., which are not limited here.

[0032] In addition, the basis for the aggregation processing of vehicle images can also be based on the characteristic information of the vehicle driver in the vehicle image, such as the appearance information of the vehicle driver, etc., or the characteristic information of the vehicle driver and the identification of the vehicle driven by the driver can be combined as the basis for the aggregation processing, which will not be described in detail here. It is equivalent to that after the aggregation processing, each vehicle driver can correspond to at least one vehicle image set, and then each vehicle image in each vehicle image set is respectively subjected to abnormality recognition processing and image quality analysis processing, as well as subsequent processing, etc., which will not be described in detail here.

[0033] Abnormal identification processing refers to abnormal identification of vehicle images to analyze whether there are any traffic abnormalities in the vehicle to be detected in the vehicle image. For example, for non-motor vehicles, it may include but is not limited to abnormal helmet wearing, abnormal number of passengers, abnormal driving (such as driving on the solid line, running a red light, and driving in the opposite direction), and abnormal awning installation. For motor vehicles, it may include but is not limited to abnormal driving (such as driving on the solid line, running a red light, and driving in the opposite direction), abnormal license plate obstruction, and abnormal vehicle appearance, etc., which will not be elaborated here.

[0034] Image quality analysis processing refers to performing image quality analysis on vehicle images to determine the image quality of each vehicle image. Exemplarily, it can be performed by performing image quality analysis through a neural network model to obtain image quality information. It can also be performed by obtaining image quality parameters of the vehicle image (such as one or more parameters of clarity, resolution, sharpness, brightness, pixel ratio of the inspected object in the vehicle image, etc.), and determining image quality information based on image quality parameters, etc., which will not be described in detail here.

[0035] Exemplarily, for each vehicle image (vehicle image set) of a vehicle to be detected, each vehicle image is subjected to abnormality recognition processing and image quality analysis processing to obtain abnormality detection information and image quality information of each vehicle image. Abnormality detection information refers to detection information obtained in the process of detecting a vehicle image, which can characterize whether there is an abnormality in the vehicle to be detected in the vehicle image and / or its abnormality type. Image quality information refers to quality information obtained in the process of analyzing a vehicle image, which can characterize the image quality of the vehicle image, such as an image quality score, etc., which is not limited here.

[0036] It should also be noted that the abnormal identification processing and image quality analysis processing of the present application can be executed in parallel or successively, and the execution order is not limited here. In order to reduce the amount of data processing and improve data processing efficiency. Before the abnormal identification processing and image quality analysis processing of the present application, the vehicle image of each vehicle to be detected can be deduplicated. For example, for a certain vehicle to be detected, its vehicle images are sorted according to the image acquisition time information. If there are two images whose acquisition time interval is less than the preset acquisition time interval threshold time_thres1, and the acquisition positions of the two images are the same, only the preferred image (such as an image with a larger resolution or an image with a larger image size, etc.) is retained. If the preferred image cannot be determined, one of the two images can be selected to be retained.

[0037] Furthermore, if it is necessary to clearly divide the vehicle types (motor vehicles and non-motor vehicles) for traffic anomaly detection in a specific application scenario, for example, if non-motor vehicles need to be detected, each vehicle image can be screened and processed to obtain vehicle images of non-motor vehicles, thereby forming a new vehicle image set.

[0038] Step S120, determining a detection result of the vehicle to be detected according to the abnormality detection information and the image quality information of each vehicle image of the vehicle to be detected.

[0039] In conjunction with the above steps, it can be understood that in the prior art, the process of abnormality recognition processing of vehicle images is easily interfered by various factors, so that the abnormality recognition result may not accurately indicate whether there is a traffic abnormality in the vehicle to be detected.

[0040] Therefore, in this application, by combining the abnormality detection information obtained by performing abnormality recognition processing on the vehicle image, and the image quality information obtained by performing image quality analysis processing on the vehicle image, a comprehensive judgment is made as to whether there is a traffic anomaly in the vehicle to be detected, thereby improving the accuracy of traffic anomaly detection and further optimizing the efficiency of traffic anomaly detection.

[0041] For example, in the present application, the abnormality detection information and image quality information of each vehicle image of the vehicle to be detected can be combined by weighted summation to determine the detection result of the vehicle to be detected. Alternatively, a logical judgment method can be used to perform a logical judgment based on the abnormality detection information and image quality information of each vehicle image of the vehicle to be detected to determine the detection result of the vehicle to be detected.

[0042] For example, for the weighted summation method, the abnormality detection information and image quality information of each vehicle image are weighted to obtain a weighted result; the weighted results of each vehicle image of the vehicle to be detected are then summed to obtain a weighted summation result; the weighted summation result is compared with a preset threshold to obtain a detection result. For the logical judgment method, it can be determined whether the vehicle image has an abnormality based on the abnormality detection information, and whether the image quality meets the standard based on the image quality information; the detection result is determined based on the above logical judgment results.

[0043] Step S130, determining whether the vehicle to be detected has any abnormal traffic event according to the detection result.

[0044] In conjunction with the above steps, the detection results in the present application may include normal detection results and abnormal detection results. Among them, the normal detection result indicates that the vehicle to be detected is driving normally, and the abnormal detection result indicates that the vehicle to be detected is driving abnormally. It should be noted that each vehicle to be detected may correspond to one or more detection results, which is not limited here. For example, the vehicle images in the vehicle image set of each vehicle to be detected are grouped and analyzed according to different time periods, and the detection results corresponding to one or more image groups can be obtained.

[0045] Therefore, the present application can determine whether there is a traffic anomaly event for the vehicle to be detected through the detection results corresponding to multiple image groups, thereby improving the accuracy of traffic anomaly detection.

[0046] It can be seen that the present application obtains abnormality detection information and image quality information of each vehicle image by performing abnormality recognition processing and image quality analysis processing on each vehicle image to be detected. In the traditional method, it is difficult to make an accurate judgment when the image quality is poor or the detected target is small, which easily leads to false detection problems; while the present application determines the detection result of the vehicle to be detected based on the abnormality detection information and image quality information of each vehicle image of the vehicle to be detected, so that the detection result is more accurate and reliable. It is determined based on the detection result whether the vehicle to be detected has a traffic abnormality event, thereby improving the detection accuracy of traffic abnormalities.

[0047] Based on the above embodiments, the present embodiment of the application describes the steps of determining whether the vehicle to be detected has an abnormal traffic event according to the detection results. The detection results include normal detection results and abnormal detection results. Specifically, the method of this embodiment includes the following steps:

[0048] The traffic anomaly rate of the vehicle to be detected is determined according to the normal detection result and the abnormal detection result; in response to the traffic anomaly rate being greater than the target anomaly threshold, it is determined that the vehicle to be detected has a traffic anomaly event.

[0049] In conjunction with the above-mentioned embodiment, one or more detection results (including normal detection results and / or abnormal detection results) of the vehicle to be detected can be determined based on the abnormal detection information and image quality information of each vehicle image of the vehicle to be detected. These detection results can correspond to the same time period or to different time periods respectively, which is not limited here.

[0050] For example, in order to improve the accuracy of traffic anomaly detection, in this application, the traffic anomaly rate of the vehicle to be detected can be used to determine whether the vehicle to be detected has a traffic anomaly event. And it can be understood that if it is determined that there is a traffic anomaly event, human resources are usually allocated to the location where the traffic anomaly event occurred for detection. Therefore, in the specific application scenario of this application, human resources can also be deployed accordingly according to the traffic anomaly rate to improve the efficiency of traffic anomaly detection.

[0051] Specifically, the method for determining the traffic anomaly rate may include but is not limited to: determining the traffic anomaly rate of the vehicle to be detected based on the proportion of abnormal detection results in the detection results; or determining the traffic anomaly rate of the vehicle to be detected based on the proportion of normal detection results in the detection results.

[0052] This application mainly describes the method of determining the traffic anomaly rate by the proportion of the number of abnormal detection results to the total number of detection results. Therefore, if the traffic anomaly rate is greater than the preset target anomaly threshold, it can be determined that the vehicle to be detected has a traffic anomaly event. On the contrary, if the traffic anomaly rate of the vehicle to be detected is determined by the proportion of the number of normal detection results, the implementation process of the subsequent embodiments needs to be adjusted in the opposite direction accordingly. For example, if the traffic anomaly rate is less than the preset target anomaly threshold, it can be determined that the vehicle to be detected has a traffic anomaly event, and the specific details are not repeated.

[0053] Based on the above embodiment, the embodiment of the present application describes the steps before determining that the vehicle to be detected has a traffic abnormality event in response to the traffic abnormality rate being greater than the target abnormality threshold, and the steps of determining that the vehicle to be detected has a traffic abnormality event in response to the traffic abnormality rate being greater than the target abnormality threshold. Specifically, the method of this embodiment includes the following steps:

[0054] A target driving distance of the vehicle to be detected is determined according to the vehicle image of the vehicle to be detected; in response to the traffic anomaly rate being greater than a target anomaly threshold and the target driving distance being greater than a target distance threshold, it is determined that a traffic anomaly event exists for the vehicle to be detected.

[0055] In conjunction with the above-mentioned embodiments, in the specific application scenarios of traffic anomaly detection, there will not only be the problem of false detection, but also the problem of abnormal judgment standard. Specifically, in some special scenarios, although it is judged that the detected vehicle has abnormal detection results within a short time or short distance, according to traffic-related laws and regulations, its abnormal detection results will not necessarily be judged as abnormal events. For example, for helmet wearing abnormality, in a certain scenario, the driver only needs the vehicle to move a short distance (for example, adjust the parking direction of the vehicle or adjust the parking position of the vehicle, etc.), so he does not choose to wear a helmet. Or for license plate occlusion abnormality, in a certain scenario, the license plate is blocked by a floating plastic bag, etc., resulting in a temporary license plate occlusion abnormality. In traditional traffic anomaly detection methods, these special situations will be judged as traffic anomalies, but in fact they are not; in the traffic anomaly detection method of the present application, the traffic anomaly rate of the vehicle to be detected and the target driving distance are combined to determine whether there is a traffic anomaly event, which can effectively avoid the abnormal judgment problem caused by such special scenarios.

[0056] In short, the longer the driving distance of the vehicle to be detected, the more accurately it can characterize whether there is a traffic abnormality event for the vehicle to be detected, and it can also characterize the risk level of the vehicle to be detected during driving. For example, if the traffic abnormality rate of a certain vehicle to be detected is greater than the target abnormality threshold, and the target driving distance of the vehicle is less than or equal to the target distance threshold, it can be determined that there is no traffic abnormality event. If the traffic abnormality rate of a certain vehicle to be detected is greater than the target abnormality threshold, and the target driving distance of the vehicle is greater than the target distance threshold, since the longer the driving distance, the greater the traffic risk, it can be determined that there is a traffic abnormality event.

[0057] Based on the above embodiment, the embodiment of the present application describes the steps before determining that the detected vehicle has a traffic abnormality event in response to the traffic abnormality rate being greater than the target abnormality threshold and the target driving distance being greater than the target distance threshold. The target abnormality threshold and the target driving distance are negatively correlated. Specifically, the method of this embodiment includes the following steps:

[0058] Obtain a preset abnormality threshold; perform negative correlation adjustment processing on the preset abnormality threshold according to the target driving distance of the vehicle to be detected, and obtain a target abnormality threshold corresponding to the vehicle to be detected.

[0059] There may be one or more preset abnormality thresholds. Each vehicle to be detected may have its corresponding preset abnormality threshold, or each vehicle to be detected may share one preset abnormality threshold, which is not limited here.

[0060] In conjunction with the above-mentioned embodiments, it can be understood that in the specific application scenarios of traditional traffic anomaly detection, a small number of abnormalities may be detected in some vehicles, but because their traffic anomaly rate does not reach the preset anomaly threshold, they will not be determined as traffic anomaly events. In the traffic anomaly detection of the present application, the preset anomaly threshold corresponding to each vehicle to be detected can be associated with the target driving distance of the vehicle, so that the preset anomaly threshold and the target driving distance are negatively correlated and dynamically adjusted.

[0061] Exemplarily, for example, the preset abnormal threshold corresponding to the current vehicle 1 to be detected is 0.5, and the target driving distance of the vehicle 1 to be detected is analyzed. If the vehicle 1 to be detected travels a certain distance (for example, 0.5 kilometers or 1 kilometers, etc.), the current abnormal threshold (which can be a preset abnormal threshold or a target abnormal threshold) is reduced (for example, reduced by 0.1 or 0.05, etc.), and the adjusted abnormal threshold is obtained, that is, the target abnormal threshold corresponding to the vehicle 1 to be detected. When performing abnormal detection on the subsequent driving process of the vehicle 1 to be detected, the target abnormal threshold can be used for judgment. It should be noted that this embodiment can be iteratively performed according to the target driving distance of the vehicle to be detected. For example, if, based on the above example, the vehicle 1 to be detected travels another distance, the current target abnormal threshold is reduced until the target abnormal threshold of the vehicle 1 to be detected is reduced to 0 or the preset lower limit of the abnormal threshold, which will not be repeated here.

[0062] Based on the above embodiment, the embodiment of the present application describes the steps of determining the target driving distance of the vehicle to be detected based on the vehicle image of the vehicle to be detected. Specifically, the method of this embodiment includes the following steps:

[0063] The target points of the vehicle to be detected during its driving process are determined according to the vehicle image of the vehicle to be detected; and the target driving distance of the vehicle to be detected is determined according to the distance between each target point.

[0064] In conjunction with the foregoing embodiments, the target driving distance of the vehicle to be detected can be determined in the present application by determining the target point where the vehicle to be detected has traveled. The target point may refer to a point where the vehicle to be detected has passed, or may be a point where the vehicle to be detected has stopped, which is not limited here. In a specific application scenario, the traffic detection system will usually be connected to one or more image acquisition devices, and the location information of each image acquisition device may be pre-recorded or each image acquisition device may automatically report the location information, thereby obtaining the location information of each vehicle image.

[0065] Taking the point where the vehicle to be detected stops (which may at least include the starting point and the end point of the vehicle to be detected on a driving path) as the target point for illustration, when the vehicle to be detected stops, its driving speed should be less than a preset speed threshold.

[0066] Therefore, the vehicle images in the vehicle image set of each vehicle to be detected can be traversed separately. For example, after the aggregation process of the above embodiment, there is a vehicle set to be detected G={g(1), g(2), ..., g(n)}, where g(1) refers to the vehicle image set of vehicle 1 to be detected, g(2) refers to the vehicle image set of vehicle 2 to be detected, ..., g(n) refers to the vehicle image set of vehicle n to be detected. Taking vehicle 1 to be detected as an example, the vehicle images in the vehicle image set g(1) are {p(1), p(2), …, p(N)}, the position information of each vehicle image is recorded as {L(1), L(2), …, L(N)}, and the time information when each vehicle image is collected is recorded as {T(1), T(2), …, T(N)}. Therefore, the speed speed between each two frames of vehicle images in the vehicle image set g(1) can be calculated based on the position information and time information (it can be two consecutive frames such as p(i) and p(i+1), or it can be two discontinuous frames such as p(i) and p(i+2), etc., which is not limited here). If the speed speed is less than the preset speed threshold speed_thres1, the target point can be determined based on the position information of the next frame of vehicle image (such as p(i+1) frame), which is equivalent to taking L(i+1) as the target point.

[0067] By using the above method, the target point of the vehicle 1 to be detected can be determined as {L(x), L(y), ...} etc. according to the vehicle images in the vehicle image set g(1). Then, the distance between two adjacent target points can be calculated to obtain the target driving distance of the vehicle 1 to be detected.

[0068] It should also be noted that the present application can also group the vehicle images in each vehicle image set according to the time information of the vehicle images in the vehicle image set, so as to avoid combining vehicle images with too large a time span for traffic anomaly detection, thereby affecting the traffic anomaly detection effect.

[0069] Exemplarily, the location information {L(1), L(2), ..., L(N)} of the vehicle images is grouped according to a preset time threshold. Specifically, the time interval between two adjacent frames of images that is greater than the preset time threshold may be used as a basis for grouping, or grouping may be performed according to a group per preset time threshold (e.g., a group per day), so that vehicle images across days are divided into two different groups.

[0070] For example, {L(1), L(2), …, L(N)} are grouped and processed as one group per day to obtain {[L(1), …, L(k)], [L(k+1), …, L(j)], …, [L(m), …, L(N)]}, where the time corresponding to k and the time corresponding to k+1 belong to different dates, and the time corresponding to j also belongs to different dates. For example, [L(1), …, L(k)] can be the vehicle image of the vehicle 1 to be detected on the first day, [L(k+1), …, L(j)] can be the vehicle image of the vehicle 1 to be detected on the second day, and [L(m), …, L(N)] can be the vehicle image of the vehicle 1 to be detected on the third day. The distance between each pair of points in each group can be calculated according to the order of each group, and the sum of the distances between the points in each group is the target driving distance of the group. For example, if [L(1),…,L(k)] is a group, the mathematical expression of the target driving distance of the group can be: dist(L(1),L(2))+dist(L(2),L(3))+…+dist(L(k-1),L(k)). Where dist() refers to the distance between two target points.

[0071] Furthermore, if a vehicle image set has multiple groups, statistical values ​​can be calculated based on the target driving distances corresponding to the multiple groups respectively (the statistical values ​​may include but are not limited to the average value, weighted average value, median, mode, etc., which are not limited here), and the final statistical value distance is used as the target driving distance of the vehicle image set, that is, the target driving distance of the vehicle to be detected.

[0072] It should also be noted that in the aforementioned embodiment, a method for determining the traffic anomaly rate is provided, which can be based on the proportion of the number of abnormal detection results in the detection results to determine the traffic anomaly rate of the vehicle to be detected. Combining it with the present embodiment, it can be known that the detection result is equivalent to the detection result obtained by performing traffic anomaly detection on each group of vehicle images in the vehicle image set of the vehicle to be detected, that is, each vehicle image group should have a corresponding detection result. Continuing to take the vehicle to be detected 1 as an example for explanation, after performing traffic anomaly detection on each group of images in its vehicle image set g(1), the detection result of the vehicle image set g(1) (including normal detection results and abnormal detection results) can be obtained. Then, the traffic anomaly rate of the vehicle to be detected is determined based on the proportion of the number of abnormal detection results in the detection results. Among them, the detection result can be recorded as r, r=0 represents it as a normal detection result, and r=1 represents it as an abnormal detection result. The detection result obtained according to g(1) is {r1(1),…r1(n)}, so the mathematical expression of the traffic anomaly rate illegal_rate of the vehicle to be detected 1 can be: illegal_rate=(the number of r(i)=1) / n.

[0073] Based on the above embodiment, the embodiment of the present application describes the steps after determining the target point of the vehicle to be detected during driving according to the vehicle image of the vehicle to be detected. Specifically, the method of this embodiment includes the following steps:

[0074] According to the normal detection results and abnormal detection results of each vehicle to be detected at each target point, the point abnormality rate of each target point is determined; and the abnormal point is determined according to the point abnormality rate of each target point.

[0075] In conjunction with the above-mentioned embodiments, in the actual process of traffic anomaly detection, more human resources should be allocated for management of road sections with frequent traffic anomalies. Therefore, in the traffic anomaly detection method of the present application, the point anomaly rate of each target point can be determined by the traffic anomaly detection system, and human resources can be subsequently deployed according to the point anomaly rate, thereby improving the detection and management efficiency of traffic anomalies.

[0076] Exemplarily, the set of vehicles to be detected G={g(1), g(2),…, g(n)} in the aforementioned embodiment is still used as an example for explanation. In this embodiment, an index table D (e.g., a key-value pair index table) can be constructed, and the set of vehicles to be detected G can be traversed, and the target point of each vehicle to be detected is obtained by the method provided in the aforementioned embodiment, and is used as the key value (index value) of the index table. For example, the target points of the vehicle 1 to be detected are A and B, and the target points of the vehicle 2 to be detected are A, B, and C. Therefore, the target points in the index table obtained after traversal should be A, B, and C.

[0077] Furthermore, for G={g(1), g(2),…}, it includes one or more vehicle image sets g. According to the time grouping method provided in the above embodiment, each vehicle image set g can be grouped and processed separately. For example, the time groups in g(1) (a total of N vehicle images) are {[p(1),…,p(k)], [p(k+1),…,p(j)],…, [p(m),…,p(N)]}, assuming that it is divided into n groups, and the detection results of each group (including normal detection results and abnormal detection results) are {r(1),…r(n)}. If the i-th group of images contains the target point local_1, the mapping value (value) of the target point local_1 in the index table D is updated. If r(i)=0, the number of normal detection results of the local_1 point is increased by 1; if r(i)=1, the number of abnormal detection results of the local_1 point is increased by 1.

[0078] For each target point, there may be one or more vehicles to be detected staying at the target point, that is, each target point may correspond to one or more detection results of one or more vehicles to be detected. Therefore, the point abnormality rate of each target point can be determined by one or more detection results corresponding to each target point. The specific process can refer to the method for calculating the traffic abnormality rate in the aforementioned embodiment, which will not be repeated here. For example, the mathematical expression for calculating the point abnormality rate can be: point abnormality rate = number of abnormal detection results / (number of abnormal detection results + number of normal detection results). That is, the point abnormality rate of the target point is equal to the ratio of the number of abnormal detection results of the point to the total number of detection results of the point.

[0079] If the point abnormality rate of a target point is greater than the preset point abnormality threshold, the target point can be determined as an abnormal point. If the point abnormality rate of a target point is less than or equal to the preset point abnormality threshold, the target point can be determined as a normal point. For multiple target points, abnormal points are more likely to have traffic accidents than normal points, so more human resources can be allocated to abnormal points for attention and detection than normal points.

[0080] Based on the above embodiments, the embodiment of the present application exemplarily illustrates the steps of performing abnormality recognition processing and image quality analysis processing on each vehicle image to be detected, and obtaining abnormality detection information and image quality information of each vehicle image. Specifically, the method of this embodiment includes the following steps:

[0081] Each vehicle image of the vehicle to be detected is input into a pre-trained traffic anomaly recognition model to obtain anomaly detection information corresponding to each vehicle image output by the traffic anomaly recognition model; each vehicle image of the vehicle to be detected is input into a pre-trained image quality assessment model to obtain image quality information corresponding to each vehicle image output by the image quality assessment model.

[0082] In conjunction with the above-mentioned embodiments, the traffic anomaly identification and processing process of the present application can be performed based on a pre-trained traffic anomaly identification model. For example, each vehicle image set is traversed, and its vehicle image is input into a pre-trained traffic anomaly identification model to obtain the anomaly detection information corresponding to each vehicle image output by the traffic anomaly identification model. Among them, the anomaly detection information may include the anomaly type (such as abnormal helmet wearing, abnormal number of passengers, etc.) and the anomaly state (whether it is abnormal, 1 for normal and 0 for abnormal).

[0083] The image quality analysis and processing process of the present application can be based on a pre-trained image quality assessment model. Among them, the image quality assessment model can adopt a general CNN model architecture. During model training, the training data can have a pre-labeled image quality score. Exemplarily, the interval range of the image quality score can be 0~10 points. During training, 0~10 is normalized to 0~1, and the MSE loss function is used for iterative optimization training until the training is completed to obtain the image quality assessment model. When the image quality assessment model is used, a picture can be input, and the image quality assessment model outputs its image quality score q. For example, traverse each vehicle image set, and input its vehicle image into the pre-trained image quality assessment model to obtain the image quality information corresponding to each vehicle image output by the image quality assessment model (which can at least include the image quality score).

[0084] Based on the above embodiment, the embodiment of the present application describes the steps of determining the detection result of the vehicle to be detected based on the abnormality detection information and image quality information of each vehicle image of the vehicle to be detected. Specifically, the method of this embodiment includes the following steps:

[0085] Based on the abnormal detection information and image quality information of each vehicle image, weighted processing is performed to obtain a weighted result of each vehicle image; the weighted results of each vehicle image of the vehicle to be detected are summed to obtain a detection result of the vehicle to be detected.

[0086] With reference to the foregoing embodiments, the detection result of the present application may be determined based on both the anomaly detection information and the image quality information, so as to prevent low-quality images from reducing the accuracy of anomaly detection.

[0087] Exemplarily, the image grouping method of the aforementioned embodiment can still be referred to, and images of different time periods in the vehicle image set can be grouped into one group. Taking the vehicle image set g(1) as an example, it includes N vehicle images {p(1), p(2), ..., p(N)}, and the images are sorted and grouped according to the time information of each image, and multiple grouped image groups {[p(1), ..., p(k)], [p(k+1), ..., p(j)], ..., [p(m), ..., p(N)]} can be obtained. Then, the voting method can be used to vote on the abnormalities of each group of images. Taking [p(1),…,p(k)] as an example, if the recognition result of the abnormal helmet wearing of each image in the group is [J(1),…,J(k)] (for example, if the helmet is worn normally, J=0, and if the helmet is worn abnormally, J=1), the quality score of each image is [q(1),…,q(k)]. The quality scores are normalized to ensure that the sum of their quality scores is 1. The mathematical expression of the detection result of the image group is: result=(J(1)*q(1)+J(2)*q(2)+…+J(k)*q(k)). This is equivalent to using the quality score of each image as a weight, performing weighted summation processing on the abnormal information of each image, and obtaining the final detection result of the image group.

[0088] Furthermore, the value of the test result can be compared with a preset result abnormality threshold to distinguish whether the test result is a normal test result or an abnormal test result. For example, the result abnormality threshold can be 0.5. If result>0.5, an abnormal test result is obtained, otherwise it is a normal test result.

[0089] It should also be noted that through the above voting method, one or more groups of detection results for each vehicle to be detected can be obtained, which can be recorded as {R(1), R(2), ..., R(N)}. For example, the detection result in the vehicle image set g(i) of the vehicle i to be detected is R(i). Among them, R(i) can be divided into multiple detection results according to the abnormality type, such as helmet wearing abnormality {r1(1), ...r1(n)}, passenger number abnormality {r2(1), ...r2(n)}, awning installation abnormality {r3(1), ...r3(n)}, etc., and n represents the number of the above groups. r(i) = 0 or 1, 0 represents a normal detection result, and 1 represents an abnormal detection result.

[0090] It should be further explained that the execution subject of the traffic anomaly detection method may be a traffic anomaly detection device, for example, the traffic anomaly detection method may be executed by a terminal device or a server or other processing device, wherein the terminal device may be a user equipment (UE), a computer, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the traffic anomaly detection method may be implemented by a processor calling a computer-readable instruction stored in a memory.

[0091] Figure 2 FIG. 1 is a block diagram of a traffic anomaly detection device shown in an exemplary embodiment of the present application. Figure 2 As shown, the exemplary traffic anomaly detection device 200 includes: an image processing module 210, a detection module 220 and an anomaly determination module 230. Specifically:

[0092] The image processing module 210 is used to perform abnormality recognition processing and image quality analysis processing on each vehicle image of the vehicle to be detected, so as to obtain abnormality detection information and image quality information of each vehicle image.

[0093] The detection module 220 is used to determine the detection result of the vehicle to be detected based on the abnormality detection information and image quality information of each vehicle image of the vehicle to be detected.

[0094] The abnormality determination module 230 is used to determine whether there is any abnormal traffic event in the vehicle to be detected according to the detection result.

[0095] In this exemplary traffic anomaly detection device, anomaly recognition processing and image quality analysis processing are performed on each vehicle image of the vehicle to be detected, thereby obtaining anomaly detection information and image quality information of each vehicle image. In the traditional method, it is difficult to make an accurate judgment when the image quality is poor or the detected target is small, which easily leads to false detection problems; however, the present application determines the detection result of the vehicle to be detected based on the anomaly detection information and image quality information of each vehicle image of the vehicle to be detected, so that the detection result is more accurate and reliable. It is determined whether the vehicle to be detected has a traffic anomaly event based on the detection result, thereby improving the detection accuracy of traffic anomalies.

[0096] It should be noted that the device provided in the above embodiment and the method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In practical applications, the device provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0097] The functions of each module can be found in the embodiment of the traffic anomaly detection method, which will not be described in detail here.

[0098] See also Figure 3 , Figure 3 1 is a schematic diagram of the structure of an embodiment of an electronic device of the present application. The electronic device 100 includes a memory 101 and a processor 102, and the processor 102 is used to execute program instructions stored in the memory 101 to implement the steps in any of the above-mentioned traffic anomaly detection method embodiments. In a specific implementation scenario, the electronic device 100 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 100 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0099] Specifically, the processor 102 is used to control itself and the memory 101 to implement the steps in any of the above-mentioned traffic anomaly detection method embodiments. The processor 102 can also be called a CPU (Central Processing Unit). The processor 102 may be an integrated circuit chip with signal processing capabilities. The processor 102 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 102 can be implemented by an integrated circuit chip.

[0100] In this exemplary electronic device, abnormality detection information and image quality information of each vehicle image to be detected are obtained by performing abnormality recognition processing and image quality analysis processing on each vehicle image to be detected. In the traditional method, it is difficult to make an accurate judgment when the image quality is poor or the detected target is small, which easily leads to false detection problems; however, the present application determines the detection result of the vehicle to be detected based on the abnormality detection information and image quality information of each vehicle image of the vehicle to be detected, so that the detection result is more accurate and reliable. It is determined whether there is a traffic abnormality event for the vehicle to be detected based on the detection result, thereby improving the detection accuracy of traffic abnormality events.

[0101] See also Figure 4 , Figure 4 The computer-readable storage medium 110 stores program instructions 111 that can be executed by a processor, and the program instructions 111 are used to implement the steps in any of the above-mentioned traffic anomaly detection method embodiments.

[0102] In this exemplary storage medium, each vehicle image of the vehicle to be detected is subjected to abnormality recognition processing and image quality analysis processing by running the program instructions in the storage medium, thereby obtaining abnormality detection information and image quality information of each vehicle image. In the traditional method, it is difficult to make an accurate judgment when the image quality is poor or the detected target is small, which easily leads to the problem of false detection; however, the present application determines the detection result of the vehicle to be detected based on the abnormality detection information and image quality information of each vehicle image of the vehicle to be detected, so that the detection result is more accurate and reliable. It is determined whether there is a traffic abnormality event for the vehicle to be detected based on the detection result, thereby improving the detection accuracy of traffic abnormality events.

[0103] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0104] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0105] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0106] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. A traffic anomaly detection method, characterized in that: The method comprises: Performing abnormality recognition processing and image quality analysis processing on each vehicle image to be detected, and obtaining abnormality detection information and image quality information of each vehicle image; Determining a detection result of the vehicle to be detected according to the abnormality detection information and the image quality information of each vehicle image of the vehicle to be detected; Determine whether the vehicle to be detected has an abnormal traffic event according to the detection result; the detection result includes a normal detection result and an abnormal detection result; The step of determining whether the vehicle to be detected has a traffic abnormality event according to the detection result comprises: determining the traffic abnormality rate of the vehicle to be detected according to the normal detection result and the abnormal detection result; in response to the traffic abnormality rate being greater than a target abnormality threshold, determining that the vehicle to be detected has a traffic abnormality event; wherein the target abnormality threshold is negatively correlated with the target driving distance of the vehicle to be detected; and the target driving distance is determined by the vehicle image of the vehicle to be detected; Determine the target point of the vehicle to be detected during the driving process according to the vehicle image of the vehicle to be detected; determine the point abnormality rate of each target point according to the normal detection results and abnormal detection results of each vehicle to be detected at each target point; determine the abnormal point according to the point abnormality rate of each target point.

2. The method according to claim 1, characterized in that: Before the step of determining that a traffic abnormality event exists for the vehicle to be detected in response to the traffic abnormality rate being greater than a target abnormality threshold, the method further includes: Determining a target driving distance of the vehicle to be detected based on the vehicle image of the vehicle to be detected; The step of determining that a traffic abnormality event exists for the vehicle to be detected in response to the traffic abnormality rate being greater than a target abnormality threshold comprises: In response to the traffic anomaly rate being greater than the target anomaly threshold and the target driving distance being greater than a target distance threshold, it is determined that a traffic anomaly event occurs for the vehicle to be detected.

3. The method according to claim 2, characterized in that The target abnormality threshold and the target driving distance are negatively correlated. Before the step of determining that a traffic abnormality event exists for the vehicle to be detected in response to the traffic abnormality rate being greater than the target abnormality threshold and the target driving distance being greater than the target distance threshold, the method further includes: Get the preset abnormal threshold; The preset abnormality threshold is negatively adjusted according to the target driving distance of the vehicle to be detected to obtain the target abnormality threshold corresponding to the vehicle to be detected.

4. The method according to claim 2, characterized in that: The step of determining the target driving distance of the vehicle to be detected based on the vehicle image of the vehicle to be detected comprises: Determining a target point position of the vehicle to be detected during its travel according to the vehicle image of the vehicle to be detected; The target driving distance of the vehicle to be detected is determined according to the distance between each target point.

5. The method according to claim 1, characterized in that The step of performing abnormality recognition processing and image quality analysis processing on each vehicle image to be detected to obtain abnormality detection information and image quality information of each vehicle image comprises: Inputting each vehicle image of the vehicle to be detected into a pre-trained traffic anomaly recognition model to obtain anomaly detection information corresponding to each vehicle image output by the traffic anomaly recognition model; Each vehicle image of the vehicle to be detected is input into a pre-trained image quality assessment model to obtain image quality information corresponding to each vehicle image output by the image quality assessment model.

6. The method according to claim 1, characterized in that The step of determining the detection result of the vehicle to be detected based on the abnormality detection information and the image quality information of each vehicle image of the vehicle to be detected comprises: Performing weighted processing based on abnormality detection information and image quality information of each vehicle image to obtain a weighted result of each vehicle image; The weighted results of the vehicle images of the vehicle to be detected are summed to obtain the detection result of the vehicle to be detected.

7. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images