Manpower tricycle detection method based on tricycle traffic illegal behavior video

Abstract

The invention relates to the technical field of image recognition, and particularly discloses a manpower tricycle detection method based on a tricycle traffic illegal behavior video, and the method comprises the steps: obtaining a tricycle traffic illegal behavior recognition result; according to the tricycle traffic illegal behavior recognition result, tricycle traffic illegal behavior videos within a period of time before and after the traffic illegal behavior of the tricycle are extracted; judging whether a tricycle in the tricycle traffic illegal behavior video is a manpower tricycle or not; if a tricycle in the tricycle traffic illegal behavior video is a manpower tricycle, the tricycle traffic illegal behavior identification result is wrong; if the tricycle in the tricycle traffic illegal behavior video is a motor tricycle, the tricycle traffic illegal behavior recognition result is correct. According to the manpower tricycle detection method based on the tricycle traffic illegal behavior video provided by the invention, the accuracy of recognizing the traffic illegal behaviors of the motor tricycle can be improved, and the workload of manual auditing is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and more specifically, to a method for detecting a human-powered tricycle based on a tricycle traffic violation video. Background Art

[0002] As the name implies, a tricycle is a vehicle with three wheels. According to the power source, it is divided into three-wheeled motorcycles (including electric tricycles) and human-powered tricycles. Human-powered tricycles are non-motorized vehicles, while three-wheeled motorcycles are motor vehicles. The Road Traffic Law applies completely different traffic rules to motor vehicles and non-motorized vehicles. The two are obviously different in terms of driving regulations, license plates, safety equipment, etc. For example, a three-wheeled motorcycle must wear a safety helmet when driving, but there is no such requirement for human-powered tricycles.

[0003] Although most three-wheeled motorcycles and human-powered tricycles have significant differences in appearance, there are still some three-wheeled motorcycles, especially some electric and power-assisted tricycles and human-powered tricycles, which have similar appearances from certain angles, resulting in false alarms in video-based traffic violation recognition algorithms, requiring a large amount of manual review, and causing a huge waste of manpower and material resources.

[0004] Therefore, there is an urgent need for a rapid identification method to distinguish human-powered tricycles while identifying traffic violations. Summary of the invention

[0005] In order to solve the deficiencies in the prior art, the present invention provides a human-powered tricycle detection method based on tricycle traffic violation videos, which can improve the accuracy of three-wheeled motorcycle traffic violation identification and reduce the workload of manual review.

[0006] As a first aspect of the present invention, a method for detecting a human-powered tricycle based on a tricycle traffic violation video is provided, comprising the following steps: Step S1: Obtaining the tricycle traffic violation identification result; Step S2: extracting the traffic violation video of the tricycle within a period of time before and after the tricycle traffic violation occurs according to the tricycle traffic violation identification result; Step S3: determining whether the tricycle in the tricycle traffic violation video is a human-powered tricycle; Step S4: If the tricycle in the tricycle traffic violation video is a human-powered tricycle, it means that the tricycle traffic violation identification result is wrong; if the tricycle in the tricycle traffic violation video is a three-wheeled motorcycle, it means that the tricycle traffic violation identification result is correct.

[0007] Furthermore, the obtaining of the tricycle traffic violation identification result also includes: The traffic violations of tricycles are preliminarily identified from the road monitoring videos collected by the road monitoring equipment to obtain the identification results of the traffic violations of tricycles.

[0008] Furthermore, the step of determining whether the tricycle in the tricycle traffic violation video is a human-powered tricycle further includes: Calculate the ratio of the tricycle driver area to the tricycle area in the tricycle traffic violation video, and determine whether the ratio is less than a lower limit value of the ratio; when the ratio is less than the lower limit value of the ratio, determine that the tricycle in the tricycle traffic violation video is a three-wheeled motorcycle; otherwise, calculate the difference between the average driving speed of the tricycle and the average driving speed of other vehicles in the tricycle traffic violation video; Determine whether the difference between the average speed of the tricycle and the average speed of the motor vehicle is greater than a first difference threshold; if it is greater than the first difference threshold, determine whether the difference between the average speed of the tricycle and the average speed of the non-motor vehicle is less than a second difference threshold; if it is not greater than the first difference threshold, determine that the tricycle in the tricycle traffic violation video is a three-wheeled motorcycle; If it is less than the second difference threshold, it is determined whether the tricycle driver in the tricycle traffic violation video is a rider; if it is not less than the second difference threshold, it is determined that the tricycle in the tricycle traffic violation video is a three-wheeled motorcycle; If the tricycle driver in the tricycle traffic violation video is a riding person, then it is judged that the tricycle in the tricycle traffic violation video is a human-powered tricycle; if the tricycle driver in the tricycle traffic violation video is a sitting person, then it is judged that the tricycle in the tricycle traffic violation video is a three-wheeled motorcycle.

[0009] Furthermore, the calculating of the ratio of the tricycle driver area to the tricycle area in the tricycle traffic violation video further includes: The deep convolutional neural network DeepLabv3 and Mask R-CNN are used to perform image semantic segmentation on the tricycle traffic violation video, extract high-level features through the convolution layer, and train on the labeled data set to identify the tricycle area and tricycle driver area in the tricycle traffic violation video.

[0010] Furthermore, the calculation of the difference between the average speed of the tricycle and the average speed of other vehicles in the tricycle traffic violation video also includes: A target detection algorithm is used to identify other vehicles in the tricycle traffic violation video, and the difference between the average driving speed of the tricycle in the tricycle traffic violation video and the average driving speed of other vehicles is calculated.

[0011] Furthermore, the determining whether the tricycle driver in the tricycle traffic violation video is a rider also includes: Tracking the tricycle driver in the tricycle traffic violation video using a multi-target tracking algorithm; A deep learning video recognition model is used to identify the types of tricycle drivers in the tricycle traffic violation video, wherein the types of tricycle drivers are divided into riding persons and sitting persons.

[0012] The human-powered tricycle detection method based on tricycle traffic violation videos provided by the present invention has the following advantages: human-powered tricycles can be identified from tricycle traffic violation videos, thereby avoiding false alarms of traffic violations, improving the accuracy of three-wheeled motorcycle traffic violation identification, and reducing the workload of manual review. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present invention, but do not constitute a limitation of the present invention.

[0014] Figure 1 The present invention provides a flow chart of a method for detecting a human-powered tricycle based on a tricycle traffic violation video.

[0015] Figure 2 A flowchart of a specific implementation method of a human-powered tricycle detection method based on tricycle traffic violation videos provided by the present invention. DETAILED DESCRIPTION

[0016] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0017] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation method, structure, characteristics and effects of a human-powered tricycle detection method based on a tricycle traffic violation video proposed by the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0018] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so as to describe the embodiments of the present invention described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0019] In this embodiment, a human-powered tricycle detection method based on tricycle traffic violation video is provided. Figure 1 As shown, the human-powered tricycle detection method based on tricycle traffic violation video includes: Step S1: Obtaining the tricycle traffic violation identification result; Preferably, the obtaining of the tricycle traffic violation identification result further includes: The traffic violations of tricycles are preliminarily identified from the road monitoring videos collected by the road monitoring equipment to obtain the identification results of the traffic violations of tricycles.

[0020] It should be noted that the present invention is used for correcting the tricycle traffic violation identification results, and for the tricycle traffic violation identification results, removes the situation in which the tricycle is a human-powered tricycle.

[0021] Step S2: extracting the traffic violation video of the tricycle within a period of time before and after the tricycle traffic violation occurs according to the tricycle traffic violation identification result; It should be noted that as a post-recognition correction algorithm, after identifying the illegal behavior related to tricycles, the previous and next videos are read for identification to minimize the consumption of computing resources and avoid affecting the operation of the main program of the recognition algorithm.

[0022] Step S3: determining whether the tricycle in the tricycle traffic violation video is a human-powered tricycle; Preferably, Figure 2 As shown, the process of determining whether the tricycle in the tricycle traffic violation video is a human-powered tricycle also includes: Calculate the ratio of the tricycle driver area to the tricycle area in the tricycle traffic violation video, and determine whether the ratio is less than a lower limit value of the ratio; when the ratio is less than the lower limit value of the ratio, determine that the tricycle in the tricycle traffic violation video is a three-wheeled motorcycle; otherwise, calculate the difference between the average driving speed of the tricycle and the average driving speed of other vehicles in the tricycle traffic violation video; Specifically, the calculating of the ratio of the tricycle driver area to the tricycle area in the tricycle traffic violation video also includes: The deep convolutional neural network DeepLabv3 and Mask R-CNN are used to perform image semantic segmentation on the tricycle traffic violation video, extract high-level features through the convolution layer, and train on the labeled data set to identify the tricycle area and tricycle driver area in the tricycle traffic violation video.

[0023] It should be noted that human-powered tricycles are relatively small in size, and tricycles that are too large can basically be excluded as human-powered tricycles. However, due to the lack of calibration in the video image, the pixel size of the tricycle cannot correspond to the actual vehicle size. The size difference of the drivers is limited. If the driver is used as a reference system, a size range of tricycles can be obtained, thereby effectively excluding some tricycles that are too large.

[0024] It should be noted that, when collecting samples of human-powered tricycles, the same method is used to obtain the pixel ratio range of the human-powered tricycle driver area and the tricycle area. If the ratio is below the lower limit, the tricycle is determined to be a three-wheeled motorcycle.

[0025] Specifically, the calculating of the difference between the average speed of the tricycle and the average speed of other vehicles in the tricycle traffic violation video also includes: A target detection algorithm is used to identify other vehicles in the tricycle traffic violation video, and the difference between the average driving speed of the tricycle in the tricycle traffic violation video and the average driving speed of other vehicles is calculated.

[0026] Determine whether the difference between the average speed of the tricycle and the average speed of the motor vehicle is greater than a first difference threshold; if it is greater than the first difference threshold, determine whether the difference between the average speed of the tricycle and the average speed of the non-motor vehicle is less than a second difference threshold; if it is not greater than the first difference threshold, determine that the tricycle in the tricycle traffic violation video is a three-wheeled motorcycle; If it is less than the second difference threshold, it is determined whether the tricycle driver in the tricycle traffic violation video is a rider; if it is not less than the second difference threshold, it is determined that the tricycle in the tricycle traffic violation video is a three-wheeled motorcycle; It should be noted that the speed of a human-powered tricycle is usually much lower than that of a motor vehicle. Although the absolute speed of the vehicle in the video cannot be calculated due to lack of calibration, the relative speed of the vehicles in the same video can be obtained. If the speed of the tricycle is close to the speed of the motor vehicle identified in the video, it means that the tricycle is likely to be a three-wheeled motorcycle rather than a human-powered tricycle.

[0027] Specifically, the vehicles in the same video are in the same reference frame, so the relative speeds are comparable. The object detection algorithms YOLO and Faster R-CNN are used to identify other vehicles in the video, and their relative speeds with the tricycle can be calculated.

[0028] In the embodiment of the present invention, if the speed is less than the average speed of motor vehicles and close to the average speed of non-motor vehicles such as bicycles, it can be judged as a human-powered tricycle. If the speed is greater than the average speed of non-motor vehicles and close to the average speed of motor vehicles, it can be judged as a three-wheeled motorcycle.

[0029] Specifically, the determining whether the tricycle driver in the tricycle traffic violation video is a rider also includes: Using the multi-target tracking algorithm Deep SORT, the tricycle driver in the tricycle traffic violation video is tracked; A deep learning video recognition model is used to identify the types of tricycle drivers in the tricycle traffic violation video, wherein the types of tricycle drivers are divided into riding persons and sitting persons.

[0030] If the tricycle driver in the tricycle traffic violation video is a riding person, then it is judged that the tricycle in the tricycle traffic violation video is a human-powered tricycle; if the tricycle driver in the tricycle traffic violation video is a sitting person, then it is judged that the tricycle in the tricycle traffic violation video is a three-wheeled motorcycle.

[0031] It should be noted that in many video surveillance angles (such as the rear of the vehicle), the power part or pedal part of the tricycle cannot be seen, so it is difficult to distinguish between some human-powered tricycles and three-wheeled motorcycles that have similar appearances. However, human-powered tricycles need to be pedaled, and even if filmed from the rear, the back of the rider will have a clear continuous change process, thus forming a clear distinction from the driver of the three-wheeled motorcycle.

[0032] In an embodiment of the present invention, a tricycle video sample is collected, a rider is identified and tracked, a continuous video of the rider is intercepted as a sample, and a 3D CNN deep learning video recognition model of the rider is trained.

[0033] In an embodiment of the present invention, when identifying traffic violations, the same method is used to capture a video of a tricycle rider, and the trained video recognition model is used to distinguish between the rider and the seated person, thereby determining whether the traffic violation video is a human-powered tricycle.

[0034] Step S4: If the tricycle in the tricycle traffic violation video is a human-powered tricycle, it means that the tricycle traffic violation identification result is wrong; if the tricycle in the tricycle traffic violation video is a three-wheeled motorcycle, it means that the tricycle traffic violation identification result is correct.

[0035] The present invention provides a human-powered tricycle detection method based on a tricycle traffic violation video, which (1) excludes non-human-powered tricycles by the relative ratio of the driver to the tricycle; (2) excludes non-human-powered tricycles by the relative speed of the tricycle and other vehicles; (3) intercepts the tricycle driver video, distinguishes the rider from the sitting person by the 3D video model of the tricycle rider, and converts the human-powered tricycle and three-wheeled motorcycle distinction algorithm into the rider and three-wheeled motorcycle distinction algorithm; the human-powered tricycle can be identified from the tricycle traffic violation video, thereby avoiding false alarms of traffic violations, improving the accuracy of tricycle traffic violation identification, and reducing the workload of manual review.

[0036] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of the technical solution of the present invention.

Claims

1. A human-powered tricycle detection method based on tricycle traffic violation video, characterized in that: The human-powered tricycle detection method based on tricycle traffic violation video comprises the following steps: Step S1: Obtaining the tricycle traffic violation identification result; Step S2: extracting the traffic violation video of the tricycle within a period of time before and after the tricycle traffic violation occurs according to the tricycle traffic violation identification result; Step S3: determining whether the tricycle in the tricycle traffic violation video is a human-powered tricycle; Step S4: If the tricycle in the tricycle traffic violation video is a human-powered tricycle, it means that the tricycle traffic violation identification result is wrong; if the tricycle in the tricycle traffic violation video is a three-wheeled motorcycle, it means that the tricycle traffic violation identification result is correct.

2. The method for detecting human-powered tricycles based on tricycle traffic violation videos according to claim 1 is characterized in that: The obtaining of the tricycle traffic violation identification result also includes: The traffic violations of tricycles are preliminarily identified from the road monitoring videos collected by the road monitoring equipment to obtain the identification results of the traffic violations of tricycles.

3. The method for detecting human-powered tricycles based on tricycle traffic violation videos according to claim 1, characterized in that: The step of determining whether the tricycle in the tricycle traffic violation video is a human-powered tricycle also includes: Calculate the ratio of the tricycle driver area to the tricycle area in the tricycle traffic violation video, and determine whether the ratio is less than a lower limit value of the ratio; when the ratio is less than the lower limit value of the ratio, determine that the tricycle in the tricycle traffic violation video is a three-wheeled motorcycle; otherwise, calculate the difference between the average driving speed of the tricycle and the average driving speed of other vehicles in the tricycle traffic violation video; Determine whether the difference between the average speed of the tricycle and the average speed of the motor vehicle is greater than a first difference threshold; if it is greater than the first difference threshold, determine whether the difference between the average speed of the tricycle and the average speed of the non-motor vehicle is less than a second difference threshold; if it is not greater than the first difference threshold, determine that the tricycle in the tricycle traffic violation video is a three-wheeled motorcycle; If it is less than the second difference threshold, it is determined whether the tricycle driver in the tricycle traffic violation video is a rider; if it is not less than the second difference threshold, it is determined that the tricycle in the tricycle traffic violation video is a three-wheeled motorcycle; If the tricycle driver in the tricycle traffic violation video is a riding person, then it is judged that the tricycle in the tricycle traffic violation video is a human-powered tricycle; if the tricycle driver in the tricycle traffic violation video is a sitting person, then it is judged that the tricycle in the tricycle traffic violation video is a three-wheeled motorcycle.

4. The method for detecting human-powered tricycles based on tricycle traffic violation videos according to claim 3 is characterized in that: The calculating of the ratio of the tricycle driver area to the tricycle area in the tricycle traffic violation video also includes: The deep convolutional neural network DeepLabv3 and Mask R-CNN are used to perform image semantic segmentation on the tricycle traffic violation video, extract high-level features through the convolution layer, and train on the labeled data set to identify the tricycle area and tricycle driver area in the tricycle traffic violation video.

5. The method for detecting human-powered tricycles based on tricycle traffic violation videos according to claim 3 is characterized in that: The calculation of the difference between the average speed of the tricycle and the average speed of other vehicles in the tricycle traffic violation video also includes: A target detection algorithm is used to identify other vehicles in the tricycle traffic violation video, and the difference between the average driving speed of the tricycle in the tricycle traffic violation video and the average driving speed of other vehicles is calculated.

6. The method for detecting human-powered tricycles based on tricycle traffic violation videos according to claim 3 is characterized in that: The step of determining whether the tricycle driver in the tricycle traffic violation video is a rider also includes: Tracking the tricycle driver in the tricycle traffic violation video using a multi-target tracking algorithm; A deep learning video recognition model is used to identify the types of tricycle drivers in the tricycle traffic violation video, wherein the types of tricycle drivers are divided into riding persons and sitting persons.

Images