Non-motor vehicle total factor management system and method

Abstract

The invention provides a non-motor vehicle total factor management system and method, and the method comprises the steps: collecting first video data of a non-motor vehicle lane at an intersection in the reverse direction of a driving direction, and collecting second video data of the non-motor vehicle lane at the intersection in the same direction of the driving direction; judging whether an illegal behavior without wearing a helmet exists or not according to the first video data, and judging whether an illegal behavior of running the red light or crossing the line or the like exists or not according to the moving track of the electric bicycle target in the red light state; according to the moving track of the electric bicycle in the second video data, judging whether the electric bicycle target has an illegal behavior of retrograde running or not; and under the condition that the illegal behavior exists, performing face recognition according to the image of the electric bicycle target, searching the contact way of the corresponding person, and sending prompt information. According to the invention, multiple detection algorithms are organically fused, accurate detection of illegal behaviors of the electric bicycle is realized, and the accuracy and reliability of detection are improved.

Description

Technical Field

[0001] The present invention relates to a traffic control system, and specifically, to a non-motor vehicle all-element management system and method. Background Art

[0002] Currently, technologies using intelligent algorithms to detect whether non-motor vehicles have illegal behaviors have started to be applied in the market. However, there are many types of illegal behaviors of electric bicycles, including: not wearing a helmet, going in the wrong direction, running a red light or crossing the stop line when the red light is on, resulting in difficulty for current intelligent algorithms to comprehensively cover the detection of illegal behaviors.

[0003] Patent document CN117275248A discloses a vehicle-mounted video and radio frequency integrated method and device for capturing illegal parking of non-motor vehicles, which needs to identify vehicle information by reading the passive RFID tag data on the electric bicycle. In practical applications, considering the penetration rate of passive RFID tags on electric bicycles, it is difficult to promote and apply.

[0004] Patent document CN113139438A discloses a method, system, device and storage medium for detecting the driving behaviors of non-motor vehicles. Since the drivers of electric bicycles usually wear helmets, it increases the difficulty of identifying the driver's identity. Therefore, this patent application contacts the vehicle owner by collecting the license plate number of the electric bicycle. However, the license plate number of the electric bicycle is small and there are problems such as non-standard installation and occlusion, and this patent has the problem of being difficult to contact the actual illegal driver. Summary of the Invention

[0005] Aiming at the defects in the prior art, the purpose of the present invention is to provide a non-motor vehicle all-element management system and method.

[0006] According to a non-motor vehicle all-element management method provided by the present invention, it includes:

[0007] Video acquisition step: reversely acquire the first video data of the non-motor vehicle lane at the intersection along the driving direction, and acquire the second video data of the non-motor vehicle lane at the intersection along the same driving direction;

[0008] First judgment step: identify the electric bicycle target according to the first video data, judge whether there is an illegal behavior of not wearing a helmet according to the image of the electric bicycle target, and judge whether there is an illegal behavior of running a red light or crossing the stop line when the red light is on according to the moving track of the electric bicycle target in the red light state;

[0009] Second judgment step: identify the electric bicycle target according to the second video data, judge whether the electric bicycle target has an illegal behavior of going in the wrong direction according to the moving track of the electric bicycle, and judge whether there is an illegal behavior of not wearing a helmet at the same time according to the image of the electric bicycle target in the case of an illegal behavior of going in the wrong direction;

[0010] Identification step: when the judgment result of the first judgment step or the second judgment step is that there is an illegal act, face recognition is performed based on the image of the electric bicycle target;

[0011] Notification steps: Based on the face recognition results, find the contact information of the corresponding person and send a prompt message.

[0012] Furthermore, the methods for determining whether there is an illegal act of not wearing a helmet include:

[0013] An image of the electric bicycle target is extracted from the first video data / the second video data, a head region is segmented from the image, contour features of the head region are extracted using an edge detection algorithm to obtain a contour point set, geometric features of the contour point set are calculated, texture features of the head region are extracted using a local binary pattern algorithm, and a feature vector of the head region is obtained by combining the geometric features and the texture features;

[0014] The head area feature vector is input into the trained classification model to calculate a probability value. When the probability value is less than the preset probability threshold T1, it is judged that there is an illegal behavior of not wearing a helmet.

[0015] Furthermore, the methods for determining whether there is a red light violation such as running a red light or crossing a line include:

[0016] In the red light state, a series of coordinate points (x i ,y i ), i = 1, 2, ... n, n is the total number of coordinate points;

[0017] Calculation judgment function:

[0018]

[0019] When D is greater than or equal to the preset stop line distance threshold T2 and the red light state has not ended, it is judged that there is a red light running violation. When D is greater than 0 and less than the preset stop line distance threshold T2, it is judged that there is a red light crossing violation.

[0020] Furthermore, the methods for determining whether the electric bicycle target has committed illegal acts of driving against traffic include:

[0021] Get the coordinate point (x t ,y t ), and the coordinate point (x t+Δt ,y t+Δt ), calculate the actual moving trajectory vector u x =xt+Δt -x t ,u y =y t+Δt -y t ;

[0022] Calculate the cosine value cosθ of the angle between the actual movement trajectory vector and the normal movement trajectory vector :

[0023]

[0024]

[0025] When the cosine value cosθ is less than the preset angle threshold T3, it is determined that there is a reverse driving violation.

[0026] Furthermore, the method for face recognition based on the image of the electric bicycle target includes:

[0027] Extract the face feature vector E = [e1, e2,... e m from the image of the electric bicycle target, and calculate the similarity S between the extracted face feature vector and the face feature vector in the database:

[0028]

[0029] where ω j is the weight corresponding to the face feature e j , e jbd is the corresponding face feature in the database, and when S is greater than the preset similarity threshold T4, the recognition is successful.

[0030] Furthermore, the first judgment step or the second judgment step further includes:

[0031] When it is determined that there is a violation, a on-site voice reminder is given, and the image of the corresponding electric bicycle target is displayed.

[0032] Furthermore, the first judgment step or the second judgment step further includes:

[0033] When it is determined that there is a violation, the images of multiple corresponding electric bicycle targets and the large face images at different times are combined and uploaded.

[0034] Furthermore, the face feature vector includes eye features, and the weight of the eye features is greater than the weights of the other parts.

[0035] According to the non-motor vehicle all-element management system provided by the present invention, it includes:

[0036] Video acquisition module: collects first video data of the non-motor vehicle lane at the intersection in the reverse direction along the driving direction, and collects second video data of the non-motor vehicle lane at the intersection in the same direction as the driving direction;

[0037] The first judgment module is used to identify the electric bicycle target according to the first video data, judge whether there is an illegal behavior of not wearing a helmet according to the image of the electric bicycle target, and judge whether there is an illegal behavior of running a red light or crossing the line according to the movement trajectory of the electric bicycle target under the red light state;

[0038] The second judgment module: identifies the electric bicycle target according to the second video data, judges whether the electric bicycle target has violated the law by driving against the flow according to the moving track of the electric bicycle, and judges whether the electric bicycle target has violated the law by not wearing a helmet according to the image of the electric bicycle target in the case of violating the law by driving against the flow;

[0039] Recognition module: when the first judgment module or the second judgment module determines that there is an illegal act, face recognition is performed based on the image of the electric bicycle target;

[0040] Notification module: Based on the face recognition results, find the contact information of the corresponding person and send a prompt message.

[0041] Furthermore, the video acquisition module and the first judgment module or the second judgment module are arranged in a front-end intelligent camera, and the recognition module and the notification module are arranged in a back-end integrated traffic management platform.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The present invention organically integrates multiple detection algorithms, realizes accurate detection of electric bicycle violations, improves detection accuracy and reliability, effectively solves the shortcomings of traditional monitoring methods, and has high practical value and promotion significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0045] Figure 1 It is a work flow chart of the present invention;

[0046] Figure 2 FIG. 4 is a system architecture diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0048] Example 1

[0049] like Figure 1 As shown, a method for managing all elements of non-motor vehicles includes:

[0050] Video collection steps: Set up a camera at the intersection, collect the first video data of the non-motorized vehicle lane at the intersection in the reverse direction of the driving direction, and collect the second video data of the non-motorized vehicle lane at the intersection in the same direction of the driving direction. The camera's setting position and angle must be accurately calculated to ensure that the non-motorized vehicle lane can be fully and clearly captured.

[0051] The first judgment step: identify the electric bicycle target based on the first video data, judge whether there is a violation of not wearing a helmet based on the image of the electric bicycle target, and judge whether there is a violation of the red light such as running a red light or crossing the line based on the movement trajectory of the electric bicycle target in the red light state.

[0052] The second judgment step: identify the electric bicycle target based on the second video data, judge whether the electric bicycle target has committed the illegal behavior of driving against the flow based on the moving trajectory of the electric bicycle, and if there is the illegal behavior of driving against the flow, judge whether there is also the illegal behavior of not wearing a helmet based on the image of the electric bicycle target.

[0053] When an illegal act is judged, a voice reminder is given on the spot and the image of the corresponding electric bicycle target is displayed. When an illegal act is judged, multiple images of corresponding electric bicycle targets and large facial images at different times are combined and uploaded to further identify the information of the offender.

[0054] Ways to determine whether there is a violation of not wearing a helmet include:

[0055] Extract the images of electric bicycle targets from the first video data / second video data, segment the head region from the images, and use an edge detection algorithm (such as the Canny algorithm) to extract the contour features of the head region, obtaining a contour point set C = {c1, c2,...}. Then calculate the geometric features of the contour point set, such as area, perimeter, and ellipse fitting parameters (major axis, minor axis, and rotation angle). These geometric features constitute part of the features of the head region. Use the Local Binary Pattern (LBP) algorithm to extract the texture features of the head region, divide the head region into multiple sub-regions, calculate the LBP feature vectors for each sub-region, and finally concatenate the LBP feature vectors of all sub-regions to obtain the texture feature vector. Combine the geometric features and texture features to obtain the head region feature vector.

[0056] Collect a large number of head image samples of electric bicycle riders with and without helmets, and divide them into a training set and a test set. For the samples in the training set, manually label the tags indicating whether they wear helmets. Use a Convolutional Neural Network (CNN) for model training, such as the classic AlexNet or VGG16 network structure. During the training process, use the head region feature vector as the input of the network and the tag as the output of the network, and continuously adjust the weights and biases of the network through the backpropagation algorithm to minimize the error (such as the cross-entropy loss function) between the prediction result of the network and the true tag. After multiple rounds of training, obtain a trained classification model.

[0057] Input the extracted head region feature vector into the trained classification model, and through a series of calculations of convolutional layers, pooling layers, and fully connected layers, finally output a probability value, which represents the probability that the rider in the current image wears a helmet. When this probability value is less than the preset probability threshold T1, it is judged that there is an illegal act of not wearing a helmet. The probability threshold T1 can be determined by conducting experiments on the test set and comprehensively considering factors such as the false positive rate and false negative rate to determine a suitable value. For example, take 0.5, that is, when the probability predicted by the model that the rider wears a helmet is less than 0.5, it is determined that the rider does not wear a helmet.

[0058] The methods for judging whether there are illegal acts such as running a red light or crossing the stop line when the red light is on include:

[0059] In the red light state, obtain a series of coordinate points (x i , y i ) of the moving trajectory of the electric bicycle target in the first video data, where i = 1, 2,... n, and n is the total number of coordinate points.

[0060] Calculate the judgment function:

[0061]

[0062] When D is greater than or equal to the preset stop line distance threshold T2 and the red light state has not ended, it is judged that there is a red light running violation. When D is greater than 0 and less than the preset stop line distance threshold T2, it is judged that there is a red light crossing violation.

[0063] Methods for judging whether an electric bicycle target has violated the law by driving against traffic include:

[0064] Get the coordinate point (x t ,y t ), and the coordinate point (x t+Δt ,y t+Δt ), calculate the actual moving trajectory vector u x =x t+Δt -x t ,u y =y t+Δt -y t The value of Δt should be reasonably determined according to the frame rate of the video. For example, for a video with a frame rate of 25fps, Δt can be set to 0.1s, that is, the movement trajectory vector is calculated every 2-3 frames.

[0065] Calculate the actual moving trajectory vector With the normal moving trajectory vector The cosine value of the angle cosθ, the normal moving trajectory vector The determination needs to be combined with the actual road planning and traffic rules of the intersection, for example, it can be determined based on the centerline direction of the lane.

[0066]

[0067] In actual calculations, in order to improve calculation efficiency, an optimized numerical calculation library (such as related functions in OpenCV) can be used to perform vector operations. When the cosine value cosθ is less than the preset angle threshold T3, it is judged that there is a violation of driving against the flow. The setting of the angle threshold T3 needs to take into account a variety of factors, such as the complexity of the road, the randomness of vehicle driving, etc. By conducting a large number of experiments in different scenarios, the distribution of the cosine value of the angle under normal driving and driving against the flow is statistically analyzed, and a suitable threshold is determined by statistical methods (such as maximum likelihood estimation). For example, in general urban road scenarios, after multiple experimental verifications, T3 can be set to 0.3. In the case of judging that there is a violation of driving against the flow, the target image of the electric bicycle is detected for not wearing a helmet, and the detection method is the same as the detection of not wearing a helmet in the previous article.

[0068] Identification step: When the judgment result of the first judgment step or the second judgment step is that there is an illegal act, face recognition is performed based on the image of the electric bicycle target.

[0069] After obtaining the target image of the electric bicycle, for the case of wearing a helmet, first use an image segmentation algorithm to separate the helmet part from the facial area. The Mask R-CNN algorithm based on deep learning is adopted to accurately segment the facial area and remove the occlusion interference of the helmet on the facial features. Then, enhance the segmented facial image. The histogram equalization method is used to expand the dynamic range of the image, improve the image contrast, and make the facial detail features more obvious. At the same time, use Gaussian filtering to remove the noise in the image and avoid affecting the subsequent feature extraction due to noise interference.

[0070] In addition to extracting the conventional facial feature vectors, increase the extraction of the depth features of the eye region. Since the helmet may occlude part of the face, the eye features are relatively more stable and unique. A convolutional neural network model specifically for the eyes is adopted to extract features such as the texture, shape, and spatial position of the eyes, form the eye feature vectors, and fuse the conventional facial feature vectors and the eye feature vectors to obtain the facial feature vector E = [e1, e2,... e m .

[0071] Calculate the similarity S between the extracted facial feature vector and the facial feature vector in the database:

[0072]

[0073] Among them, ω j is the weight corresponding to the facial feature e j , e jdb is the corresponding facial feature in the database. When S is greater than the preset similarity threshold T4, the recognition is successful. For the fused feature vector extracted in the case of wearing a helmet, dynamically adjust the weights of each feature dimension. According to the stability and discriminability of different features in the helmet-wearing scenario, automatically learn and adjust the weights through a machine learning algorithm (such as the gradient descent method). For example, for the eye feature dimension less affected by the helmet, appropriately increase its weight; for the facial edge feature dimension easily occluded, reduce its weight.

[0074] Notification step: According to the face recognition result, search for the contact information of the corresponding person in the personnel information database and send a prompt message to inform them of the illegal act and relevant handling measures. For example, only send a text message reminder for the first violation, send a text message reminder and require the person to participate in publicity and education for occasional violations, and impose fines and other penalties for multiple violations. Guide the driver to drive the electric bicycle in accordance with the law and regulations, reduce the illegal phenomenon of electric bicycles, reduce the traffic accidents of electric bicycles, and improve the management level of electric bicycles.

[0075] Example 2

[0076] Such as Figure 2As shown, the present invention also provides a non-motor vehicle full-factor management system, which can be implemented by executing the process steps of the non-motor vehicle full-factor management method, that is, those skilled in the art can understand the non-motor vehicle full-factor management method as a preferred implementation of the non-motor vehicle full-factor management system. The system includes:

[0077] Video acquisition module: It is set in the front-end intelligent camera, collects the first video data of the non-motor vehicle lane at the intersection in the reverse direction of the driving direction, and collects the second video data of the non-motor vehicle lane at the intersection in the same direction as the driving direction.

[0078] The first judgment module is set in the intelligent camera at the front end, identifies the electric bicycle target according to the first video data, judges whether there is a violation of not wearing a helmet according to the image of the electric bicycle target, and judges whether there is a violation of the red light such as running a red light or crossing the line according to the movement trajectory of the electric bicycle target in the red light state.

[0079] The second judgment module is arranged in the intelligent camera at the front end, identifies the electric bicycle target according to the second video data, judges whether the electric bicycle target has committed the illegal behavior of going against the flow according to the moving trajectory of the electric bicycle, and judges whether the illegal behavior of not wearing a helmet also exists according to the image of the electric bicycle target when there is the illegal behavior of going against the flow.

[0080] Identification module: A comprehensive traffic management platform is set up at the back end. When the first judgment module or the second judgment module determines that there is an illegal behavior, face recognition is performed based on the image of the electric bicycle target.

[0081] Notification module: Based on the face recognition results, find the contact information of the corresponding person and send a prompt message.

[0082] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0083] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A non-motor vehicle full-element management method, characterized in that include: Video collection step: collecting first video data of the non-motor vehicle lane at the intersection in the reverse direction along the driving direction, and collecting second video data of the non-motor vehicle lane at the intersection in the same direction as the driving direction; The first judgment step: identifying the electric bicycle target according to the first video data, judging whether there is an illegal behavior of not wearing a helmet according to the image of the electric bicycle target, and judging whether there is an illegal behavior of running a red light or crossing the line according to the movement trajectory of the electric bicycle target under the red light state; The second judgment step: identifying the electric bicycle target according to the second video data, judging whether the electric bicycle target has committed a violation of driving against the flow according to the moving trajectory of the electric bicycle, and judging whether the violation of driving against the flow has also occurred according to the image of the electric bicycle target; Identification step: when the judgment result of the first judgment step or the second judgment step is that there is an illegal act, face recognition is performed based on the image of the electric bicycle target; Notification steps: Based on the face recognition results, find the contact information of the corresponding person and send a prompt message.

2. The non-motor vehicle full-element management method according to claim 1, wherein Ways to determine whether there is a violation of not wearing a helmet include: An image of the electric bicycle target is extracted from the first video data / the second video data, a head region is segmented from the image, contour features of the head region are extracted using an edge detection algorithm to obtain a contour point set, geometric features of the contour point set are calculated, texture features of the head region are extracted using a local binary pattern algorithm, and a feature vector of the head region is obtained by combining the geometric features and the texture features; The head area feature vector is input into the trained classification model to calculate a probability value. When the probability value is less than the preset probability threshold T1, it is judged that there is an illegal behavior of not wearing a helmet.

3. The non-motor vehicle full-element management method according to claim 1, wherein Ways to determine whether there is a red light violation such as running a red light or crossing a red light include: In the red light state, a series of coordinate points (x i , y i ) of the target moving trajectory of the electric bicycle are obtained, where i = 1, 2,..., n, and n is the total number of coordinate points; Calculation judgment function: When D is greater than or equal to the preset stop line distance threshold T2 and the red light state has not ended, it is judged that there is a red light running violation. When D is greater than 0 and less than the preset stop line distance threshold T2, it is judged that there is a red light crossing violation.

4. The non-motor vehicle full-element management method according to claim 1, characterized in that Methods for judging whether an electric bicycle target has violated the law by driving against traffic include: Obtain the coordinate points (x t , y t ) of the electric bicycle target at time t in the second video data, and the coordinate points (x t+Δt , y t+Δt ) at time t+Δt, and calculate the actual movement trajectory vector u x = x t+Δt - x t , u y = y t+Δt - y t ; Calculate the actual movement trajectory vector and the normal movement trajectory vector for the cosine value of the included angle cosθ: When the cosine value cosθ is less than the preset angle threshold T3, it is determined that there is a wrong-way illegal behavior.

5. The non-motor vehicle full-element management method according to claim 1, characterized in that, Methods for performing face recognition based on images of electric bicycle targets include: According to the image extraction of the electric bicycle target, the face feature vector E = [e1, e2, … e m is obtained. Calculate the similarity S between the extracted face feature vector and the face feature vector in the database: Among them, ω j is the weight corresponding to the human face feature e j , and e jdb is the corresponding human face feature in the database. Recognition is successful when S is greater than the preset similarity threshold T4.

6. The non-motor vehicle full-element management method according to claim 1, wherein The first determination step or the second determination step further includes: When it is determined that there is an illegal behavior, an on-site voice reminder will be given and an image of the corresponding electric bicycle target will be displayed.

7. The non-motor vehicle full-element management method according to claim 1, characterized in that The first determination step or the second determination step further includes: When determining whether there is an illegal behavior, multiple corresponding electric bicycle target images and large face images at different times are combined and uploaded.

8. The non-motor vehicle all-element management method according to claim 5, characterized in that The facial feature vector includes eye features, and the weight of the eye features is greater than the weights of other parts.

9. A non-motor vehicle all-element management system, characterized in that include: Video acquisition module: collects first video data of the non-motor vehicle lane at the intersection in the reverse direction along the driving direction, and collects second video data of the non-motor vehicle lane at the intersection in the same direction as the driving direction; The first judgment module is used to identify the electric bicycle target according to the first video data, judge whether there is an illegal behavior of not wearing a helmet according to the image of the electric bicycle target, and judge whether there is an illegal behavior of running a red light or crossing the line according to the movement trajectory of the electric bicycle target under the red light state; The second judgment module: identifies the electric bicycle target according to the second video data, judges whether the electric bicycle target has violated the law by driving against the flow according to the moving track of the electric bicycle, and judges whether the electric bicycle target has violated the law by not wearing a helmet according to the image of the electric bicycle target in the case of violating the law by driving against the flow; Recognition module: when the first judgment module or the second judgment module determines that there is an illegal act, face recognition is performed based on the image of the electric bicycle target; Notification module: Based on the face recognition results, find the contact information of the corresponding person and send a prompt message.

10. The non-motor vehicle full-element management system according to claim 9, characterized in that The video acquisition module and the first judgment module or the second judgment module are arranged in the front-end intelligent camera, and the recognition module and the notification module are arranged in the back-end comprehensive traffic management platform.

Images