Intersection dangerous behavior detection method based on attitude analysis and head attribute recognition

Abstract

The invention relates to the field of intelligent traffic, in particular to an intersection dangerous behavior detection method based on posture analysis and head attribute recognition. The method comprises the following steps: firstly, acquiring an intersection electronic police video stream, cutting and enhancing, then, acquiring a posture analysis list by using a computer vision algorithm, then, carrying out attribute identification on a head area, carrying out serial number matching, and finally, judging whether a dangerous behavior exists or not based on the posture analysis list and a matching result, and carrying out reminding. Whether dangerous traffic behaviors exist or not is judged by using posture analysis and head attribute classification methods in electronic police monitoring real-time video pictures of urban congested intersections, notification is carried out in modes of broadcasting, display screen reminding and the like, the traffic safety of pedestrians and non-motor vehicles at the intersections is improved, the traffic environment is improved, and the traffic safety is improved. And a more intelligent and safer solution is provided for urban traffic management.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation, and in particular to a method for detecting dangerous behaviors at intersections based on posture analysis and head attribute recognition. Background Art

[0002] As artificial intelligence application scenarios become increasingly complex, posture analysis has become an important research direction in the field of machine vision. It can not only accurately locate the position of the target in the image, but also determine the movement form of the target object. The continuous upgrading and development of posture analysis technology has enabled it to play an important role in smart parks, smart transportation, drones and other fields.

[0003] With the continuous advancement of urbanization, the increasing demand for traffic and the imbalance of time and space for citizens' commuting needs have increased the congestion of urban roads. With the concept of smart city construction being proposed, a large number of methods for regulating motor vehicle driving behavior have been used, but there are few regulations for dangerous behaviors of pedestrians and non-motor vehicles. According to statistics from the Jinan Traffic Police Detachment, there are 4.04 million motor vehicles and 4.698 million electric bicycles in Jinan. In Jinan, the number of pedestrians and non-motor vehicles is already greater than that of motor vehicles. As the "weak" party of traffic subjects, they deserve attention. Pedestrians and non-motor vehicles have the characteristics of small targets, large numbers, and more concealment. If target detection is performed directly on them, it is easy to identify the wrong target (such as the driver and passenger of an electric vehicle bound to another electric vehicle). Therefore, a dangerous behavior detection method for intersections based on posture analysis and head attribute recognition is needed. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method for detecting dangerous behaviors at intersections based on posture analysis and head attribute recognition, which uses posture analysis and head attribute classification methods to determine whether there is dangerous traffic behavior, and has high detection accuracy.

[0005] In order to solve the technical problem, the technical solution adopted by the present invention is: a dangerous behavior detection method at an intersection based on posture analysis and head attribute recognition, comprising the following steps: S01, obtain the video stream information of the intersection electric police camera, divide the video stream into frames, and obtain an image set consisting of t frames of intersection electric police images. ,in is the i-th frame of the intersection electric police image, ; S02, image set Intercept the traffic area at the intersection to obtain the image set of the traffic area at the intersection , and in the image set of the intersection traffic area The pedestrian and non-motor vehicle traffic areas are determined by the bid; S03, using computer vision algorithm to analyze the image of the intersection traffic area in the i-th frame Perform target detection and posture analysis on pedestrians, motor vehicles, and non-motor vehicles, and obtain a posture analysis list of motor vehicles, pedestrians, and non-motor vehicles; S04, performing image classification on the head area in the target detection frame to obtain the head attributes of each person; and matching the head with the person, and the non-motor vehicle with the passenger; S05. Identify dangerous behaviors based on the posture analysis list and matching results.

[0006] Furthermore, step S02 is specifically as follows: S21, using the slicing operation to crop the intersection electric police image set X, to obtain the cropped image set ; S22, yes Perform contrast enhancement to obtain an image set of pedestrian and non-motor vehicle traffic areas ; S23, Manually mark pedestrian and non-motor vehicle traffic areas Indicates the area number and sets the threshold for judging the gathering of people in each area .

[0007] Furthermore, step S03 is specifically as follows: S31. Use the YOLOv11m-pose model to analyze the image set Perform target detection and obtain the target detection list , where i represents the i-th frame, j represents the j-th detected target, x is the horizontal coordinate of the center point of the detection frame, y is the vertical coordinate of the center point of the detection frame, w is the width of the detection frame, and h is the height of the detection frame; S32. Use the YOLOv11m-pose model to perform posture analysis and key point marking on the target detection results to obtain a posture analysis list , where pose_point represents the key point of the detected target.

[0008] In step S03, the targets detected by target detection include: people, heads, non-motor vehicles, and motor vehicles. Non-motor vehicles include bicycles, electric vehicles, and motorcycles. The key points for detecting a person are [head, shoulders, elbows, wrists, hips, knees, ankles], and the key points for detecting a non-motor vehicle are [handlebars, front wheel, rear wheel, front seat, rear seat]. There are no key points for detecting people or motor vehicles.

[0009] Furthermore, step S04 is specifically as follows: S41. Generate a bipartite graph by combining the point set A whose key point is the head and the center point set B whose key point is the head in the posture analysis list of all targets as human beings , where the distance between the edge of point set A and any point in point set B is its Euclidean distance, and the Hungarian matching algorithm is used for Perform matching and make one-to-one correspondence between the person detection frame and the head detection frame; S42, generating a bipartite graph from the front seat key point coordinate point set C and the rear seat key point coordinate point set D in the posture analysis list of all targets that are non-motor vehicles and the hip key point coordinate point set E in the posture analysis list of the target that is a person , , where the distance between the edge of point set C and any point in point set E is its Euclidean distance; the distance between the edge of point set D and any point in point set E is its Euclidean distance. The Hungarian matching algorithm is used to , Matching is performed to determine whether someone is driving the non-motor vehicle and whether there is a person in the back seat of the non-motor vehicle ,in represents j people who have been detected, They represent walking, driving a non-motorized vehicle, and riding in the back seat of a non-motorized vehicle respectively; S43, traverse all posture analysis lists where the target is a head, expand the size of the head detection frame, and obtain the coordinates of the head attribute classification frame ; S44, use slice operation Crop the corresponding position, use the Resnet-18 network to classify the cropped image, and obtain the head attribute status: , where j represents the person's number. If there is a behavior, the corresponding list element is 1, otherwise it is 0.

[0010] Furthermore, step S05 is specifically as follows: S51, connecting the detected hip key points and ankle key points of the person, and determining whether the angle between the connecting line and the ground plane normal vector is greater than 75°. If the condition is continuously met for more than 3 seconds, it is determined that there is a dangerous behavior: the person falls; S52, according to the head attribute status Determine whether the target person has dangerous behavior: make or receive phone calls at intersections, combined with head attribute status Matching results Determine whether there are dangerous behaviors: driving a non-motor vehicle without wearing a helmet, carrying passengers in the back seat of a non-motor vehicle, or smoking while driving a non-motor vehicle; S53: Determine whether the number of people detected in the pedestrian and non-motor vehicle traffic area is greater than ,If this condition is met, it is judged that there is a dangerous behavior: gathering of people.

[0011] Furthermore, a list of dangerous behaviors is set before executing step S05. , For the number of people, Represents dangerous behaviors, with values ​​of [driving a non-motor vehicle without wearing a helmet, carrying passengers in the back seat of a non-motor vehicle, smoking while driving a non-motor vehicle, making phone calls at intersections, crowds of people, people falling]. Through the judgment of steps S51 to S53, the corresponding dangerous behavior Rb value is added to the dangerous behavior list, and broadcast reminders or display screen reminders are made based on the dangerous behaviors at the current intersection fed back by the dangerous behavior list, and the dangerous behavior list is synchronized to the back-end traffic police platform.

[0012] Furthermore, when there is a gathering of people, =0.

[0013] Furthermore, in step S01, the video stream information of the electric police camera at the intersection is obtained through the ONVIF protocol.

[0014] Furthermore, the pedestrian and non-motor vehicle traffic area calibrated in step S02 includes pedestrian and non-motor vehicle traffic light waiting areas and non-motor vehicle lanes, and .

[0015] Beneficial effects of the present invention: In the real-time video images of the electric police monitoring at the city's congested intersections, the present invention uses posture analysis and head attribute classification methods to determine whether there is dangerous traffic behavior. The posture analysis obtains a posture analysis list of motor vehicles, pedestrians, and non-motor vehicles, and classifies the human head area in the target detection frame to obtain the head attributes of each person; and matches the human head with the person, and the non-motor vehicle with the passenger, to avoid the situation of target recognition errors. Notifications are made through broadcasting, display screen reminders, etc., to improve the safety of pedestrians and non-motor vehicles at intersections, improve the traffic environment, and provide a more intelligent and safe solution for urban traffic management. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of the method is shown in FIG. Figure 2 This is the actual effect diagram of this method. DETAILED DESCRIPTION

[0017] The present invention is further described and defined below in conjunction with the accompanying drawings and specific implementations.

[0018] Example 1 This embodiment discloses a method for detecting dangerous behaviors at intersections based on posture analysis and head attribute recognition. Figure 1 As shown, the following steps are included: S01. Obtain the video stream information of the intersection electric police camera through the ONVIF protocol, divide the video stream into frames, and obtain an image set consisting of t frames of intersection electric police images. ,in is the i-th frame of the intersection electric police image, ; S02, image set Intercept the traffic area at the intersection to obtain the image set of the traffic area at the intersection , and in the image set of the intersection traffic area The pedestrian and non-motor vehicle traffic areas are determined by the bid; S03, using computer vision algorithm to analyze the image of the intersection traffic area in the i-th frame Perform target detection and posture analysis on pedestrians, motor vehicles, and non-motor vehicles, and obtain a posture analysis list of motor vehicles, pedestrians, and non-motor vehicles; S04, performing image classification on the "head" region in the target detection frame to obtain the head attributes of each person; and matching the "head" with the "person", and the "non-motor vehicle" with the "person" on board; S05. Identify dangerous behaviors based on the posture analysis list and matching results.

[0019] In this embodiment, step S02 is specifically as follows: S21. Use the slice operation in the C++ language opencv library to crop the intersection electric police image set X to obtain the cropped image set ; S22, use cv::equalizeHist in the C++ language opencv library Perform contrast enhancement to obtain an image set of pedestrian and non-motor vehicle traffic areas ; S23, Manually mark pedestrian and non-motor vehicle traffic areas Indicates the area number and sets the threshold for judging the gathering of people in each area The calibrated pedestrian and non-motor vehicle traffic areas include pedestrian and non-motor vehicle traffic light waiting areas and non-motor vehicle lanes. , can be dynamically adjusted according to the size of the area and the conditions of the intersection size.

[0020] In this embodiment, step S03 is specifically as follows: S31. Use the YOLOv11m-pose model to analyze the image set Perform target detection and obtain the target detection list , where i represents the i-th frame, j represents the j-th detected target, x is the horizontal coordinate of the center point of the detection frame, y is the vertical coordinate of the center point of the detection frame, w is the width of the detection frame, and h is the height of the detection frame.

[0021] In this embodiment, the target detection categories are: [person, head, non-motor vehicle, motor vehicle]; non-motor vehicles include bicycles, electric vehicles, and motorcycles (classified as non-motor vehicles for detection in this embodiment).

[0022] S32. Use the YOLOv11m-pose model to perform posture analysis and key point marking on the target detection results to obtain a posture analysis list , where pose_point represents the key point of the detected target.

[0023] The key points of the detection target "person" are [head, shoulders, elbows, wrists, hips, knees, ankles], the key points of the detection target "non-motor vehicle" are [handlebars, front wheel, rear wheel, front seat, rear seat], and the detection targets "human head" and "motor vehicle" are not key points.

[0024] In this embodiment, step S04 is specifically as follows: S41, the point set A whose key point is the head in the posture analysis list of all targets as "people" The key points of the detection box are the center point set B of the head Generate a bipartite graph , where the distance between the edge of point set A and any point in point set B is its Euclidean distance , using the Hungarian matching algorithm Match the "person" detection frame with the "head" detection frame one by one.

[0025] S42, the key point coordinate point set C of the "front seat" in the posture analysis list of all targets "non-motor vehicle" 、“Backseat” key point coordinate set D The coordinate point set E of the key point "hip" in the posture analysis list with the target "person" Generate a bipartite graph , , where the distance between the edge of point set C and any point in point set E is its Euclidean distance ; Similarly, the distance between the edge of point set D and any point in point set E is its Euclidean distance , using the Hungarian matching algorithm , Matching is performed to determine whether someone is driving the non-motor vehicle and whether there is a person in the back seat of the non-motor vehicle ,in Indicates the number of the person. They respectively represent walking, driving a non-motorized vehicle, and riding in the back seat of a non-motorized vehicle.

[0026] S43, traverse all posture analysis lists whose target is "human head", expand the size of the "human head" detection box, and obtain the coordinates of the head attribute classification box ; S44, use slice operation Crop the corresponding position, use the Resnet-18 network to classify the cropped image, and obtain the head attribute status: , where j represents the number of the "person". If there is a behavior, the corresponding list element is 1, otherwise it is 0.

[0027] In this embodiment, step S05 is specifically as follows: S51. Set up a list of dangerous behaviors , is the number of "person", Indicates dangerous behavior, with values ​​of [driving a non-motor vehicle without a helmet, carrying passengers in the back seat of a non-motor vehicle, smoking while driving a non-motor vehicle, making phone calls at intersections, crowds of people, people falling]. Through the judgment of steps S51 to S53, the corresponding dangerous behavior Rb value is added to the dangerous behavior list. Special attention: a person can have multiple dangerous behaviors; when there are collective dangerous behaviors such as crowds of people, =0.

[0028] S52, detect the number Draw a line between the key points of the hip and ankle of the “person” and determine whether the angle between the line and the normal vector of the ground plane is greater than 75°. If this condition is met for more than 3 seconds, it is determined that there is a dangerous behavior: the person falls; Add to There are dangerous behaviors and people fall.

[0029] S53, according to the head attribute status The judgment number is Whether the person is engaging in dangerous behavior: making phone calls at intersections, and adding the results to In. Combined with the header attribute status Matching results Determine whether there is dangerous behavior: driving a non-motor vehicle without wearing a helmet, carrying passengers in the back seat of a non-motor vehicle, smoking while driving a non-motor vehicle, and add the results to In. The number is state=1, indicating that he drives a non-motor vehicle, and If there is a reference to "smoking", it means that there is a dangerous behavior of smoking while driving a non-motor vehicle.

[0030] S54: Determine whether the number of people detected in the pedestrian and non-motor vehicle traffic area is greater than ,If this condition is met, it is judged that there is a dangerous behavior: gathering of people.

[0031] S55. According to the list of dangerous behaviors The dangerous behaviors at the current intersection are reported through broadcast or display reminders, and the list of dangerous behaviors is synchronized to the back-end traffic police platform, so that the staff can understand the situation at the intersection and make personalized management of the intersection.

[0032] like Figure 2 As shown, it is an actual effect diagram of the present invention. Through this method, dangerous behaviors such as smoking, making phone calls, and carrying passengers by pedestrians or non-motor vehicles at intersections can be identified.

[0033] The above description is only the basic principle and preferred embodiments of the present invention. Improvements and substitutions made by those skilled in the art based on the present invention belong to the protection scope of the present invention.

Claims

1. A method for detecting dangerous behaviors at intersections based on posture analysis and head attribute recognition, characterized in that: The following steps are involved: S01, obtain the video stream information of the intersection electric police camera, divide the video stream into frames, and obtain an image set consisting of t frames of intersection electric police images. ,in is the i-th frame of the intersection electric police image, ; S02, image set Intercept the traffic area at the intersection to obtain the image set of the traffic area at the intersection , and in the image set of the intersection traffic area The area for pedestrians and non-motor vehicles is determined by the bid; S03, using computer vision algorithm to analyze the image of the intersection traffic area in the i-th frame Perform target detection and posture analysis on pedestrians, motor vehicles, and non-motor vehicles, and obtain a posture analysis list of motor vehicles, pedestrians, and non-motor vehicles; S04, performing image classification on the head area in the target detection frame to obtain the head attributes of each person; And match heads with people, non-motor vehicles with passengers; S05. Identify dangerous behaviors based on the posture analysis list and matching results.

2. The method for detecting dangerous behaviors at intersections based on posture analysis and head attribute recognition according to claim 1, characterized in that: Step S02 is specifically as follows: S21, using the slicing operation to crop the intersection electric police image set X, to obtain the cropped image set ; S22, yes Perform contrast enhancement to obtain an image set of pedestrian and non-motor vehicle traffic areas ; S23. Demarcate pedestrian and non-motor vehicle traffic areas Indicates the area number and sets the threshold for judging the gathering of people in each area .

3. The method for detecting dangerous behaviors at intersections based on posture analysis and head attribute recognition according to claim 1, characterized in that: Step S03 is specifically as follows: S31. Use the YOLOv11m-pose model to analyze the image set Perform target detection and obtain the target detection list , where i represents the i-th frame, j represents the j-th detected target, x is the horizontal coordinate of the center point of the detection frame, y is the vertical coordinate of the center point of the detection frame, w is the width of the detection frame, and h is the height of the detection frame; S32. Use the YOLOv11m-pose model to perform posture analysis and key point marking on the target detection results to obtain a posture analysis list , where pose_point represents the key point of the detected target.

4. The method for detecting dangerous behaviors at intersections based on posture analysis and head attribute recognition according to claim 3, characterized in that: The targets detected by target detection include: people, heads, non-motor vehicles, and motor vehicles. Non-motor vehicles include bicycles, electric vehicles, and motorcycles. The key points of the detection target for humans are [head, shoulders, elbows, wrists, hips, knees, ankles], and the key points of the detection target for non-motor vehicles are [handlebars, front wheel, rear wheel, front seat, rear seat]. Key points are not marked for detection targets such as humans and motor vehicles.

5. The method for detecting dangerous behaviors at intersections based on posture analysis and head attribute recognition according to claim 4, characterized in that: Step S04 is specifically as follows: S41. Generate a bipartite graph by combining the point set A whose key point is the head and the center point set B whose key point is the head in the posture analysis list of all targets as human beings , where the distance between the edge of point set A and any point in point set B is its Euclidean distance, and the Hungarian matching algorithm is used for Perform matching and make one-to-one correspondence between the person detection frame and the head detection frame; S42, generating a bipartite graph from the front seat key point coordinate point set C and the rear seat key point coordinate point set D in the posture analysis list of all non-motor vehicles and the hip key point coordinate point set E in the posture analysis list of all human targets , , where the distance between the edge of point set C and any point in point set E is its Euclidean distance; the distance between the edge of point set D and any point in point set E is its Euclidean distance. The Hungarian matching algorithm is used to , Matching is performed to determine whether someone is driving the non-motor vehicle and whether there is a person in the back seat of the non-motor vehicle ,in represents j people who have been detected, They represent walking, driving a non-motorized vehicle, and riding in the back seat of a non-motorized vehicle respectively; S43, traverse all posture analysis lists where the target is a head, expand the size of the head detection frame, and obtain the coordinates of the head attribute classification frame ; S44, use slice operation Crop the corresponding position, use the Resnet-18 network to classify the cropped image, and obtain the head attribute status: , where j represents the person's number. If there is a behavior, the corresponding list element is 1, otherwise it is 0.

6. The method for detecting dangerous behaviors at intersections based on posture analysis and head attribute recognition according to claim 5, characterized in that: Step S05 is specifically as follows: S51, connecting the detected hip key points and ankle key points of the person, and determining whether the angle between the connecting line and the ground plane normal vector is greater than 75°. If the condition is continuously met for more than 3 seconds, it is determined that there is a dangerous behavior: the person falls; S52, according to the head attribute status Determine whether the target person has dangerous behavior: make or receive phone calls at intersections, combined with head attribute status Matching results Determine whether there are dangerous behaviors: driving a non-motor vehicle without wearing a helmet, carrying passengers in the back seat of a non-motor vehicle, or smoking while driving a non-motor vehicle; S53: Determine whether the number of people detected in the pedestrian and non-motor vehicle traffic area is greater than If this condition is met, it is judged that there is a dangerous behavior: gathering of people.

7. The method for detecting dangerous behaviors at intersections based on posture analysis and head attribute recognition according to claim 6, characterized in that: Set up a list of dangerous behaviors before executing step S05 , For the number of people, Represents dangerous behaviors, with values ​​of [driving a non-motor vehicle without wearing a helmet, carrying passengers in the back seat of a non-motor vehicle, smoking while driving a non-motor vehicle, making phone calls at intersections, crowds of people, people falling]. Through the judgment of steps S51 to S53, the corresponding dangerous behavior Rb value is added to the dangerous behavior list, and broadcast reminders or display screen reminders are made based on the dangerous behaviors at the current intersection fed back by the dangerous behavior list, and the dangerous behavior list is synchronized to the back-end traffic police platform.

8. The method for detecting dangerous behaviors at intersections based on posture analysis and head attribute recognition according to claim 6, characterized in that: When there are crowds of people, =0.

9. The method for detecting dangerous behaviors at intersections based on posture analysis and head attribute recognition according to claim 1, characterized in that: In step S01, the video stream information of the intersection electric police camera is obtained through the ONVIF protocol.

10. The method for detecting dangerous behaviors at intersections based on posture analysis and head attribute recognition according to claim 2, characterized in that: The pedestrian and non-motor vehicle traffic area marked in step S02 includes pedestrian and non-motor vehicle traffic light waiting area and non-motor vehicle lane, and .

Images