A traffic police gesture recognition method in a congested intersection scenario

Abstract

The present invention belongs to the field of autonomous driving safety assistance technology, and specifically, is a method for traffic police gesture recognition in a congested intersection scenario. Step 1: Arrangement and calibration of road-side high-definition cameras and vehicle-side high-definition cameras; Step 2: Collection, labeling, and production of data sets of traffic police targets and gesture feature data at the intersection; Step 3: Determination of road-side traffic police targets and training of gesture recognition algorithms; Step 4: Vehicle-side distance measurement and fusion of vehicle-side and road-side information, and recognition of traffic police gestures in congested intersection scenarios. This recognition method integrates information from multiple sensors, such as multiple road-side high-definition cameras and vehicle-side binocular high-definition cameras, and is applied to traffic scenarios with congested vehicles and pedestrians, where vehicles have blind spots in their field of vision for traffic police or traffic police gestures, and obtains traffic police three-dimensional gestures in real time.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving safety assistance, and specifically is a method for traffic police gesture recognition in a congested intersection scenario. Background Art

[0002] With the continuous development and maturity of computer vision technology, the continuous updating and improvement of artificial intelligence algorithms, autonomous driving technology has also gradually developed. The emergence of autonomous driving can also bring great improvements to traffic safety and efficiency. Object detection and recognition in congested scenes has always been the research focus of autonomous driving and computer fields. Especially in the field of autonomous driving, the information of congested scenes is more complex, and vehicles, pedestrians, traffic signs, etc. make autonomous driving complicated. In this congested scene, where traffic police and traffic signs coexist, how to make correct plans based on correct information recognition has always been a hot and difficult issue in research. Summary of the invention

[0003] The present invention provides a method for recognizing traffic police gestures in a congested intersection scenario. The recognition method integrates information from multiple sensors, such as multiple high-definition cameras on the road side and binocular high-definition cameras on the vehicle side, and is applied to traffic scenarios with congested vehicles and pedestrians, where there is a blind spot in the vehicle's field of vision for traffic police or traffic police gestures, to obtain the traffic police's three-dimensional gestures in real time.

[0004] The technical solution of the present invention is described as follows in conjunction with the accompanying drawings:

[0005] A method for traffic police gesture recognition in a congested intersection scenario includes the following steps:

[0006] Step 1: Arrangement and calibration of roadside HD cameras and vehicle-side HD cameras;

[0007] Step 2: Collect, label and create a data set of traffic police targets and gesture feature data at the intersection;

[0008] Step 3: Determine the target of the roadside traffic police and train the gesture recognition algorithm;

[0009] Step 4: Vehicle-side distance measurement and vehicle-side road-side information fusion, and recognition of traffic police gestures in congested intersection scenarios.

[0010] The specific method of step one is as follows:

[0011] a) In an intersection with a scale of w×b, where b is the intersection width facing the vehicle and w is the intersection road width in the direction perpendicular to b, the turning point of each road edge is set as the reference point, and a roadbed with a height of h is placed. n roadside high-definition cameras are placed on the roadbed, where n≥4, and at least one high-definition camera is placed on the top of the roadbed at a height of h from the ground. Each of the roadside high-definition cameras detects synchronized high-definition video information in the intersection road from different directions, including traffic police, vehicles and pedestrians. The video content detected by all high-definition cameras should include all information in the vertical space of the intersection; the high-definition cameras are individually calibrated using Zhang Youzheng's camera calibration method, and all high-definition cameras are jointly calibrated using the epipolar geometry constraint method to accurately identify and locate targets;

[0012] b) The vehicle-side binocular high-definition camera is formed by combining two high-definition monocular cameras; the binocular high-definition camera is used by the vehicle-side to perceive the target information on the road ahead; the binocular high-definition camera uses Zhang Youzheng's camera calibration method to calibrate the high-definition camera separately, and uses the epipolar geometry constraint method to calibrate the high-definition camera so as to accurately identify and locate the target;

[0013] c) A conspicuous sign that is different from a traffic sign is set at each roadbed height h0 as a reference device for matching and associating the intersection space detected by the road-side high-definition camera with the intersection space detected by the vehicle-side binocular high-definition camera.

[0014] The specific method of step 2 is as follows:

[0015] A high-definition camera at the roadside is used to shoot synchronized high-definition videos from different perspectives of intersections with congested vehicles and pedestrians and normal operation of traffic lights. The high-definition videos are used as training data sets. The training data set annotation benchmark is as follows:

[0016] a) Define the vehicle coordinate system;

[0017] The vehicle's forward direction is the y-axis, the right direction perpendicular to the y-axis is the x-axis, and the upward direction perpendicular to the x- and y-axis planes is the z-axis; the roadbed coordinate system is defined as: the right direction of the intersection facing the vehicle is the x'-axis, the direction perpendicular to the x'-axis and consistent with the direction of the vehicle's x-axis is the y'-axis, and the roadbed direction facing the vehicle and closest to the left side perpendicular to the ground is the z'-axis direction;

[0018] b) Mark the traffic police category;

[0019] The categories for marking are: traffic police, traffic lights, vehicles, pedestrians and obstacles;

[0020] When a traffic policeman is present, the priority of the traffic policeman is higher than that of the traffic lights. Before recognizing the traffic policeman’s gestures, the traffic policeman’s target must be correctly identified. The traffic policeman in uniform must be marked as a traffic policeman according to the standard of whether the traffic policeman has uniform features. The upper garment has a higher priority than the hat. At the same time, the direction of the traffic policeman must be marked. The direction of the traffic policeman and the gesture jointly determine the direction of command.

[0021] c) Label the joints and bones of the traffic police’s arms;

[0022] The traffic police's traffic control gestures are completed by both arms, that is, only 10 finger joints, 1 wrist joint, 1 elbow joint and 1 shoulder joint of each arm of the traffic police are marked. Each joint point detected by the road-side high-definition camera or the vehicle-side binocular high-definition camera contains three-dimensional coordinate information. The finger joint points (h1…h 10 ) coordinates (h x1 ,h y1 ,h z1 )…(h x10 ,h y10 ,h z10 ), wrist joint point w coordinate (w x1 ,w y1 ,w z1 ), elbow joint point e coordinate (e x1 ,e y1 ,e z1 ) and shoulder joint coordinates s:(s x1 ,s y1 ,s z1 ), all the nodes form a label list Joints, all the nodes are connected to form a skeleton Limb, forming a skeleton label list; when a high-definition camera does not detect a joint point, the coordinate value is set to 0; the real position of each joint point of the traffic police in the three-dimensional space is calculated by triangulation method; as follows: Joints = [(h x1 ,h y1 ,h z1 )…(h xk ,h yk ,h zk ),(w x1 ,w y1 ,w z1 ),(e x1 ,e y1 ,e z1 ),(s x1 ,s y1 ,s z1 )]

[0023] Limb=[(h1,h2),(h3,h4),(h5,h6),(h7,h8),(h9,h 10),(h2,w),(h4,w),(h6,w),(h8,w),(h 10 ,w),(w,e),(e,s), where w is the coordinate of the wrist joint; e is the coordinate of the elbow joint; s is the coordinate of the shoulder joint;

[0024] d) Mark the traffic police’s hand gestures;

[0025] According to the different gestures of actual traffic police, different gestures are classified and labeled for training different gesture recognition.

[0026] The specific method of step three is as follows:

[0027] a) Using the high-resolution output neural network model HRNet as the backbone network for feature extraction to train 2D joint point detection for each HD camera;

[0028] b) The same joint point from different high-definition cameras is calculated through triangulation to restore the 3D joint point information of the traffic police's arm in space; the training data set of the vehicle-side and road-side algorithms is the same, but the number of high-definition cameras is different; the training process is to improve the detection accuracy of the 3D joint point position, and the loss function Loss when training 3D joint point recovery is defined as the detection value k of all related nodes d and the truth value k gt That is, the 2 norm of the annotation distance error. This loss function is also applicable to the training of the vehicle-side binocular camera detecting the traffic police joint points when there is no blind spot in the vehicle-side detection field of view. The specific definition of Loss is:

[0029]

[0030] Among them, k d Detect the coordinate values ​​of all relevant node spaces; k gt It is the real coordinate value of all related nodes in space.

[0031] The specific method of step 4 is as follows:

[0032] After the vehicle-side and road-side high-definition cameras are calibrated and the algorithm training is completed, the reference device is used as the medium to integrate the target information of the rectangular area with a range of w×b×h and the internal target information detected by the road-side multi-view as a whole, and the spatial and target information of the same area detected by the vehicle-side. b is the width of the intersection facing the vehicle, w is the width of the intersection road in the vertical direction of b, and h is the height of the high-definition camera on the top of the roadbed from the ground; the vehicle-side binocular camera detects that the actual distance between the vehicle and the two nearest roadbed reference devices is d1 and d2 respectively, the horizontal angle between the x-axis of the vehicle-side binocular camera coordinate system and the x′ axis of the intersection roadbed coordinate system is α, and the detection of the roadside The information will be rotated around the z′ axis by an angle of α to the same direction as the x-axis of the vehicle-side camera coordinate system based on the origin O of the roadbed coordinate system, and then scaled and displayed in the on-board computer according to the scaling factor m. The on-board computer will display and broadcast in real time whether there is a traffic police and the command prompt voice of the traffic police gesture. The detection result is used as input and transmitted to the automatic driving path planning algorithm for driving intention and intersection driving path planning; at the same time, the driver can zoom in and out the details of the traffic police gesture information on the detected spatial information by touch; the specific calculation method of the horizontal angle α between the x-axis of the vehicle-side binocular camera coordinate system and the x′ axis of the intersection roadbed coordinate system is:

[0033]

[0034] When the vehicle-side binocular camera is blocked by the vehicle in front or other obstacles, the distance detection can be directly transmitted to the vehicle-side; h0 is the height of the roadbed reference device from the ground;

[0035] When the vehicle-side binocular high-definition camera detects the traffic police and the traffic police gesture, the road-side detection space information is rotated by an angle of α to the same direction as the x-axis of the vehicle-side binocular high-definition camera coordinate system, and the Hungarian algorithm is used to match the same joint points. Considering the missing and error of the joint information detected by the vehicle-side binocular high-definition camera and the joint information restored by the road-side high-definition camera, the center point of the matched joint point is calculated as the position of the fused joint point;

[0036] In the extreme case where the traffic police are in plain clothes and it is impossible to detect whether the traffic police target exists through the traffic police uniform features, the target whose traffic control gesture exceeds the time period threshold t is defined as the traffic police.

[0037] The beneficial effects of the present invention are:

[0038] 1) The present invention enables vehicles in blind spots to obtain traffic police gestures in real time when they cannot obtain traffic police gesture information, and immediately plan driving intentions in real time, which can effectively improve traffic efficiency

[0039] 2) The present invention can identify traffic police and their gestures as true value information, and identify vehicles and pedestrians other than traffic police as background information that must be avoided. While improving traffic efficiency, it effectively improves the safety of the traffic system. This method can complete all related detection and recognition tasks at a low cost using only a few cameras. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 A schematic diagram of the arrangement of high-definition cameras at the roadside of an intersection and their positional relationship with vehicles;

[0042] Figure 2 is a flow chart of the present invention;

[0043] Figure 3 Schematic diagram of the traffic police arm joint point and skeleton dataset. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] A traffic police gesture recognition method in a congested intersection scenario is mainly used in intersection scenarios where vehicles and pedestrians are congested. Traffic lights are not sufficient to meet the needs of directing traffic, and traffic police are required to use gestures to dispatch vehicles and pedestrians.

[0046] See also Figure 2 , including the following steps:

[0047] Step 1: Arrangement and calibration of roadside HD cameras and vehicle-side HD cameras;

[0048] The specific method is as follows:

[0049] See also Figure 1, a) In an intersection of scale w×b, where b is the intersection width facing the vehicle, and w is the intersection road width in the direction perpendicular to b, set the turning point of each road edge as a reference point (4), and place a roadbed with a height of h, and place n roadside high-definition cameras on the roadbed, where n≥4, and at least one high-definition camera is placed on the top of each roadbed at a height of h from the ground, each of the roadside high-definition cameras detects synchronized high-definition video information in the intersection road from different directions, including traffic police, vehicles and pedestrians, and the video content detected by all high-definition cameras contains all information in the vertical space of the intersection; the high-definition cameras are calibrated individually using Zhang Youzheng's camera calibration method, and all high-definition cameras are jointly calibrated using the epipolar geometry constraint method to accurately identify and locate targets;

[0050] b) The vehicle-side binocular high-definition camera is formed by a combination of two high-definition monocular cameras; the binocular high-definition camera is used by the vehicle-side to perceive target information on the road ahead; the binocular high-definition camera uses Zhang Youzheng's camera calibration method to calibrate the high-definition camera separately, and uses the epipolar geometry constraint method to calibrate the high-definition camera so as to accurately identify and locate the target.

[0051] c) A conspicuous sign that is different from a traffic sign is set at each roadbed height h0 as a reference device for matching and associating the intersection space detected by the road-side high-definition camera with the intersection space detected by the vehicle-side binocular high-definition camera.

[0052] Step 2: Collect, label and create a data set of traffic police targets and gesture feature data at the intersection;

[0053] The specific method is as follows:

[0054] A high-definition camera at the roadside is used to shoot synchronized high-definition videos from different perspectives of intersections with congested vehicles and pedestrians and normal operation of traffic lights. The high-definition videos are used as training data sets. The training data set annotation benchmark is as follows:

[0055] a) Define the vehicle coordinate system;

[0056] The vehicle's forward direction is the y-axis, the right direction perpendicular to the y-axis is the x-axis, and the upward direction perpendicular to the x- and y-axis planes is the z-axis; the roadbed coordinate system is defined as: the right direction of the intersection facing the vehicle is the x'-axis, the direction perpendicular to the x'-axis and consistent with the direction of the vehicle's x-axis is the y'-axis, and the roadbed direction facing the vehicle and closest to the left side perpendicular to the ground is the z'-axis direction;

[0057] b) Mark the traffic police category;

[0058] The categories for marking are: traffic police, traffic lights, vehicles, pedestrians and obstacles;

[0059] When traffic police are present, their priority is higher than that of traffic lights. As public servants wearing specific uniforms, traffic police have distinct characteristics. Before identifying traffic police gestures, it is necessary to correctly identify traffic police targets and mark traffic police targets in uniform as traffic police based on whether they have uniform characteristics. The priority of tops is higher than that of hats. At the same time, the direction of traffic police must be marked. The direction of traffic police and gestures jointly determine the direction of command.

[0060] See also Figure 3 ,c) mark the joints and bones of the traffic police’s arm;

[0061] The traffic police's traffic control gestures are completed by both arms, that is, only 10 finger joints, 1 wrist joint, 1 elbow joint and 1 shoulder joint of each arm of the traffic police are marked. Each joint point detected by the road-side high-definition camera or the vehicle-side binocular high-definition camera contains three-dimensional coordinate information. The finger joint points (h1…h 10 ) coordinates (h x1 ,h y1 ,h z1 )…(h x10 ,h y10 ,h z10 ), wrist joint point w coordinate (w x1 ,w y1 ,w z1 ), elbow joint point e coordinate (e x1 ,e y1 ,e z1 ) and shoulder joint coordinates s:(s x1 ,s y1 ,s z1 ), all the nodes form a label list Joints, all the nodes are connected to form a skeleton Limb, forming a skeleton label list; when a high-definition camera does not detect a joint point, the coordinate value is set to 0; the real position of each joint point of the traffic police in the three-dimensional space is calculated by triangulation method; as follows: Joints = [(h x1 ,h y1 ,h z1 )…(h xk ,h yk ,h zk ),(w x1 ,w y1 ,w z1 ),(e x1 ,e y1 ,e z1 ),(s x1 ,s y1 ,s z1 )]

[0062] Limb=[(h1,h2),(h3,h4),(h5,h6),(h7,h8),(h9,h 10 ),(h2,w),(h4,w),(h6,w),(h8,w),(h 10 ,w),(w,e),(e,s), where w is the coordinate of the wrist joint; e is the coordinate of the elbow joint; s is the coordinate of the shoulder joint;

[0063] d) Mark the traffic police’s hand gestures;

[0064] According to the different gestures of actual traffic police, different gestures are classified and labeled for training different gesture recognition.

[0065] Step 3: Determine the target of the roadside traffic police and train the gesture recognition algorithm;

[0066] The specific method is as follows:

[0067] a) Using the high-resolution output neural network model HRNet as the backbone network for feature extraction to train 2D joint point detection for each high-definition camera. The backbone network includes but is not limited to the HRNet network and can be replaced according to specific needs;

[0068] b) The same joint point from different high-definition cameras is calculated through triangulation to restore the 3D joint point information of the traffic police's arm in space; the training data set of the vehicle-side and road-side algorithms is the same, but the number of high-definition cameras is different; the training process is to improve the detection accuracy of the 3D joint point position, and the loss function Loss when training 3D joint point recovery is defined as the detection value k of all related nodes d and the truth value k gt That is, the 2 norm of the annotation distance error. This loss function is also applicable to the training of the vehicle-side binocular camera detecting the traffic police joint points when there is no blind spot in the vehicle-side detection field of view. The specific definition of Loss is:

[0069]

[0070] Among them, k d Detect the coordinate values ​​of all relevant node spaces; k gt It is the real coordinate value of all related nodes in space.

[0071] Step 4: Vehicle-side distance measurement and vehicle-side road-side information fusion, and recognition of traffic police gestures in congested intersection scenarios.

[0072] The specific method is as follows:

[0073] After the calibration and algorithm training of the vehicle-side and road-side high-definition cameras are completed, the reference device, which is the device used to match and associate the intersection space detected by the road-side high-definition camera with the intersection space detected by the vehicle-side binocular high-definition camera, is used as a medium. The rectangular area with a range of w×b×h and the target information inside detected by the road-side multi-view are taken as a whole and fused with the space and target information of the same area detected by the vehicle-side. b is the width of the intersection facing the vehicle, w is the width of the intersection road in the direction perpendicular to b, and h is the height of the high-definition camera at the top of the roadbed from the ground; the vehicle-side binocular camera detects that the actual distances between the vehicle and the two nearest roadbed reference devices are d1 and d2 respectively, and the x-axis of the vehicle-side binocular camera coordinate system is The horizontal angle of the x′ axis of the roadbed coordinate system at the intersection is α. The detection information at the roadside will be based on the origin O of the roadbed coordinate system, rotated around the z′ axis by an angle of α to the same direction as the x axis of the vehicle-side camera coordinate system, and then scaled and displayed in the on-board computer according to the scaling factor m. The on-board computer displays and broadcasts the presence of traffic police and the command prompt voice of the traffic police gesture in real time. The detection results are used as input and transmitted to the automatic driving path planning algorithm for driving intention and planning of the intersection driving path; at the same time, the driver can zoom in and out the details of the traffic police gesture information on the detected spatial information by touch; the specific calculation method of the horizontal angle α between the x axis of the vehicle-side binocular camera coordinate system and the x′ axis of the intersection roadbed coordinate system is:

[0074]

[0075] When the vehicle-side binocular camera is blocked by the vehicle in front or other obstacles, the distance detection can be directly transmitted to the vehicle-side.

[0076] When the vehicle-side binocular high-definition camera detects the traffic police and the traffic police gesture, the road-side detection space information is rotated by an angle of α to the same direction as the x-axis of the vehicle-side binocular high-definition camera coordinate system, and the Hungarian algorithm is used to match the same joint points. Considering the missing and error of the joint information detected by the vehicle-side binocular high-definition camera and the joint information restored by the road-side high-definition camera, the center point of the matched joint point is calculated as the position of the fused joint point;

[0077] In the extreme case where the traffic police are in plain clothes and it is impossible to detect whether the traffic police target exists through the traffic police uniform features, the target whose traffic control gesture exceeds the time period threshold t is defined as the traffic police.

[0078] Example

[0079] Taking the traffic police's stop gesture as an example, the gesture description is: the traffic police raises his left arm to the horizontal level, with his palm facing the vehicle coming from the opposite direction of the traffic police, and his fingers stretched upward. After the road-side high-definition camera and the vehicle-side high-definition binocular camera have been calibrated, the road-side high-definition camera is used to collect and annotate information about the traffic police, their gestures, and road vehicles, pedestrians and other obstacles. For traffic police gestures, for example, descriptions that match the stop gesture are annotated as stop gestures. In order to improve the robustness of the method, we will collect multiple and diverse intersection video information to create a data set during the data collection phase, and then train it. The trained method is embedded in the road-side and vehicle-side high-definition camera perception systems to jointly complete the traffic police gesture recognition in congested intersection scenarios.

[0080] In summary, the present invention can effectively cope with the real-time recognition and judgment of traffic police gestures when the driver has a blind spot in the field of vision of the traffic police in the congested intersection scenario, and can improve the work efficiency and traffic safety of the autonomous driving traffic system.

[0081] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for traffic police gesture recognition in a congested intersection scenario, characterized in that: The following steps are involved: Step 1: Arrangement and calibration of roadside HD cameras and vehicle-side HD cameras; Step 2: Collect, label and create a data set of traffic police targets and gesture feature data at the intersection; Step 3: Determine the target of the roadside traffic police and train the gesture recognition algorithm; Step 4: Vehicle-side distance measurement and vehicle-side road-side information fusion, and recognition of traffic police gestures at congested intersections; Step one includes: At an intersection of scale w×b, where b is the intersection width facing the vehicle and w is the intersection road width in the direction perpendicular to b, the turning point of each road edge is set as the reference point, and a roadbed of height h is placed, on which n roadside high-definition cameras are placed, where n≥4, and at least one high-definition camera is placed on the top of the road at a height h from the ground. Each of the roadside high-definition cameras detects synchronized high-definition video information in the intersection road from different directions, including traffic police, vehicles and pedestrians. The video content detected by all high-definition cameras should include all information in the vertical space of the intersection; The vehicle-side binocular high-definition camera is formed by combining two high-definition monocular cameras; the binocular high-definition camera is used by the vehicle-side to perceive the target information on the road ahead; At each roadbed height h0, a conspicuous sign that is different from a traffic sign is set as a reference device for matching and associating the intersection space detected by the road-side high-definition camera with the intersection space detected by the vehicle-side binocular high-definition camera; Step 2 includes: Use roadside HD cameras to shoot synchronized HD videos from different perspectives at intersections where vehicles and pedestrians are congested and traffic lights are working properly; Step three includes: The high-resolution output neural network model HRNet is used as the backbone network for feature extraction to train 2D joint point detection for each HD camera; The same joint point from different high-definition cameras is calculated through triangulation to restore the 3D joint point information of the traffic police's arm in space; Step 4 includes: Using the reference device as a medium, the rectangular area with a range of w×b×h and the target information inside it detected by the road-side multi-view are taken as a whole and fused with the spatial and target information of the same area detected by the vehicle-side; When the vehicle-side binocular camera is blocked by the vehicle in front or other obstacles, the distance detection can be directly transmitted to the vehicle-side. When the vehicle-side binocular high-definition camera detects the traffic police and the traffic police gesture, the road-side detection space information is rotated by an angle of α to the same direction as the x-axis of the vehicle-side binocular high-definition camera coordinate system, and the Hungarian algorithm is used to match the same joint points. Considering the missing and error of the joint information detected by the vehicle-side binocular high-definition camera and the joint information restored by the road-side high-definition camera, the center point of the matched joint point is calculated as the position of the fused joint point; In the extreme case where there is a traffic police in plain clothes, the target who makes a traffic control gesture for more than the time period threshold t is defined as a traffic police.

2. The method for traffic police gesture recognition in a congested intersection scenario according to claim 1 is characterized in that: The step one also includes: The high-definition cameras are individually calibrated using Zhang Youzheng's camera calibration method, and all high-definition cameras are jointly calibrated using an epipolar geometry constraint method to accurately identify and locate targets; The binocular high-definition camera uses Zhang Youzheng's camera calibration method to calibrate the high-definition camera separately, and uses the epipolar geometry constraint method to calibrate the high-definition camera so as to accurately identify and locate the target.

3. The method for traffic police gesture recognition in a congested intersection scenario according to claim 1 is characterized in that: The step 2 also includes: The high-definition video is used as a training data set; the training data set annotation benchmark is as follows: a) Define the vehicle coordinate system; The vehicle's forward direction is the y-axis, the right direction perpendicular to the y-axis is the x-axis, and the upward direction perpendicular to the x- and y-axis planes is the z-axis; the roadbed coordinate system is defined as: the right direction of the intersection facing the vehicle is the x'-axis, the direction perpendicular to the x'-axis and consistent with the direction of the vehicle's x-axis is the y'-axis, and the roadbed direction facing the vehicle and closest to the left side perpendicular to the ground is the z'-axis direction; b) Mark the traffic police category; The categories for marking are: traffic police, traffic lights, vehicles, pedestrians and obstacles; When a traffic policeman is present, the priority of the traffic policeman is higher than that of the traffic lights. Before recognizing the traffic policeman’s gestures, the traffic policeman’s target must be correctly identified. The traffic policeman in uniform must be marked as a traffic policeman according to the standard of whether the traffic policeman has uniform features. The upper garment has a higher priority than the hat. At the same time, the direction of the traffic policeman must be marked. The direction of the traffic policeman and the gesture jointly determine the direction of command. c) Label the joints and bones of the traffic police’s arms; The traffic police's traffic control gestures are completed by both arms, that is, only 10 finger joints, 1 wrist joint, 1 elbow joint and 1 shoulder joint of each arm of the traffic police are marked. Each joint point detected by the road-side high-definition camera or the vehicle-side binocular high-definition camera contains three-dimensional coordinate information. The finger joint points (h1…h 10 ) coordinates (h x1 ,h y1 ,h z1 )…(h x10 ,h y10 ,h z10 ), wrist joint point w coordinate (w x1 ,w y1 ,w z1 ), elbow joint point e coordinate (e x1 ,e y1 ,e z1 ) and shoulder joint coordinates s:(s x1 ,s y1 ,s z1 ), all the nodes form a label list Joints, all the nodes are connected to form a skeleton Limb, forming a skeleton label list; when a high-definition camera does not detect a joint point, the coordinate value is set to 0; the real position of each joint point of the traffic police in the three-dimensional space is calculated by triangulation method; as follows: Joints = [(h x1 ,h y1 ,h z1 )…(h xk ,h yk ,h zk ),(w x1 ,w y1 ,w z1 ),(e x1 ,e y1 ,e z1 ),(s x1 ,s y1 ,s z1 )] Limb=[(h1,h2),(h3,h4),(h5,h6),(h7,h8),(h9,h 10 ),(h2,w),(h4,w),(h6,w),(h8,w),(h 10 ,w),(w,e),(e,s), where w is the coordinate of the wrist joint; e is the coordinate of the elbow joint; s is the coordinate of the shoulder joint; d) Mark the traffic police’s hand gestures; According to the different gestures of actual traffic police, different gestures are classified and labeled for training different gesture recognition.

4. The method for traffic police gesture recognition in a congested intersection scenario according to claim 1, characterized in that: Step three also includes: The training data sets of the vehicle-side and road-side algorithms are the same, except for the number of high-definition cameras. In order to improve the detection accuracy of the 3D joint point positions during the training process, the loss function Loss when training 3D joint point recovery is defined as the detection value k of all related nodes. d and the truth value k gt That is, the 2 norm of the annotation distance error. This loss function is also applicable to the training of the vehicle-side binocular camera detecting the traffic police joint points when there is no blind spot in the vehicle-side detection field of view. The specific definition of Loss is: Among them, k d Detect the coordinate values ​​of all relevant node spaces; k gt It is the real coordinate value of all related nodes in space.

5. The method for traffic police gesture recognition in a congested intersection scenario according to claim 1, characterized in that: Step 4 also includes: The vehicle-side binocular camera detects that the actual distances between the vehicle-side and the two nearest roadbed reference devices are d1 and d2 respectively. The horizontal angle between the x-axis of the vehicle-side binocular camera coordinate system and the x′-axis of the intersection roadbed coordinate system is α. The detection information of the roadside will be based on the origin O of the roadbed coordinate system, rotated around the z′-axis by an angle of α to the same direction as the x-axis of the vehicle-side camera coordinate system, and then scaled according to the scaling factor m and displayed in the on-board computer. The on-board computer displays and broadcasts in real time whether there is a traffic police and the command prompt voice of the traffic police gesture. The detection results are used as input and transmitted to the automatic driving path planning algorithm for driving intention and intersection driving path planning; at the same time, the driver can zoom in and out the details of the traffic police gesture information on the detected spatial information by touch; the specific calculation method of the horizontal angle α between the x-axis of the vehicle-side binocular camera coordinate system and the x′-axis of the intersection roadbed coordinate system is: Where h0 is the height of the roadbed reference device from the ground.

Images