A method and system for identifying non-motorized vehicles and pedestrians in crowded scenarios

Abstract

This invention relates to a method and system for identifying non-motorized vehicles and pedestrians in congested traffic scenarios, belonging to the field of intelligent traffic management technology. To address the problem of accurate identification of non-motorized vehicles and pedestrians in congested traffic conditions, this invention collects image data from traffic cameras at traffic intersections. It then uses a target detection model to process the image data, detecting pedestrian and non-motorized vehicle targets to obtain bounding boxes for both. Next, it determines whether the image data of the traffic intersection belongs to a congested scene, obtaining congested scene image data. Key points for pedestrians and non-motorized vehicles are extracted from the congested scene image data. These key points are then correlated to identify the associated pedestrian and vehicle targets, obtaining the associated pedestrian and vehicle targets. Finally, image tracking technology is used to identify whether the associated pedestrian and vehicle targets exhibit traffic violations. If so, the identity information of the pedestrian targets is identified for traffic violation warnings and processing.

Description

Technical Field

[0001] This invention belongs to the field of intelligent traffic management technology, specifically relating to a method and system for identifying non-motorized vehicles and pedestrians in congested scenarios. Background Technology

[0002] In the field of intelligent traffic management, accurate association between non-motorized vehicles and cyclists is a key technology for achieving traffic accident risk warning. Current technologies for association recognition in mixed pedestrian and non-motorized vehicle scenarios mainly utilize deep learning models to extract the visual features of pedestrians and non-motorized vehicles, and achieve association through feature distance calculation. However, in scenarios with drastic changes in lighting, partial occlusion of targets, and multiple targets mixed and interspersed, visual features are prone to significant drift, leading to association failure. Therefore, in high-density scenarios with mixed pedestrian and non-motorized vehicles, the accuracy of human-vehicle association is low. Summary of the Invention

[0003] The problem to be solved by this invention is the accurate identification of non-motorized vehicles and pedestrians in crowded scenarios, and a method and system for identifying non-motorized vehicles and pedestrians in crowded scenarios are proposed.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A method for identifying non-motorized vehicles and pedestrians in crowded scenarios includes the following steps:

[0006] S1. Collect image data of traffic intersections through traffic cameras, and use a target detection model to perform target detection processing on the image data to detect personnel targets and non-motorized vehicle targets, and obtain personnel target boxes and non-motorized vehicle target boxes;

[0007] S2. Based on the distribution density of the person target box and the non-motor vehicle target box obtained in step S1, determine whether the scene category of the traffic intersection image data is a congested scene, and obtain the congested scene image data;

[0008] S3. Extract key points of people and non-motorized vehicles from the crowded scene image data obtained in step S2;

[0009] S4. Associate the personnel key points and non-motorized vehicle key points obtained in step S3, determine the personnel key points associated with the non-motorized vehicle key points, and obtain the personnel-vehicle association target;

[0010] S5. Use image tracking technology to identify whether the human-vehicle related targets obtained in step S4 have traffic violations. If so, identify the identity information of the human targets for traffic violation warning and handling.

[0011] Furthermore, the method for training the object detection model in step S1 includes the following steps:

[0012] S1.1. Input the sample image of the traffic intersection into the target detection model to be trained. The target detection model to be trained outputs the detection results, which are the target boxes for people and non-motorized vehicles.

[0013] S1.2. Perform a first error calculation between the person target bounding box detected in step S1.1 and the person bounding box labeled in the sample image to obtain a first error value;

[0014] The second error value is obtained by performing a second error calculation between the non-motorized vehicle target box detected in step S1.1 and the non-motorized vehicle target box labeled in the sample image;

[0015] The total error value is calculated based on the first error value and the second error value, expressed as follows:

[0016]

[0017] Where LOSS1 is the first error value and LOSS2 is the second error value; the first error value and the second error value are calculated using the cross-entropy loss function or the squared difference loss function; The weight of the first error value, The weight of the second error value;

[0018] S1.3. With the goal of minimizing the total error value, backpropagation is performed using the total error value to adjust the model parameters in the target detection model to be trained, and the adjustment process of the model parameters is iterated until the total error value reaches the preset value or the number of iterations reaches the preset number, at which point training stops and a trained target detection model is obtained.

[0019] Furthermore, the method for calculating the weights of the first error value and the second error value in step S1.2 is as follows:

[0020] ;

[0021] ;

[0022]

[0023]

[0024] in, The number of bounding boxes for people in the sample image. This refers to the number of non-motorized vehicle frames marked in the sample image. This refers to the total number of bounding boxes for people and bounding boxes for non-motorized vehicles in the sample image. ; For the sample image belonging to The first person in the labeled personnel box Individual frame, For the sample image belonging to The first person in the labeled personnel box Individual personnel frame; For the sample image belonging to The first in all the marked boxes A box, For the sample image belonging to The first in all the marked boxes One box; For the sample image belonging to The first one in the marked non-motorized vehicle box One non-motorized vehicle frame, For the sample image belonging to The first one in the marked non-motorized vehicle box One non-motorized vehicle frame; among them , This indicates the intersection-union ratio between two boxes. This indicates the center distance between the two boxes; and As an intermediate coefficient, This indicates taking the maximum number.

[0025] Furthermore, the method for determining whether the scene category of the traffic intersection image data in step S2 is a congested scene is to cluster the target boxes based on the center position of the target boxes to obtain clusters of target boxes. If the number of target boxes in the cluster with the most target boxes is greater than a preset value, and the average IOU between two target boxes in the cluster is greater than a preset value, then the scene category of the traffic intersection image data is determined to be a congested scene.

[0026] Furthermore, in step S3, the key points for personnel include wrist joints, elbow joints, knee joints, and ankle joints; and the key points for non-motorized vehicles include handlebar key points and pedal key points.

[0027] Furthermore, in step S4, the key points of personnel and non-motorized vehicles are associated based on the prior distance between the key points of personnel and non-motorized vehicles.

[0028] The distance prior is set as the distance constraint between different personnel key points and non-motorized vehicle key points, including the following constraints:

[0029] The distance between the wrist joint and the handlebar key point is less than the first distance value; the distance between the elbow joint and the handlebar key point is less than the second distance value and greater than the third distance value; the third distance value is less than the second distance value; and the third distance value is greater than or equal to the first distance value.

[0030] The distance between the ankle joint and the pedal key point is less than the fourth distance value; the distance between the knee joint and the pedal key point is less than the fifth distance value and greater than the sixth distance value; the sixth distance value is less than the fifth distance value; and the sixth distance value is greater than or equal to the fourth distance value.

[0031] Furthermore, in step S4, the association of personnel key points and non-motorized vehicle key points obtained in step S3 includes associating one non-motorized vehicle target with one personnel target, and associating multiple non-motorized vehicle targets and multiple personnel targets separately.

[0032] Furthermore, the method for associating multiple non-motorized vehicle targets and multiple pedestrian targets in step S4 includes the following steps:

[0033] S4.1. Sort the key point coordinates corresponding to multiple human targets, non-motorized vehicle targets and parallel non-motorized vehicle targets to obtain a key point sequence. The sorting direction is that the key points closer to the camera are in front and the key points farther away from the camera are in back. The key point sequence is calibrated by the drift calibration vector corresponding to the camera to obtain a calibrated key point sequence.

[0034] S4.2. In the calibrated key point sequence, determine the candidate key point that is closest to the non-motorized vehicle key point. For target candidate key points that belong to the same non-motorized vehicle target, determine the personnel target with the most unique personal identifiers as the personnel target associated with the non-motorized vehicle target.

[0035] If the candidate key points belonging to multiple personnel targets fail to match with the parallel non-motorized vehicle targets, then the attitude of the multiple personnel targets is estimated based on the candidate key points of the multiple personnel targets. The attitude estimation results include whether it is a riding posture and the confidence of the riding posture. The personnel target with the riding posture type and the highest riding posture confidence is determined as the personnel target associated with the non-motorized vehicle target.

[0036] A non-motorized vehicle and pedestrian recognition system for congested scenarios includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed, it implements the steps of the non-motorized vehicle and pedestrian recognition method for congested scenarios.

[0037] The beneficial effects of this invention are:

[0038] The present invention provides a method for identifying non-motorized vehicles in crowded scenarios, which can effectively solve the problem in the prior art that, in scenarios where pedestrians and non-motorized vehicles are mixed, visual features are prone to significant drift under conditions such as drastic changes in lighting, partial occlusion of targets, and multiple targets mixed and interspersed, leading to association failure and low accuracy of human-vehicle association in high-density scenarios where pedestrians and non-motorized vehicles are mixed. Attached Figure Description

[0039] Figure 1 This is a flowchart of a non-motorized vehicle and pedestrian recognition method in a crowded scenario according to the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described specific embodiments are merely a part of the embodiments of the invention, and not all of them. The components of the specific embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations, and the invention may also have other embodiments.

[0041] Therefore, the following detailed description of specific embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected specific embodiments of the invention. All other specific embodiments obtained by those skilled in the art based on these specific embodiments without inventive effort are within the scope of protection of this invention.

[0042] To further understand the invention's content, features, and effects, the following specific embodiments are provided, along with accompanying drawings. Figure 1 Detailed explanation is as follows:

[0043] Example 1:

[0044] A method for identifying non-motorized vehicles and pedestrians in crowded scenarios includes the following steps:

[0045] S1. Collect image data of traffic intersections through traffic cameras, and use a target detection model to perform target detection processing on the image data to detect personnel targets and non-motorized vehicle targets, and obtain personnel target boxes and non-motorized vehicle target boxes;

[0046] Furthermore, the method for training the object detection model in step S1 includes the following steps:

[0047] S1.1. Input the sample image of the traffic intersection into the target detection model to be trained. The target detection model to be trained outputs the detection results, which are the target boxes for people and non-motorized vehicles.

[0048] S1.2. Perform a first error calculation between the person target bounding box detected in step S1.1 and the person bounding box labeled in the sample image to obtain a first error value;

[0049] The second error value is obtained by performing a second error calculation between the non-motorized vehicle target box detected in step S1.1 and the non-motorized vehicle target box labeled in the sample image;

[0050] The total error value is calculated based on the first error value and the second error value, expressed as follows:

[0051]

[0052] Where LOSS1 is the first error value and LOSS2 is the second error value; the first error value and the second error value are calculated using the cross-entropy loss function or the squared difference loss function; The weight of the first error value, The weight of the second error value;

[0053] Furthermore, the method for calculating the weights of the first error value and the second error value in step S1.2 is as follows:

[0054] ;

[0055] ;

[0056]

[0057]

[0058] in, The number of bounding boxes for people in the sample image. This refers to the number of non-motorized vehicle frames marked in the sample image. This refers to the total number of bounding boxes for people and bounding boxes for non-motorized vehicles in the sample image. ; For the sample image belonging to The first person in the labeled personnel box Individual frame, For the sample image belonging to The first person in the labeled personnel box Individual personnel frame; For the sample image belonging to The first in all the marked boxes A box, For the sample image belonging to The first in all the marked boxes One box; For the sample image belonging to The first one in the marked non-motorized vehicle box One non-motorized vehicle frame, For the sample image belonging to The first one in the marked non-motorized vehicle box One non-motorized vehicle frame; among them , This indicates the intersection-union ratio between two boxes. This indicates the center distance between the two boxes; and As an intermediate coefficient, This indicates taking the maximum number.

[0059] S1.3. With the goal of minimizing the total error value, backpropagation is performed using the total error value to adjust the model parameters in the target detection model to be trained, and the adjustment process of the model parameters is iterated until the total error value reaches the preset value or the number of iterations reaches the preset number, at which point training stops and a trained target detection model is obtained.

[0060] Furthermore, the object detection model can be a Convolutional Neural Network (CNN), such as any one of LeNet, AlexNet, VGGNet, R-CNN, Fast R-CNN, Faster R-CNN, Mask R-CNN, YOLO, or SSD. To improve the accuracy of the object detection model in detecting people and non-motorized vehicles, a dynamic error algorithm is used to calculate the total error value. During training, the weights corresponding to the first and second error values ​​are determined based on the labeled person and non-motorized vehicle bounding boxes in the sample images. That is, for each sample image, based on the distribution of the labeled person and non-motorized vehicle bounding boxes in the sample image, there will be a corresponding set of weights used to calculate the total error value corresponding to the first and second error values. This set of weights can be associated with the sample images in the form of labels. Specifically, the weights corresponding to the first and second error values ​​can be determined based on the number, overlap, and distance of the labeled person and non-motorized vehicle bounding boxes.

[0061] After obtaining the trained object detection model, image data is input into the model for detection, and the corresponding person and non-motorized vehicle bounding boxes are output. There are multiple person bounding boxes and at least one non-motorized vehicle bounding box. If no non-motorized vehicle is detected, the non-motorized vehicle-person association process will not be performed.

[0062] S2. Based on the distribution density of the person target box and the non-motor vehicle target box obtained in step S1, determine whether the scene category of the traffic intersection image data is a congested scene, and obtain the congested scene image data;

[0063] After obtaining the bounding boxes for people and non-motorized vehicles corresponding to the image data, the distribution density of the bounding boxes can be used to determine whether the scene category to which the image data belongs is a crowded scene. If so, it indicates that people and non-motorized vehicles are in a complex scene, and the accuracy of human-vehicle association recognition needs to be improved. The distribution density of the bounding boxes can be determined based on the center position of the bounding boxes, the number of bounding boxes, and the degree of overlap between the bounding boxes.

[0064] Furthermore, the method for determining whether the scene category of the traffic intersection image data in step S2 is a congested scene is to cluster the target boxes based on the center position of the target boxes to obtain clusters of target boxes. If the number of target boxes in the cluster with the most target boxes is greater than a preset value, and the average IOU between two target boxes in the cluster is greater than a preset value, then the scene category of the traffic intersection image data is determined to be a congested scene.

[0065] Furthermore, the center points of the bounding boxes for people and non-motorized vehicles are extracted. These center points are then used to construct a Delaunay triangle network. In a Delaunay triangle network, no other points exist within the circumcircle of any triangle. The vertices of the triangles are the center points of the bounding boxes. The area of ​​each triangle is calculated. If the number of triangles with areas smaller than a preset number is greater than a preset number of triangles, or if the proportion of triangles with areas smaller than a preset number reaches a preset proportion, then the scene category to which the image data belongs can be determined as a congested scene. This method allows for a more accurate determination of whether the scene category to which the image data belongs is a congested scene, providing reliable image data for subsequent human-vehicle association.

[0066] S3. Extract key points of people and non-motorized vehicles from the crowded scene image data obtained in step S2;

[0067] Furthermore, in step S3, the key points for personnel include wrist joints, elbow joints, knee joints, and ankle joints; and the key points for non-motorized vehicles include handlebar key points and pedal key points.

[0068] Furthermore, key points for people are associated with the corresponding person targets in the image data through unique person identifiers, and key points for non-motorized vehicles are associated with the corresponding non-motorized vehicle targets in the image data through unique non-motorized vehicle identifiers. When extracting key points for people, the image data within the person target bounding box can be used to extract key points for people. Similarly, when extracting key points for non-motorized vehicles, the image data within the non-motorized vehicle target bounding box can be used to extract key points for non-motorized vehicles.

[0069] More specifically, for a non-motorized vehicle target box, the personnel target boxes that intersect with the non-motorized vehicle target box can be identified as the personnel target boxes to be extracted. Personnel key points are extracted from the personnel target boxes to be extracted. In this way, it is not necessary to extract key points from personnel target boxes that are far away from non-motorized vehicles, which can reduce the computational load of personnel key point extraction.

[0070] S4. Associate the personnel key points and non-motorized vehicle key points obtained in step S3, determine the personnel key points associated with the non-motorized vehicle key points, and obtain the personnel-vehicle association target;

[0071] Furthermore, in step S4, the key points of personnel and non-motorized vehicles are associated based on the prior distance between the key points of personnel and non-motorized vehicles.

[0072] The distance prior is set as the distance constraint between different personnel key points and non-motorized vehicle key points, including the following constraints:

[0073] The distance between the wrist joint and the handlebar key point is less than the first distance value; the distance between the elbow joint and the handlebar key point is less than the second distance value and greater than the third distance value; the third distance value is less than the second distance value; and the third distance value is greater than or equal to the first distance value.

[0074] The distance between the ankle joint and the pedal key point is less than the fourth distance value; the distance between the knee joint and the pedal key point is less than the fifth distance value and greater than the sixth distance value; the sixth distance value is less than the fifth distance value; and the sixth distance value is greater than or equal to the fourth distance value.

[0075] Furthermore, the first distance value is set to 5cm, the second distance value is set to 30cm, and the third distance value is set to 5cm;

[0076] Furthermore, the fourth distance value is set to 3cm, the fifth distance value is set to 30cm, and the sixth distance value is set to 15cm;

[0077] Furthermore, in step S4, the association of personnel key points and non-motorized vehicle key points obtained in step S3 includes associating one non-motorized vehicle target with one personnel target, and associating multiple non-motorized vehicle targets and multiple personnel targets separately.

[0078] Furthermore, for a non-motorized vehicle target, the key points of personnel and non-motorized vehicles can be matched based on the prior distance between the key points of personnel and non-motorized vehicles. The matched key points of personnel are used as candidate key points. Based on the unique identifier of the personnel corresponding to the candidate key points, if the candidate key points belong to the same personnel target, the personnel target can be associated with the non-motorized vehicle target.

[0079] Specifically, for a non-motorized vehicle target and a human target, the key points corresponding to the human target and the key points corresponding to the non-motorized vehicle target are calculated based on distance priors. For each non-motorized vehicle key point, there is at least one corresponding human key point with a distance prior constraint. The distance between the human key point and the non-motorized vehicle key point is calculated. If the distances between all human key points and the non-motorized vehicle key point satisfy the distance prior constraint, the match is considered successful. If the distances between some human key points and the non-motorized vehicle key point satisfy the distance prior constraint, while the distances between other human key points and the non-motorized vehicle key point do not satisfy the distance prior constraint, then it is further determined whether there are any human key points in the other part of the human key points that are closer to the non-motorized vehicle key point in terms of distance prior. If they exist, the match is considered unsuccessful; if not, it is further determined whether there are any human key points in the other part of the human key points that are closer to the non-motorized vehicle key point in terms of distance prior. If they exist, the match is considered successful; if not, the match is considered unsuccessful. Alternatively, we can first determine whether there are any personnel key points in a group of personnel key points that are prior to the non-motorized vehicle key points. If not, the matching is determined to be unsuccessful. If they exist, we can further determine whether there are any personnel key points in another group of personnel key points that are prior to the non-motorized vehicle key points. If they exist, the matching is determined to be unsuccessful; if not, the matching is determined to be unsuccessful.

[0080] Furthermore, the method for associating multiple non-motorized vehicle targets and multiple pedestrian targets in step S4 includes the following steps:

[0081] S4.1. Sort the key point coordinates corresponding to multiple human targets, non-motorized vehicle targets and parallel non-motorized vehicle targets to obtain a key point sequence. The sorting direction is that the key points closer to the camera are in front and the key points farther away from the camera are in back. The key point sequence is calibrated by the drift calibration vector corresponding to the camera to obtain a calibrated key point sequence.

[0082] S4.2. In the calibrated key point sequence, determine the candidate key point that is closest to the non-motorized vehicle key point. For target candidate key points that belong to the same non-motorized vehicle target, determine the personnel target with the most unique personal identifiers as the personnel target associated with the non-motorized vehicle target.

[0083] If the candidate key points belonging to multiple personnel targets fail to match with the parallel non-motorized vehicle targets, then the attitude of the multiple personnel targets is estimated based on the candidate key points of the multiple personnel targets. The attitude estimation results include whether it is a riding posture and the confidence of the riding posture. The personnel target with the riding posture type and the highest riding posture confidence is determined as the personnel target associated with the non-motorized vehicle target.

[0084] Furthermore, by matching non-motorized vehicle targets and individual personnel targets separately, the successfully matched personnel key points are used as candidate key points. If multiple personnel targets have successfully matched personnel key points, meaning the candidate key points belong to multiple personnel targets, then it is first determined whether there are parallel non-motorized vehicle targets. If there are parallel non-motorized vehicle targets, it is determined whether the candidate key points belonging to multiple personnel targets also match the parallel non-motorized vehicle targets. If the candidate key points belonging to multiple personnel targets also match the parallel non-motorized vehicle targets, then the coordinates of the key points corresponding to the multiple personnel targets, non-motorized vehicle targets, and their parallel non-motorized vehicle targets are sorted.

[0085] Furthermore, the drift calibration vector is trained based on the camera's orientation and the labeled congestion image. Specifically, the labeled congestion image includes sample human targets and sample non-motorized vehicle targets with pre-defined relationships. First, human key points and non-motorized vehicle key points are extracted from the congestion image using the key point extraction method described above. The extracted sample human key points and sample non-motorized vehicle key points are arranged from near to far according to the camera's orientation, resulting in a sample key point sequence. Then, based on the labeled sample personnel targets and sample non-motorized vehicle targets with pre-defined associations, the extracted sample personnel key points are arranged according to their associations to obtain a reference key point sequence. An initial calibration vector is constructed, and the initial calibration vector is multiplied with the sample key point sequence to obtain a calibrated key point sequence. The error between the calibrated key point sequence and the reference key point sequence is calculated to obtain the error value between the calibrated key point sequence and the reference key point sequence. The goal is to minimize the error value. The initial calibration vector is numerically adjusted using a backpropagation algorithm. The above numerical adjustment process is iterated until the error value between the calibrated key point sequence and the reference key point sequence is less than a preset value or reaches a preset number of iterations. The iteration is then stopped, and the finally adjusted calibration vector is determined as the drift calibration vector.

[0086] S5. Use image tracking technology to identify whether the human-vehicle related targets obtained in step S4 have traffic violations. If so, identify the identity information of the human targets for traffic violation warning and handling.

[0087] Example 2:

[0088] A non-motorized vehicle and pedestrian recognition system for congested scenarios includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed, it implements the steps of the non-motorized vehicle and pedestrian recognition method for congested scenarios as described in Embodiment 1.

[0089] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0090] Although this application has been described above with reference to specific embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of this application. In particular, as long as there is no structural conflict, the features in the specific embodiments disclosed in this application can be combined with each other in any way. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, this application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for identifying non-motorized vehicles and pedestrians in congested scenarios, characterized in that, Includes the following steps: S1. Collect image data of traffic intersections through traffic cameras, and use a target detection model to perform target detection processing on the image data to detect personnel targets and non-motorized vehicle targets, and obtain personnel target boxes and non-motorized vehicle target boxes; S2. Based on the distribution density of the person target box and the non-motor vehicle target box obtained in step S1, determine whether the scene category of the traffic intersection image data is a congested scene, and obtain the congested scene image data; S3. Extract key points of people and non-motorized vehicles from the crowded scene image data obtained in step S2; S4. Associate the personnel key points and non-motorized vehicle key points obtained in step S3, determine the personnel key points associated with the non-motorized vehicle key points, and obtain the personnel-vehicle association target; In step S4, the key points of personnel and non-motorized vehicles are associated based on the prior distance between the key points of personnel and non-motorized vehicles. The distance prior is set as the distance constraint between different personnel key points and non-motorized vehicle key points, including the following constraints: The distance between the wrist joint and the handlebar key point is less than the first distance value; the distance between the elbow joint and the handlebar key point is less than the second distance value and greater than the third distance value; the third distance value is less than the second distance value; and the third distance value is greater than or equal to the first distance value. The distance between the ankle joint and the pedal key point is less than the fourth distance value; the distance between the knee joint and the pedal key point is less than the fifth distance value and greater than the sixth distance value; the sixth distance value is less than the fifth distance value; and the sixth distance value is greater than or equal to the fourth distance value. Step S4 involves associating the personnel key points and non-motorized vehicle key points obtained in step S3, including associating one non-motorized vehicle target with one personnel target, and associating multiple non-motorized vehicle targets and multiple personnel targets separately. The method for associating multiple non-motorized vehicle targets and multiple pedestrian targets in step S4 includes the following steps: S4.

1. Sort the key point coordinates corresponding to multiple human targets, non-motorized vehicle targets and parallel non-motorized vehicle targets to obtain a key point sequence. The sorting direction is that the key points closer to the camera are in front and the key points farther away from the camera are in back. The key point sequence is calibrated by the drift calibration vector corresponding to the camera to obtain a calibrated key point sequence. S4.

2. In the calibrated key point sequence, determine the candidate key point that is closest to the non-motorized vehicle key point. For target candidate key points that belong to the same non-motorized vehicle target, determine the personnel target with the most unique personal identifiers as the personnel target associated with the non-motorized vehicle target. If the candidate key points belonging to multiple personnel targets fail to match with the parallel non-motorized vehicle targets, then the attitude of multiple personnel targets is estimated based on the candidate key points of multiple personnel targets. The attitude estimation results include whether it is a riding attitude and the riding attitude confidence. The personnel target with the riding attitude type and the highest riding attitude confidence is determined as the personnel target associated with the non-motorized vehicle target. S5. Use image tracking technology to identify whether the human-vehicle related targets obtained in step S4 have traffic violations. If so, identify the identity information of the human targets for traffic violation warning and handling.

2. The method for identifying non-motorized vehicles and pedestrians in congested scenarios according to claim 1, characterized in that, Step S1, training the object detection model, includes the following steps: S1.

1. Input the sample image of the traffic intersection into the target detection model to be trained. The target detection model to be trained outputs the detection results, which are the target boxes for people and non-motorized vehicles. S1.

2. Perform a first error calculation between the person target bounding box detected in step S1.1 and the person bounding box labeled in the sample image to obtain a first error value; The second error value is obtained by performing a second error calculation between the non-motorized vehicle target box detected in step S1.1 and the non-motorized vehicle target box labeled in the sample image; The total error value is calculated based on the first error value and the second error value, expressed as follows: LOSS= LOSS1+ LOSS2; Where LOSS1 is the first error value and LOSS2 is the second error value; the first error value and the second error value are calculated using the cross-entropy loss function or the squared difference loss function; The weight of the first error value. The weight of the second error value; S1.

3. With the goal of minimizing the total error value, backpropagation is performed using the total error value to adjust the model parameters in the target detection model to be trained, and the adjustment process of the model parameters is iterated until the total error value reaches the preset value or the number of iterations reaches the preset number, at which point training stops and a trained target detection model is obtained.

3. The method for identifying non-motorized vehicles and pedestrians in congested scenarios according to claim 2, characterized in that, The method for calculating the weights of the first error value and the second error value in step S1.2 is as follows: ； ； ； ； in, The number of bounding boxes for people in the sample image. This refers to the number of non-motorized vehicle frames marked in the sample image. This refers to the total number of bounding boxes for people and bounding boxes for non-motorized vehicles in the sample image. ; For the sample image belonging to The first person in the labeled personnel box Individual frame, For the sample image belonging to The first person in the labeled personnel box Individual personnel frame; For the sample image belonging to The first in all the marked boxes A box, For the sample image belonging to The first in all the marked boxes One box; For the sample image belonging to The first one in the marked non-motorized vehicle box One non-motorized vehicle frame, For the sample image belonging to The first one in the marked non-motorized vehicle box One non-motorized vehicle frame; among them , This indicates the intersection-union ratio between two boxes. This indicates the center-to-center distance between the two boxes; and As an intermediate coefficient, This indicates taking the maximum number.

4. The method for identifying non-motorized vehicles and pedestrians in congested scenarios according to claim 3, characterized in that, The method for determining whether the scene category of the traffic intersection image data in step S2 is a congested scene is to cluster the target boxes based on the center position of the target boxes to obtain the target box clusters. If the number of target boxes in the cluster with the most target boxes is greater than a preset value, and the average IOU between two target boxes in the cluster is greater than a preset value, then the scene category of the traffic intersection image data is determined to be a congested scene.

5. The method for identifying non-motorized vehicles and pedestrians in congested scenarios according to claim 4, characterized in that, Step S3 sets key points for personnel, including wrist joint, elbow joint, knee joint, and ankle joint; and sets key points for non-motorized vehicles, including handlebar key points and pedal key points.

6. A non-motorized vehicle and pedestrian recognition system for congested scenarios, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed, it implements the steps of a non-motorized vehicle and pedestrian recognition method in a congested scenario as described in any one of claims 1-5.

Images