Non-motor vehicle, passenger and vehicle identification method and system in crowded scene

Abstract

The invention discloses a non-motor vehicle passenger-vehicle identification method and system in a crowded scene, and belongs to the technical field of intelligent traffic management. In order to solve the problem of accurate recognition of non-motor vehicles, persons and vehicles in a crowded scene, image data of a traffic intersection is collected through a traffic camera, target detection processing is performed on the image data by using a target detection model, and a person target and a non-motor vehicle target are detected to obtain a person target frame and a non-motor vehicle target frame; judging whether the scene category to which the image data of the traffic intersection belongs is a crowded scene or not to obtain crowded scene image data; personnel key points and non-motor vehicle key points in the crowded scene image data are extracted; associating the personnel key points with the non-motor vehicle key points to determine personnel key points associated with the non-motor vehicle key points to obtain a personnel-vehicle association target; and identifying whether the person-vehicle associated target has traffic violation behaviors or not through an image tracking technology, and if so, identifying the identity information of the person target for traffic violation early warning and processing.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent traffic management, and particularly relates to a non-motor vehicle and person identification method and system in a crowded scene. BACKGROUND

[0002] In the field of intelligent traffic management, accurate association of non-motor vehicles and riders is a key technology for realizing traffic accident risk early warning. In the prior art, for the association identification in a mixed scene of pedestrians and non-motor vehicles, a deep learning model is mainly used to extract visual features of pedestrians and non-motor vehicles, and association is realized through feature distance calculation. However, in scenes such as dramatic changes in light, partial occlusion of targets, and mixed insertion of multiple targets, visual features are prone to significant drift, resulting in failure of association. Therefore, in a high-density scene of mixed pedestrians and non-motor vehicles, the accuracy of person-vehicle association is low. SUMMARY

[0003] The problem to be solved by the application is the accurate identification of non-motor vehicles and persons in a crowded scene, and a non-motor vehicle and person identification method and system in a crowded scene are proposed.

[0004] To achieve the above object, the application realizes the following technical scheme:

[0005] A non-motor vehicle and person identification method in a crowded scene, comprising the following steps:

[0006] S1. Collecting image data of a traffic intersection through a traffic camera, and performing target detection processing on the image data by using a target detection model to detect personnel targets and non-motor vehicle targets, and obtaining personnel target frames and non-motor vehicle target frames;

[0007] S2. Judging whether the scene category to which the image data of the traffic intersection belongs is a crowded scene according to the distribution density of the personnel target frames and the non-motor vehicle target frames obtained in step S1, and obtaining crowded scene image data;

[0008] S3. Extracting personnel key points and non-motor vehicle key points in the crowded scene image data obtained in step S2;

[0009] S4. Associating the personnel key points and the non-motor vehicle key points obtained in step S3 to determine personnel key points associated with non-motor vehicle key points, and obtaining person-vehicle association targets;

[0010] S5. Identifying whether the person-vehicle association targets obtained in step S4 exist traffic violation behaviors through image tracking technology, and if so, identifying the identity information of the personnel targets for traffic violation early warning and processing.

[0011] Further, the method for training the target detection model in step S1 comprises the following steps:

[0012] S1.1. Input the sample image of the traffic intersection into the target detection model to be trained. The target detection model to be trained outputs the detection results, which are the target boxes for people and non-motorized vehicles.

[0013] S1.2. Perform a first error calculation between the person target bounding box detected in step S1.1 and the person bounding box labeled in the sample image to obtain a first error value;

[0014] The second error value is obtained by performing a second error calculation between the non-motorized vehicle target box detected in step S1.1 and the non-motorized vehicle target box labeled in the sample image;

[0015] The total error value is calculated based on the first error value and the second error value, expressed as follows:

[0016]

[0017] Where LOSS1 is the first error value and LOSS2 is the second error value; the first error value and the second error value are calculated using the cross-entropy loss function or the squared difference loss function; The weight of the first error value. The weight of the second error value;

[0018] S1.3. With the goal of minimizing the total error value, backpropagation is performed using the total error value to adjust the model parameters in the target detection model to be trained, and the adjustment process of the model parameters is iterated until the total error value reaches the preset value or the number of iterations reaches the preset number, at which point training stops and a trained target detection model is obtained.

[0019] Furthermore, the method for calculating the weights of the first error value and the second error value in step S1.2 is as follows: ; ;

[0020]

[0021]

[0022] in, The number of bounding boxes for people in the sample image. This refers to the number of non-motorized vehicle frames marked in the sample image. This refers to the total number of bounding boxes for people and bounding boxes for non-motorized vehicles in the sample image. ; For the sample image belonging to The first person in the labeled personnel box Individual frame, is the first person frame in the annotated person frames in the sample image, is the first person frame in the annotated person frames in the sample image, is the first person frame in the annotated person frames in the sample image, is the first person frame in the annotated person frames in the sample image, is the first person frame in the annotated person frames in the sample image, is the first person frame in the annotated person frames in the sample image, is the first person frame in the annotated person frames in the sample image, is the first person frame in the annotated person frames in the sample image, is the first person frame in the annotated person frames in the sample image, is the first non-motor vehicle frame in the annotated non-motor vehicle frames in the sample image, is the first non-motor vehicle frame in the annotated non-motor vehicle frames in the sample image, is the first non-motor vehicle frame in the annotated non-motor vehicle frames in the sample image, is the first non-motor vehicle frame in the annotated non-motor vehicle frames in the sample image, is the first non-motor vehicle frame in the annotated non-motor vehicle frames in the sample image, is the first non-motor vehicle frame in the annotated non-motor vehicle frames in the sample image, , represents the intersection over union between two frames, represents the center distance between two frames; and is the intermediate coefficient, represents the maximum number.

[0023] Further, the judgment method of whether the scene category to which the image data of the traffic intersection in step S2 belongs is a crowded scene is that the target frame is clustered according to the center position of the target frame to obtain a clustering cluster of the target frame, if the number of target frames in the clustering cluster with the largest number of target frames is greater than a preset number value, and the average value of the IOU between two target frames in the clustering cluster is greater than a preset value, it is judged that the scene category to which the image data of the traffic intersection belongs is a crowded scene.

[0024] Further, the person key points set in step S3 include wrist joint points, elbow joint points, knee joint points and ankle joint points; the non-motor vehicle key points set includes handlebar key points and pedal key points.

[0025] Further, in step S4, the distance prior between the person key points and the non-motor vehicle key points is used to associate the person key points and the non-motor vehicle key points;

[0026] The distance prior is set as the distance constraint between different person key points and non-motor vehicle key points, including the following constraints:

[0027] The distance between the wrist joint point and the handlebar key point is less than a first distance value, the distance between the elbow joint point and the handlebar key point is less than a second distance value and greater than a third distance value, the third distance value is less than the second distance value, and the third distance value is greater than or equal to the first distance value;

[0028] The distance between the ankle joint and the pedal key point is less than a fourth distance value, the distance between the knee joint and the pedal key point is less than a fifth distance value and greater than a sixth distance value, the sixth distance value is less than the fifth distance value, and the sixth distance value is greater than or equal to the fourth distance value.

[0029] Further, the association of the personnel key points and the non-motor vehicle key points in step S4 includes the association of one non-motor vehicle target and one personnel target, and the association of multiple non-motor vehicle targets and multiple personnel targets, respectively.

[0030] Further, the method for the association of the multiple non-motor vehicle targets and the multiple personnel targets in step S4 includes the following steps:

[0031] S4.1. Sort the key point coordinates of the multiple personnel targets and the non-motor vehicle targets and the parallel non-motor vehicle targets to obtain a key point sequence, and the sorting direction is that the key points close to the camera are in front and the key points far from the camera are in back; and calibrate the key point sequence by using the drift calibration vector corresponding to the camera to obtain a calibrated key point sequence.

[0032] S4.2. Determine the candidate key points closest to the non-motor vehicle key points in the calibrated key point sequence, and determine the personnel target with the most unique personnel identifier as the personnel target associated with the non-motor vehicle target for the target candidate key points belonging to one non-motor vehicle target.

[0033] If the candidate key points belonging to multiple personnel targets are not matched with the parallel non-motor vehicle targets successfully, the multiple personnel targets are subjected to pose estimation according to the candidate key points of the multiple personnel targets, the pose estimation result includes whether it is a riding pose and a riding pose confidence, and the personnel target with the riding pose type and the maximum riding pose confidence is determined as the personnel target associated with the non-motor vehicle target.

[0034] A non-motor vehicle person-vehicle identification system in a crowded scene includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the computer program realizes the steps of the non-motor vehicle person-vehicle identification method in a crowded scene when running.

[0035] The beneficial effects of the present application are:

[0036] The non-motor vehicle person-vehicle identification method in a crowded scene can effectively solve the problem that in the prior art, in the mixed scene of pedestrians and non-motor vehicles, the visual features are prone to significant drift under the scenes of severe light changes, target partial occlusion, and mixed insertion of multiple targets, leading to association failure, and thus the person-vehicle association accuracy is low in the high-density scene of mixed pedestrians and non-motor vehicles. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 A flowchart of the non-motor vehicle person-vehicle identification method in a crowded scene. DETAILED DESCRIPTION

[0038] In order to make the objectives, technical solutions and advantages of the present application clearer, the following further describes the present application in detail with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application, that is, the described specific embodiments are only a part of the embodiments of the present application, but not all the specific embodiments. The components of the specific embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations, and the present application can also have other embodiments.

[0039] Therefore, the following detailed description of the specific embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but only represents selected specific embodiments of the present application. Based on the specific embodiments of the present application, all other specific embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0040] In order to further understand the inventive content, characteristics and effects of the present application, the following specific embodiments are exemplified, and the accompanying drawings are Figure 1 The detailed description is as follows:

[0041] Example 1

[0042] A non-motor vehicle person-vehicle identification method in a crowded scene includes the following steps:

[0043] S1. Collecting image data of a traffic intersection through a traffic camera, using a target detection model to perform target detection processing on the image data, detecting personnel targets and non-motor vehicle targets, and obtaining personnel target frames and non-motor vehicle target frames;

[0044] Further, the method for training the target detection model in step S1 includes the following steps:

[0045] S1.1. Input the sample image of the traffic intersection into the target detection model to be trained. The target detection model to be trained outputs the detection results, which are the target boxes for people and non-motorized vehicles.

[0046] S1.2. Perform a first error calculation between the person target bounding box detected in step S1.1 and the person bounding box labeled in the sample image to obtain a first error value;

[0047] The second error value is obtained by performing a second error calculation between the non-motorized vehicle target box detected in step S1.1 and the non-motorized vehicle target box labeled in the sample image;

[0048] The total error value is calculated based on the first error value and the second error value, expressed as follows:

[0049]

[0050] Where LOSS1 is the first error value and LOSS2 is the second error value; the first error value and the second error value are calculated using the cross-entropy loss function or the squared difference loss function; The weight of the first error value. The weight of the second error value;

[0051] Furthermore, the method for calculating the weights of the first error value and the second error value in step S1.2 is as follows: ; ;

[0052]

[0053]

[0054] in, The number of bounding boxes for people in the sample image. This refers to the number of non-motorized vehicle frames marked in the sample image. This refers to the total number of bounding boxes for people and bounding boxes for non-motorized vehicles in the sample image. ; For the sample image belonging to The first person in the labeled personnel box Individual frame, For the sample image belonging to The first person in the labeled personnel box Individual personnel frame; For the sample image belonging to The first in all the marked boxes A box, For the sample image belonging to The first in all the marked boxes a box; is the first person box in the sample image; is the first non-motor vehicle box in the sample image; is the first non-motor vehicle box in the sample image; is the first person box in the sample image; is the first non-motor vehicle box in the sample image; is the first non-motor vehicle box in the sample image; wherein, , represents the intersection over union between two boxes, represents the center distance between two boxes; and is an intermediate coefficient, represents the maximum number.

[0055] S1.3. With the minimum total error value as the optimization objective, adjust the model parameters in the target detection model to be trained through back propagation of the total error value, and iterate the adjustment process of the model parameters until the total error value reaches a preset value or the iteration number reaches a preset number, stop training, and obtain the trained target detection model.

[0056] Further, the target detection model can be a convolutional neural network (CNN), such as any one of LeNet, AlexNet, VGGNet, R-CNN, Fast R-CNN, Faster R-CNN, Mask R-CNN, YOLO, and SSD. To improve the detection accuracy of the target detection model for personnel targets and non-motor vehicle targets, a dynamic error algorithm is used to calculate the total error value. In the training process, the weights corresponding to the first error value and the second error value are determined according to the annotated personnel boxes and the annotated non-motor vehicle boxes in the sample image. That is, for each sample image, according to the distribution of the annotated personnel boxes and the annotated non-motor vehicle boxes in the sample image, there will be a corresponding set of weights for calculating the total error value corresponding to the first error value and the second error value, and the set of weights can be associated with the sample image in the form of labels. Specifically, the weights corresponding to the first error value and the second error value can be determined according to the number, overlap degree, and distance of the annotated personnel boxes and the annotated non-motor vehicle boxes.

[0057] After obtaining the trained target detection model, image data is input into the trained target detection model for detection, and the corresponding personnel target box and non-motor vehicle target box are output. The number of personnel target boxes is multiple, and the number of non-motor vehicle target boxes is at least one. If no non-motor vehicle target is detected, the non-motor vehicle vehicle-person association process will not be performed.

[0058] S2. According to the distribution density of the personnel target frame and the non-motor vehicle target frame obtained in step S1, it is judged whether the scene category to which the image data of the traffic intersection belongs is a crowded scene, and crowded scene image data is obtained;

[0059] After obtaining the personnel target frame and the non-motor vehicle target frame corresponding to the image data, it can be judged according to the distribution density of the target frame whether the scene category to which the image data belongs is a crowded scene. It is explained that the personnel and the non-motor vehicle are in a complex scene, and the accuracy of the person-vehicle association recognition needs to be improved. The distribution density of the target frame can be determined according to the center position of the target frame, the number of target frames, and the overlapping degree between the target frames.

[0060] Further, the judgment method of whether the scene category to which the image data of the traffic intersection belongs is a crowded scene in step S2 is to cluster according to the center position of the target frame to obtain a clustering cluster of the target frame. If the number of target frames in the clustering cluster with the largest number of target frames is greater than a preset number value, and the average value of the IOU between two target frames in the clustering cluster is greater than a preset value, it is judged that the scene category to which the image data of the traffic intersection belongs is a crowded scene.

[0061] Further, the center points of the personnel target frame and the center points of the non-motor vehicle target frame are extracted, and a Delaunay triangular network is constructed using these center points. There is no other point in the surrounding of the circumcircle of any triangle in the Delaunay triangular network, and the vertices of the triangle are the center points of the target frame. The area of each triangle is calculated. If the number of triangles with an area less than a preset area is greater than a preset triangle number, or the proportion of the number of triangles with an area less than a preset area reaches a preset proportion, it can be judged that the scene category to which the image data belongs is a crowded scene. Through the above judgment method, it can be more accurately judged whether the scene category to which the image data belongs is a crowded scene, and reliable image data is provided for subsequent person-vehicle association.

[0062] S3. Extracting the personnel key points and the non-motor vehicle key points in the crowded scene image data obtained in step S2;

[0063] Further, in step S3, the personnel key points include wrist joint points, elbow joint points, knee joint points, and ankle joint points; and the non-motor vehicle key points include handlebar key points and pedal key points.

[0064] Further, the personnel key points are associated with the personnel targets in the corresponding image data through personnel unique identifiers, and the non-motor vehicle key points are associated with the non-motor vehicle targets in the corresponding image data through non-motor vehicle unique identifiers. When the personnel key points are extracted, the image data in the personnel target frame can be extracted in combination with the personnel target frame. Similarly, when the non-motor vehicle key points are extracted, the image data in the non-motor vehicle target frame can be extracted in combination with the non-motor vehicle target frame.

[0065] More specifically, for a non-motor vehicle target frame, the personnel target frame having an intersection with the non-motor vehicle target frame can be determined as a to-be-extracted personnel target frame, and the personnel key points are extracted from the to-be-extracted personnel target frame. In this way, the key points of the personnel target frame far away from the non-motor vehicle do not need to be extracted, and the calculation amount of the personnel key points extraction can be reduced.

[0066] S4. The personnel key points and the non-motor vehicle key points obtained in step S3 are associated to determine the personnel key points associated with the non-motor vehicle key points, and a person-vehicle association target is obtained;

[0067] Further, the personnel key points and the non-motor vehicle key points are associated in step S4 according to the distance prior between the personnel key points and the non-motor vehicle key points;

[0068] The distance prior is set as a distance constraint between different personnel key points and non-motor vehicle key points, including the following constraints:

[0069] The distance between the wrist joint node and the handlebar key point is less than a first distance value, the distance between the elbow joint node and the handlebar key point is less than a second distance value and greater than a third distance value, the third distance value is less than the second distance value, and the third distance value is greater than or equal to the first distance value;

[0070] The distance between the ankle joint node and the pedal key point is less than a fourth distance value, the distance between the knee joint node and the pedal key point is less than a fifth distance value and greater than a sixth distance value, the sixth distance value is less than the fifth distance value, and the sixth distance value is greater than or equal to the fourth distance value.

[0071] Further, the first distance value is set to 5 cm, the second distance value is set to 30 cm, and the third distance value is set to 5 cm;

[0072] Further, the fourth distance value is set to 3 cm, the fifth distance value is set to 30 cm, and the sixth distance value is set to 15 cm;

[0073] Further, the associating the personnel key points and the non-motor vehicle key points obtained in step S3 in step S4 comprises associating one non-motor vehicle target and one personnel target, and respectively associating multiple non-motor vehicle targets and multiple personnel targets.

[0074] Further, for one non-motor vehicle target, the personnel key points and the non-motor vehicle key points can be matched according to the distance prior between the personnel key points and the non-motor vehicle key points, the personnel key points that are matched successfully are taken as candidate key points, and the personnel target corresponding to the candidate key points can be associated with the non-motor vehicle target according to the unique identifier of the personnel target.

[0075] Specifically, for one non-motor vehicle target and one personnel target, the personnel key points corresponding to the personnel target and the non-motor vehicle key points corresponding to the non-motor vehicle target are calculated according to the distance prior, for each non-motor vehicle key point, there is at least one personnel key point corresponding to the non-motor vehicle key point having the distance prior constraint, the distance between the personnel key points and the non-motor vehicle key point is calculated, if the distance between all personnel key points and the non-motor vehicle key point satisfies the distance prior constraint, it is determined that the matching is successful. If the distance between a part of the personnel key points and the non-motor vehicle key point satisfies the distance prior constraint, and the distance between another part of the personnel key points and the non-motor vehicle key point does not satisfy the distance prior constraint, it is further determined whether there is a personnel key point in the other part of the personnel key points that is close to the non-motor vehicle key point in the distance prior, if there is, it is determined that the matching is unsuccessful, if there is not, it is further determined whether there is a personnel key point in the part of the personnel key points that is close to the non-motor vehicle key point in the distance prior, if there is, it is determined that the matching is successful, if there is not, it is determined that the matching is unsuccessful. Of course, it can also be determined whether there is a personnel key point in the part of the personnel key points that is close to the non-motor vehicle key point in the distance prior, if there is not, it is determined that the matching is unsuccessful, if there is, it is further determined whether there is a personnel key point in the other part of the personnel key points that is close to the non-motor vehicle key point in the distance prior, if there is, it is determined that the matching is unsuccessful, if there is not, it is determined that the matching is unsuccessful.

[0076] Further, the method of respectively associating multiple non-motor vehicle targets and multiple personnel targets in step S4 comprises the following steps:

[0077] S4.1. sorting the key point coordinates of the multiple personnel targets and the non-motor vehicle targets and the parallel non-motor vehicle targets corresponding thereto to obtain a key point sequence, the sorting direction being that the key points close to the camera are in front and the key points far from the camera are in back; and performing calibration processing on the key point sequence through the drift calibration vector corresponding to the camera to obtain a calibrated key point sequence;

[0078] S4.2. In the calibrated key point sequence, determine the candidate key point closest to the non-motor vehicle key point, and for the target candidate key points belonging to the same non-motor vehicle target, determine the personnel target with the most unique personnel identification as the personnel target associated with the non-motor vehicle target;

[0079] If the candidate key points belonging to multiple personnel targets are not successfully matched with the parallel non-motor vehicle target, perform pose estimation on the multiple personnel targets according to the candidate key points of the multiple personnel targets, and the pose estimation result includes whether it is a riding pose and a riding pose confidence, and determine the personnel target with the riding pose type and the maximum riding pose confidence as the personnel target associated with the non-motor vehicle target.

[0080] Further, by matching the non-motor vehicle target and each personnel target respectively, taking the matched personnel key points as candidate key points, if there are multiple personnel target corresponding personnel key points matched successfully, i.e. the candidate key points belong to multiple personnel targets, first determine whether there is a parallel non-motor vehicle target, if there is a parallel non-motor vehicle target, then judge whether the candidate key points belonging to multiple personnel targets are also matched with the parallel non-motor vehicle target, if the candidate key points belonging to multiple personnel targets are also matched with the parallel non-motor vehicle target, sort the key point coordinates corresponding to the multiple personnel targets and the non-motor vehicle target and its parallel non-motor vehicle target.

[0081] Further, the drift calibration vector is obtained by training according to the orientation of the camera and the annotated crowded image. Specifically, the annotated crowded image includes sample personnel targets and sample non-motor vehicle targets with annotated association. First, the personnel key points and non-motor vehicle key points are extracted from the crowded image according to the above key point extraction method, and the extracted sample personnel key points and sample non-motor vehicle key points are arranged according to the orientation of the camera and the distance from near to far to obtain a sample key point sequence. Then, according to the sample personnel targets and sample non-motor vehicle targets with annotated association, the extracted sample personnel key points are arranged according to the association to obtain a reference key point sequence, and an initial calibration vector is constructed. The initial calibration vector is multiplied by the sample key point sequence to obtain a calibrated key point sequence. The error between the calibrated key point sequence and the reference key point sequence is calculated to obtain an error value between the calibrated key point sequence and the reference key point sequence. With the minimum error value optimization target, the initial calibration vector is numerically adjusted through the back propagation algorithm, and the above numerical adjustment process is iterated until the error value between the calibrated key point sequence and the reference key point sequence is less than a preset value, or the preset iteration number is reached, and the iteration is stopped. The last adjusted calibration vector is determined as the drift calibration vector.

[0082] S5. Identifying whether the person-vehicle associated target obtained in step S4 exists traffic violation behavior through image tracking technology, if exists, identifying the identity information of the person target for traffic violation early warning and processing.

[0083] Embodiment 2:

[0084] A non-motor vehicle person-vehicle identification system in a crowded scene, comprising a processor, a memory and a computer program stored in the memory and executable on the processor, the computer program realizes the steps of a non-motor vehicle person-vehicle identification method in a crowded scene as described in embodiment 1 when running.

[0085] It should be noted that the relational terms such as first and second and the like are used solely to distinguish one from another entity or action, without necessarily requiring or implying such entity or action to be the same or different from, or to be ordered in time before or after, any other entity or action. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0086] Although the present application has been described with reference to specific implementations, it is apparent that various modifications and changes can be made thereto without departing from the scope of the present application. In particular, various features that are disclosed in the specific implementations of the present application can be combined together in any way, unless such combinations are not technically possible. The fact that such combinations have not been explicitly described in the present specification is merely for the sake of brevity and to avoid unnecessarily obscuring the present application. Therefore, the present application is not limited to the specific implementations disclosed in the present specification, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for identifying non-motorized vehicles and pedestrians in congested scenarios, characterized in that, Includes the following steps: S1. Collect image data of traffic intersections through traffic cameras, and use a target detection model to perform target detection processing on the image data to detect personnel targets and non-motorized vehicle targets, and obtain personnel target boxes and non-motorized vehicle target boxes; S2. Based on the distribution density of the person target box and the non-motor vehicle target box obtained in step S1, determine whether the scene category of the traffic intersection image data is a congested scene, and obtain the congested scene image data; S3. Extract key points of people and non-motorized vehicles from the crowded scene image data obtained in step S2; S4. Associate the personnel key points and non-motor vehicle key points obtained in step S3, determine the personnel key points associated with the non-motor vehicle key points, and obtain the personnel-vehicle association target; S5. Use image tracking technology to identify whether the human-vehicle related targets obtained in step S4 have traffic violations. If so, identify the identity information of the human targets for traffic violation warning and handling.

2. The method for identifying non-motorized vehicles and pedestrians in congested scenarios according to claim 1, characterized in that, Step S1, training the object detection model, includes the following steps: S1.

1. Input the sample image of the traffic intersection into the target detection model to be trained. The target detection model to be trained outputs the detection results, which are the target boxes for people and non-motorized vehicles. S1.

2. Perform a first error calculation between the person target bounding box detected in step S1.1 and the person bounding box labeled in the sample image to obtain a first error value; The second error value is obtained by performing a second error calculation between the non-motorized vehicle target box detected in step S1.1 and the non-motorized vehicle target box labeled in the sample image; The total error value is calculated based on the first error value and the second error value, expressed as follows: ； Where LOSS1 is the first error value and LOSS2 is the second error value; the first error value and the second error value are calculated using the cross-entropy loss function or the squared difference loss function; The weight of the first error value. The weight of the second error value; S1.

3. With the goal of minimizing the total error value, backpropagation is performed using the total error value to adjust the model parameters in the target detection model to be trained, and the adjustment process of the model parameters is iterated until the total error value reaches the preset value or the number of iterations reaches the preset number, at which point training stops and a trained target detection model is obtained.

3. The method for identifying non-motorized vehicles and pedestrians in congested scenarios according to claim 2, characterized in that, The method for calculating the weights of the first error value and the second error value in step S1.2 is as follows: ； ； ； ； in, The number of bounding boxes for people in the sample image. This refers to the number of non-motorized vehicle frames marked in the sample image. This refers to the total number of bounding boxes for people and bounding boxes for non-motorized vehicles in the sample image. ; For the sample image belonging to The first person in the labeled personnel box Individual frame, For the sample image belonging to The first person in the labeled personnel box Individual personnel frame; For the sample image belonging to The first in all the marked boxes A box, For the sample image belonging to The first in all the marked boxes One box; For the sample image belonging to The first one in the marked non-motorized vehicle box One non-motorized vehicle frame, For the sample image belonging to The first one in the marked non-motorized vehicle box One non-motorized vehicle frame; among them , This indicates the intersection-union ratio between two boxes. This indicates the center distance between the two boxes; and As an intermediate coefficient, This indicates taking the maximum number.

4. The method for identifying non-motorized vehicles and pedestrians in congested scenarios according to claim 3, characterized in that, The method for determining whether the scene category of the traffic intersection image data in step S2 is a congested scene is to cluster the target boxes based on the center position of the target boxes to obtain the target box clusters. If the number of target boxes in the cluster with the most target boxes is greater than a preset value, and the average IOU between two target boxes in the cluster is greater than a preset value, then the scene category of the traffic intersection image data is determined to be a congested scene.

5. The method for identifying non-motorized vehicles and pedestrians in congested scenarios according to claim 4, characterized in that, Step S3 sets key points for personnel, including wrist joint, elbow joint, knee joint, and ankle joint; and sets key points for non-motorized vehicles, including handlebar key points and pedal key points.

6. The method for identifying non-motorized vehicles and pedestrians in congested scenarios according to claim 5, characterized in that, In step S4, the key points of personnel and non-motorized vehicles are associated based on the prior distance between the key points of personnel and non-motorized vehicles. The distance prior is set as the distance constraint between different personnel key points and non-motorized vehicle key points, including the following constraints: The distance between the wrist joint and the handlebar key point is less than the first distance value; the distance between the elbow joint and the handlebar key point is less than the second distance value and greater than the third distance value; the third distance value is less than the second distance value; and the third distance value is greater than or equal to the first distance value. The distance between the ankle joint and the pedal key point is less than the fourth distance value; the distance between the knee joint and the pedal key point is less than the fifth distance value and greater than the sixth distance value; the sixth distance value is less than the fifth distance value; and the sixth distance value is greater than or equal to the fourth distance value.

7. The method for identifying non-motorized vehicles and pedestrians in congested scenarios according to claim 6, characterized in that, Step S4 involves associating the key points of personnel and non-motorized vehicles obtained in step S3. This includes associating one non-motorized vehicle target with one personnel target, and associating multiple non-motorized vehicle targets and multiple personnel targets separately.

8. A method for identifying non-motorized vehicles and pedestrians in congested scenarios according to claim 7, characterized in that, The method for associating multiple non-motorized vehicle targets and multiple pedestrian targets in step S4 includes the following steps: S4.

1. Sort the key point coordinates corresponding to multiple personnel targets, non-motorized vehicle targets and parallel non-motorized vehicle targets to obtain a key point sequence. The sorting direction is that the key points closer to the camera are in front and the key points farther away from the camera are in back. The key point sequence is calibrated by the drift calibration vector corresponding to the camera to obtain a calibrated key point sequence. S4.

2. In the calibrated key point sequence, determine the candidate key point that is closest to the non-motorized vehicle key point. For target candidate key points that belong to the same non-motorized vehicle target, determine the personnel target with the most unique personal identifiers as the personnel target associated with the non-motorized vehicle target. If the candidate key points belonging to multiple personnel targets fail to match with the parallel non-motorized vehicle targets, then the attitude of the multiple personnel targets is estimated based on the candidate key points of the multiple personnel targets. The attitude estimation results include whether it is a riding posture and the confidence of the riding posture. The personnel target with the riding posture type and the highest riding posture confidence is determined as the personnel target associated with the non-motorized vehicle target.

9. A non-motorized vehicle and pedestrian recognition system for congested scenarios, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed, implements the steps of a non-motorized vehicle and pedestrian recognition method in a congested scenario as described in any one of claims 1-8.

Images