A method of non-motor vehicle provenance

Abstract

The application provides a non-motor vehicle tracing method, a collection device group is constructed, monitoring images of non-motor vehicles and drivers and passengers are collected from four angles of front, back, left and right, then vehicle features and personnel features are extracted based on the monitoring images of known shooting angles, the uniqueness of the vehicle and the driver and passenger is confirmed by comparing the features of the non-motor vehicle and the driver and passenger, the dependence on the license plate of the non-motor vehicle is got rid of, the non-motor vehicle and the driver and passenger can be accurately judged, and the identification probability of the vehicle and the driver and passenger is improved; in the method, each motor vehicle and driver and passenger is filed, each non-motor vehicle and driver and passenger is marked by a feature ID, all to-be-traced pictures are compared with the filed features, the method is not limited by the installation position of the image collection device and the image collection time, and the method is more practical.

Description

Technical Field

[0001] This invention relates to the field of intelligent traffic control technology, specifically to a method for tracing the origin of non-motorized vehicles. Background Technology

[0002] In addition to motor vehicles, non-motorized vehicles such as electric bicycles, motorcycles, and tricycles also travel on urban roads. Electric bicycles lack a standardized license plate format, making license plate identification difficult and hindering vehicle identification. Motorcycles typically only have license plates at the rear, with no front plate or the plate not visible from the front. Tricycles usually lack license plates or have inconsistent license plate formats, making license plate identification challenging and hindering unique identification of two-wheeled and three-wheeled vehicles. Therefore, even if images and license plates are captured at the scene of an accident or incident involving these non-motorized vehicles, it is difficult to track their movements once they leave the area. Furthermore, electric bicycle and motorcycle riders usually wear helmets, making facial recognition identification even more difficult. Summary of the Invention

[0003] To address the problem of difficulty in uniquely identifying non-motorized vehicles and their riders in existing images, this invention provides a non-motorized vehicle tracing method that can effectively identify the uniqueness of non-motorized vehicles and trace their origins using existing equipment.

[0004] The technical solution of the present invention is as follows: a method for tracing the origin of non-motorized vehicles, characterized by comprising the following steps:

[0005] S1: Select an intersection for data acquisition, and install a group of image acquisition devices around the center of the intersection, denoted as: acquisition device group;

[0006] The shooting angle of each image acquisition device in the acquisition device group is set based on the road direction entering the intersection, and is respectively facing the center of the intersection; the acquisition device group simultaneously acquires images of each vehicle entering the intersection from the front, rear, left and right directions, which are denoted as: surveillance images;

[0007] S2: Establish a connection between the coordinate systems of all image acquisition devices, and calibrate the corresponding positions within each image acquisition device;

[0008] S3: Construct a non-motorized vehicle target detection model to detect the monitoring images and mark the found non-motorized vehicles as: target vehicles; in the image acquisition devices of the acquisition device group, find and store images of the target vehicle from the front, back, left and right angles, and at the same time, store the vehicle features extracted by the non-motorized vehicle target detection model when detecting the target vehicle according to the front, back, left and right angles respectively.

[0009] S4: Obtain the stored images of the target vehicle from four angles, denoted as: images to be tracked;

[0010] Color correction is performed on the non-motorized vehicle image area in each of the images to be tracked, and contrast stretching is performed on the non-motorized vehicle image area and the background to obtain the corrected image to be tracked.

[0011] S5: Construct a human image detection model for vehicle occupants;

[0012] S6: Based on the vehicle occupant face detection model, input four corrected images to be tracked respectively, and calculate the driver and passengers corresponding to the target vehicle according to the model output results;

[0013] Simultaneously, the facial features extracted by the vehicle occupant facial detection model when detecting the driver and passengers are stored separately according to the four angles of front, back, left, and right.

[0014] S7: Create a file for the target vehicle and the corresponding driver and passengers;

[0015] Each target vehicle and each person is assigned a unique ID; simultaneously, relationships are established between the target vehicles and the people.

[0016] The file format is: Vehicle Characteristic ID_Driver / Passenger Sign_Personnel Characteristic ID;

[0017] Among them, the vehicle feature ID is a unique ID for non-motorized vehicles;

[0018] Personnel characteristic ID is a unique ID for a person within the system;

[0019] The driver and passenger sign is used to distinguish between the driver and the passenger.

[0020] The vehicle feature ID is simultaneously associated with the monitoring images and vehicle features from four different angles corresponding to the target vehicle.

[0021] The personnel feature ID is simultaneously associated with the monitoring images and facial features of the driver and passengers from four different angles.

[0022] S8: When other image acquisition devices acquire monitoring images of non-motorized vehicles, vehicle features are extracted through the non-motorized vehicle target detection model, and personnel features are extracted through the vehicle driver and passenger face detection model. Then, the features are compared with all target vehicles and drivers and passengers stored in the system to trace the source of the vehicles and personnel.

[0023] Its further features are:

[0024] In step S1, the intersection includes: a three-way intersection, a crossroads, a T-junction, and a roundabout;

[0025] When selecting the intersection for data collection, prioritize crossroads or T-junctions;

[0026] In step S2, a connection is established between the coordinate systems of all image acquisition devices, specifically including the following steps:

[0027] a1: Construct a grid on the road surface of the intersection where the data is collected;

[0028] The grid is a square grid, and the side length of the grid is adapted to the length of the non-motorized vehicle to be tracked;

[0029] a2: In the grid, find the overlapping area of ​​the shooting areas of all image acquisition devices, denoted as: shooting overlap area;

[0030] a3: Based on the aforementioned overlapping shooting area, establish a connection between physical world coordinates and the coordinates of the four-channel camera images, map the physical world grid to a coordinate space grid, and establish a connection between the coordinate systems of all image acquisition devices;

[0031] In step S3, the monitoring image is detected based on the non-motorized vehicle target detection model, specifically including the following steps:

[0032] b1: In the monitoring image, the detection box of the non-motorized vehicle is marked based on the non-motorized vehicle target detection model and is denoted as: vehicle detection box to be confirmed.

[0033] b2: Find all grids in the monitoring image that intersect with the detection frame of the vehicle to be confirmed, and denot them as: grids to be calculated;

[0034] Assume that the detection frame of the vehicle to be confirmed intersects with n grids to be calculated, where n≥1;

[0035] Confirm the shooting angle of the current monitoring image, and calculate the positional relationship between the detection frame area of ​​the vehicle to be confirmed and the n grids to be calculated;

[0036] If it's a front-to-back angle, proceed to step b3;

[0037] If it's a left or right angle, proceed to step b4;

[0038] b3: Calculate the intersection-union ratio (IOU) between the detection frame of the vehicle to be confirmed and the n grids to be calculated;

[0039] Compare each IOU with a preset intersection / merge threshold;

[0040] If any IOU > the intersection-union threshold, it means that the vehicle to be confirmed falls completely in the corresponding grid. The grid corresponding to the IOU is denoted as the grid to be confirmed; proceed to step b5.

[0041] Otherwise, it means the vehicle to be confirmed did not fall completely within any grid; execute b1;

[0042] b4: Locate the center of the two wheels in the vehicle detection frame to be confirmed, and record it as: center of non-motorized vehicle wheel;

[0043] Confirm whether the centers of the two non-motorized vehicle wheels fall within either of the grid cells to be calculated;

[0044] If so, it means that the vehicle to be confirmed falls completely in the corresponding grid, and the corresponding grid is marked as: grid to be confirmed; proceed to step b5;

[0045] Otherwise, it means the vehicle to be confirmed did not fall completely within any grid; execute b1;

[0046] b5: Confirm whether there is only one non-motorized vehicle in the grid to be confirmed;

[0047] If so, then mark the vehicle to be confirmed as: Target Vehicle, and proceed to step b6;

[0048] Otherwise, if there are multiple non-motorized vehicles in a grid, the current judgment ends and steps b1~b4 are executed repeatedly.

[0049] b6: Based on the shooting time of the monitoring image and the position of the grid to be confirmed, find and store images of the target vehicle from the front, rear, left and right angles in the image acquisition device of the acquisition device group. At the same time, store the vehicle features extracted by the non-motorized vehicle target detection model when detecting the target vehicle according to the front, rear, left and right angles respectively.

[0050] The vehicle characteristics include: visual characteristics and physical characteristics;

[0051] In step S4, color correction is performed on each of the images to be tracked using a correction function. The color correction function is as follows:

[0052] ;

[0053] Among them, C outTo correct the RGB values ​​of the output image pixels, C in To correct the RGB values ​​of the input image pixels, M corresponds to the median value of a specific color component in the RGB color space; C is calculated for each of the R, G, and B color branches. out ;

[0054] In step S6, based on the model output, the driver and passengers corresponding to the target vehicle are calculated, specifically including the following steps:

[0055] c1: Take any of the corrected images to be tracked, and based on the vehicle driver and passenger portrait detection model, detect all portrait images included in the non-motorized vehicle image area, which are denoted as: portraits to be confirmed;

[0056] Construct a portrait region bounding box for each of the portraits to be confirmed;

[0057] c2: Establish an image coordinate system with the upper left corner of the detected target vehicle area as the origin;

[0058] In the image coordinate system, confirm the coordinates of the bounding boxes corresponding to the portrait regions of all portraits to be confirmed;

[0059] c3: Based on the angle corresponding to the storage of the corrected image to be tracked, determine the direction of the target vehicle in the corrected image to be tracked, and determine whether the target vehicle is the front, rear or side of the vehicle.

[0060] If it is the front of the vehicle, proceed to step c4;

[0061] If it's the rear of the vehicle, proceed to step c5;

[0062] Otherwise, proceed to step c6;

[0063] c4: The center point of the image region bounding box for the person to be confirmed is the driver of the vehicle if the maximum value of the horizontal and vertical coordinates is the center point of the image region bounding box; the others are passengers.

[0064] c5: The person whose center point of the rectangular frame of the portrait area has the minimum value on both the horizontal and vertical coordinates is the driver; the others are passengers.

[0065] c6: Locate the front of the vehicle, and identify the driver and passengers in sequence from the front to the rear. The first passenger is the driver, and the others are passengers.

[0066] Step S8 specifically includes the following steps:

[0067] d1: Surveillance images of non-motorized vehicles captured by other image acquisition devices are denoted as: images to be traced.

[0068] The image acquisition device corresponding to the image to be traced is referred to as: tracking acquisition device;

[0069] d2: Confirm the shooting angle of the tracking and acquisition device and the deviation angle of the road it photographs, and determine the shooting direction of the image to be traced;

[0070] Based on the determined shooting direction of the image to be traced, the image acquisition device with the closest shooting angle to the image to be traced is found in the acquisition device group, and is denoted as: the comparison reference device;

[0071] d3: Correct the image of the image to be traced to the orientation of the monitoring image captured by the reference device for comparison, and obtain: the corrected traced image;

[0072] d4: Vehicle features are extracted from the corrected source-tracing image using the non-motorized vehicle target detection model, denoted as: source-tracing vehicle features;

[0073] The human features extracted from the corrected source-tracing image using a vehicle occupant facial detection model are denoted as: source-tracing human features.

[0074] The vehicle features and personnel features used for tracing are compared with the vehicle features and personnel features corresponding to the images captured by the comparison reference device stored in the system.

[0075] If a feature with a similarity that meets a preset threshold can be found, it means that the source object has been found, and the source vehicle feature or the source personnel feature of the source object is added to the source object's file.

[0076] This application provides a method for tracing non-motorized vehicles. It constructs a data acquisition device group to capture monitoring images of non-motorized vehicles and their riders from four angles: front, rear, left, and right. Then, based on the monitoring images captured from known angles, vehicle and rider features are extracted. By comparing the features of the non-motorized vehicles and their riders, the uniqueness of the vehicles and riders is confirmed, eliminating reliance on non-motorized vehicle license plates. This method not only accurately identifies non-motorized vehicles and their riders but also improves the probability of vehicle and rider identification. In this method, a file is created for each motorized vehicle and its rider, and each non-motorized vehicle and its rider is labeled with a feature ID. All images to be traced are compared with the archived features, making this method unrestricted by the installation location of the image acquisition device or the image acquisition time, thus ensuring greater practicality. Attached Figure Description

[0077] Figure 1 Example of an installation structure for a data acquisition device group;

[0078] Figure 2 Example of creating a grid for an intersection;

[0079] Figure 3This is an example of the positional relationship between the driver and the passenger. Detailed Implementation

[0080] This invention includes a method for tracing the origin of non-motorized vehicles, which includes the following steps.

[0081] S1: Select an intersection for data acquisition, and install a group of image acquisition devices around the center of the intersection. This group is called the acquisition device group.

[0082] The shooting angle of each image acquisition device in the acquisition equipment group is set based on the road direction entering the intersection, pointing towards the center of the intersection. The acquisition equipment group simultaneously acquires images of each vehicle entering the intersection from four directions: front, rear, left, and right, denoted as: surveillance images. This method uses the acquisition equipment group to acquire images of non-motorized vehicles entering the intersection from four directions, serving as the basis for subsequent comparisons.

[0083] In reality, intersections include: three-way intersections, crossroads, T-junctions, and roundabouts. When selecting intersections for installing data collection equipment in urban roads, intersections with high traffic volume are preferred, especially crossroads or T-junctions. This is because existing monitoring equipment at crossroads or T-junctions can capture data from vehicles entering the intersection in all four directions (front, back, left, and right). Furthermore, calculating the angles of the four-directional data collection equipment at crossroads or T-junctions is relatively simple. Installing data collection equipment at crossroads or T-junctions reduces design complexity.

[0084] like Figure 1 Taking a crossroads as an example, the installation method of the data collection equipment in four directions is explained. Camera groups are installed at four positions A, B, C, and D at the intersection, as shown in the installation diagram. The shooting direction is the center point O of the intersection. Point A shoots the rear of the non-motorized vehicle, point B shoots the front of the non-motorized vehicle, point C shoots the left side of the non-motorized vehicle, and point D shoots the right side of the non-motorized vehicle.

[0085] S2: Establish a connection between the coordinate systems of all image acquisition devices, and calibrate the corresponding positions within each image acquisition device.

[0086] In step S2, a connection is established between the coordinate systems of all image acquisition devices, specifically including the following steps:

[0087] a1: Construct a grid on the road surface of the intersection where data is collected, such as... Figure 2 As shown;

[0088] The grid consists of squares, and the grid side length is adapted to the length of the non-motorized vehicle to be tracked.

[0089] If we statistically analyze the average length of existing electric bicycles and motorcycles, we can use approximately 1.5m x 1.5m as the size of a single grid to mark the road surface. Typically, ordinary intersections can be divided into roughly 5x5 grids.

[0090] a2: In the grid, find the overlapping area of ​​the shooting areas of all image acquisition devices, denoted as: shooting overlap area.

[0091] In practice, the intersection is divided into a physical space grid by drawing lines on the ground. Four cameras are used to photograph the road grid, and the images are observed manually. The complete grid and its number that can be photographed by all four cameras are recorded. This part of the grid is the overlapping area of ​​the images.

[0092] a3: Based on the overlapping shooting area, establish the connection between the physical world coordinates and the coordinates of the four-camera imaging images, map the physical world grid to the coordinate space grid, and establish a connection between the coordinate systems of all image acquisition devices.

[0093] Based on the overlapping shooting area, the pixel coordinate space position in the image space is calibrated in the four cameras respectively, the relationship between the physical world coordinates and the image coordinates of the four cameras is established, and the physical world grid is mapped to the coordinate space grid.

[0094] S3: Construct a non-motorized vehicle target detection model. In specific implementation, the non-motorized vehicle target detection model is constructed based on existing neural network models, such as CNN, ResNet, YOLO, etc.

[0095] Image acquisition devices installed at intersections capture images of non-motorized vehicles passing through the intersection, obtaining surveillance images from four angles: front, rear, left, and right. Based on a non-motorized vehicle target detection model, the surveillance images are used to detect and mark the detected non-motorized vehicles as "target vehicles." The images of the target vehicle from the four angles are stored in the image acquisition equipment group. Simultaneously, the vehicle features extracted by the non-motorized vehicle target detection model during the target vehicle detection are stored separately for each of the four angles.

[0096] In step S3, the monitoring image is detected based on the non-motorized vehicle target detection model, which specifically includes the following steps.

[0097] b1: In the surveillance image, the detection box of non-motorized vehicles is marked based on the non-motorized vehicle target detection model, and is denoted as: vehicle detection box to be confirmed.

[0098] b2: Find all grids in the surveillance image that intersect with the detection frame of the vehicle to be confirmed, and denot them as: grids to be calculated;

[0099] Assume that the detection frame of the vehicle to be confirmed intersects with n grids to be calculated, where n≥1;

[0100] Confirm the shooting angle of the current monitoring image, and calculate the positional relationship between the detection frame area of ​​the vehicle to be confirmed and the n grids to be calculated;

[0101] If it's a front-to-back angle, proceed to step b3;

[0102] If it's a left or right angle, proceed to step b4.

[0103] b3: Calculate the Intersection over Union (IOU) of the detection frame of the vehicle to be confirmed with each of the n grids to be calculated;

[0104] Intersection over Union (IoU) = A∩B / A∪B;

[0105] The definition of IoU is the ratio of the intersection area to the union area of ​​two bounding boxes (or segmentation masks) A and B.

[0106] Compare each IOU with a preset intersection / merge threshold;

[0107] If any IOU > the intersection-union threshold, it means that the vehicle to be confirmed falls completely in the corresponding grid. The grid corresponding to the IOU is denoted as the grid to be confirmed; proceed to step b5.

[0108] Otherwise, it means that the vehicle to be confirmed has not completely fallen into any grid; execute b1.

[0109] b4: Locate the center of the two wheels in the vehicle detection frame to be confirmed, and record it as: center of non-motorized vehicle wheel;

[0110] Confirm whether the centers of the two non-motorized vehicle wheels fall within any of the grid cells to be calculated;

[0111] If so, it means that the vehicle to be confirmed falls completely in the corresponding grid, and the corresponding grid is marked as: grid to be confirmed; proceed to step b5;

[0112] Otherwise, it means that the vehicle to be confirmed has not completely fallen into any grid; execute b1.

[0113] b5: Confirm whether there is only one non-motorized vehicle in the grid to be confirmed;

[0114] If so, then mark the vehicle to be confirmed as: Target Vehicle, and proceed to step b6;

[0115] Otherwise, if there are multiple non-motorized vehicles in a grid, the current judgment ends and steps b1 to b4 are executed in a loop.

[0116] b6: Based on the shooting time of the surveillance images and the location of the grid to be confirmed, find and store images of the target vehicle from the front, rear, left, and right angles in the image acquisition equipment of the acquisition equipment group. At the same time, store the vehicle features extracted by the non-motorized vehicle target detection model when detecting the target vehicle according to the front, rear, left, and right angles respectively.

[0117] Vehicle features include visual features and physical features; visual features include vehicle image region hash value, shape, color, texture, etc.; physical features include overall shape, length, width, height, etc.; the specific features extracted depend on the type of non-motorized vehicle target detection model.

[0118] S4: Obtain images of the target vehicle from four angles, denoted as: images to be tracked;

[0119] Color correction is performed on the non-motorized vehicle image region in each image to be tracked, and contrast stretching is applied to the non-motorized vehicle image region and the background to obtain the corrected image to be tracked. The specific color correction and contrast stretching methods can be implemented based on existing technologies. In this embodiment, a color correction function is set for the color correction method. The color correction function is used to perform color correction on each image to be tracked; the color correction function is as follows:

[0120] ;

[0121] Among them, C out To correct the RGB values ​​of the output image pixels, C in To correct the RGB values ​​of the input image pixels, M corresponds to the median value of a specific color component in the RGB color space; C is calculated for each of the R, G, and B color branches. out .

[0122] Based on historical surveillance images collected at the intersection, the RGB values ​​of normal skin color in the images are approximately (240, 180, 180). Therefore, in this embodiment, M is set to 240 when calculating the R component, 180 when calculating the G component, and 180 when calculating the B component. Using the median values ​​of the above three skin color components (R, G, and B) and a slope of 5, color correction functions for the R, G, and B color spaces are constructed according to the formula of the color correction function.

[0123] This method achieves the distinction between non-motorized vehicle riders and the background through color correction and contrast stretching. The color correction method enhances the color difference between the non-motorized vehicle image area and the background image, reducing interference from excessively bright or dark external light, and light from sunshades.

[0124] S5: Construct a human image detection model for vehicle occupants;

[0125] S6: Based on the vehicle occupant face detection model, input four corrected images to be tracked, and calculate the driver and passengers corresponding to the target vehicle according to the model output results;

[0126] At the same time, the facial features extracted by the vehicle occupant facial detection model when detecting drivers and passengers are stored separately according to the four angles of front, back, left, and right.

[0127] In step S6, the driver and passengers corresponding to the target vehicle are calculated based on the model output results, which specifically includes the following steps.

[0128] c1: Take any corrected image to be tracked, and based on the vehicle driver and passenger face detection model, detect all the human images included in the non-motorized vehicle image area, which are denoted as: human images to be confirmed;

[0129] Construct a bounding box for each portrait to be confirmed;

[0130] c2: Establish an image coordinate system with the upper left corner of the detected target vehicle area as the origin;

[0131] In the image coordinate system, confirm the coordinates of the bounding boxes corresponding to the portrait regions of all portraits to be confirmed;

[0132] c3: Based on the angle corresponding to the stored image after correction, determine the direction of the target vehicle in the image after correction and determine whether the target vehicle is the front, rear or side of the vehicle.

[0133] If it is the front of the vehicle, proceed to step c4;

[0134] If it's the rear of the vehicle, proceed to step c5;

[0135] Otherwise, proceed to step c6;

[0136] c4: The center point of the image region bounding the unconfirmed person is the driver if the maximum value of the horizontal and vertical coordinates is the center point of the vehicle's driver area; the others are passengers.

[0137] c5: The person whose center point of the rectangular frame of the portrait area has the minimum value on both the horizontal and vertical coordinates is the driver; the others are passengers.

[0138] c6: Locate the front of the vehicle, and identify the driver and passengers in sequence from the front to the rear. The first passenger is the driver, and the others are passengers.

[0139] S7: Create a profile for the target vehicle and its corresponding driver and passengers;

[0140] Assign a unique ID to each target vehicle and each person; at the same time, establish relationships between the target vehicles and people.

[0141] The file format is: Vehicle Characteristic ID_Driver / Passenger Sign_Personnel Characteristic ID;

[0142] Among them, the vehicle feature ID is a unique ID for non-motorized vehicles, which is automatically generated by the system;

[0143] Personnel Feature ID is a unique ID for each person in the system, which is automatically generated by the system;

[0144] Driver and passenger signs are used to distinguish between the driver and the passenger. For example, "1" represents the driver and "2" represents the passenger.

[0145] The vehicle feature ID is simultaneously associated with the monitoring images and vehicle features from four different angles corresponding to the target vehicle.

[0146] The personnel feature ID is simultaneously associated with the monitoring images and facial features from four different angles corresponding to the driver and passengers.

[0147] Based on the actual situation, there are cases in the archived data where the same vehicle feature ID has multiple personnel features, that is, multiple people driving or riding in one vehicle. For example, there may be a driver and a passenger in one vehicle at the same time, or one vehicle may be driven or ridden by different people at different times. There are also cases where the same personnel feature ID has multiple vehicle feature IDs, that is, one person driving or riding in multiple vehicles at different times.

[0148] like Figure 3 In the non-motorized vehicle image area shown, there are two occupants, P1 and P2, on the non-motorized vehicle. According to... Figure 3 Based on the direction of the chariot diagram, we can see that P2 is the driver and P1 is the passenger.

[0149] S8: When other image acquisition devices acquire monitoring images of non-motorized vehicles, vehicle features are extracted through the non-motorized vehicle target detection model, and personnel features are extracted through the vehicle driver and passenger face detection model. Then, the features are compared with all target vehicles and drivers and passengers stored in the system to trace the source of the vehicles and personnel.

[0150] Step S8 specifically includes the following steps:

[0151] d1: Surveillance images of non-motorized vehicles captured by other image acquisition devices are denoted as: images to be traced.

[0152] The image acquisition device corresponding to the image to be traced is denoted as: tracking acquisition device;

[0153] d2: Confirm the shooting angle of the tracking and acquisition device and the deviation angle of the road it photographs, and determine the shooting direction of the image to be traced;

[0154] Based on the shooting direction of the image to be traced, find the image acquisition device with the closest shooting angle to the image to be traced in the acquisition device group, and denot it as: the reference device for comparison.

[0155] d3: Correct the image of the image to be traced to the orientation of the monitoring image taken by the reference device, and obtain: the corrected traced image.

[0156] The specific method involves correcting the road's orientation in the image to be traced to the orientation of the image captured by the reference device; for example... Figure 3 As shown, the reference device used for comparison is the camera angle at the front of the vehicle. In the image it captures, the road direction r is perpendicular to the x-axis from top to bottom.

[0157] The direction r' of the road in the image to be traced has an angle α with the direction r. Therefore, the direction r' of the road in the image to be traced is first corrected to the direction r to obtain the correction matrix. Then, the image in the image to be traced is corrected based on this correction matrix to obtain the corrected traceable image.

[0158] The included angle α can be obtained based on the equipment parameters retained during the installation of the tracking and acquisition equipment, or it can be calculated based on existing technology.

[0159] d4: Vehicle features are extracted from the corrected source-tracing image using a non-motorized vehicle target detection model, denoted as: source-tracing vehicle features;

[0160] The human features extracted from the corrected source-tracing image using a vehicle occupant facial detection model are denoted as: source-tracing human features.

[0161] The vehicle and personnel features used for tracing are compared with the vehicle and personnel features corresponding to the images taken by the reference equipment stored in the system. If a feature with a similarity that meets the preset threshold is found, it means that the tracing object has been found, and the vehicle or personnel features used for tracing the found tracing object are added to the tracing object's file.

[0162] That is, the corrected vehicle image is compared with the non-motorized vehicle image features recorded by the system. If a match is found, the non-motorized vehicle is added to the file. Similarly, the corrected personnel image is compared with the personnel image features recorded by the system. If a match is found, the person's image is added to the file.

[0163] By using the technical solution of this invention, based on the existing roadside-installed image acquisition equipment, it is possible to perform feature comparison on non-motorized vehicles such as two-wheeled vehicles and three-wheeled vehicles, confirm the uniqueness of vehicles and drivers and passengers, and effectively identify the uniqueness of non-motorized vehicles and trace the source of vehicles and drivers and passengers at a lower cost.

Claims

1. A method of non-motor vehicle provenance, characterized in that, It comprises the following steps: S1: selecting a collection intersection, installing a set of image collection devices around the center of the collection intersection, denoted as: collection device set; The shooting angle of each image collection device in the collection device set is set based on the road direction entering the intersection, respectively towards the intersection center position; Through the collection device set, the front, rear, left and right four direction pictures of each vehicle entering the collection intersection are collected simultaneously, denoted as: monitoring pictures; S2: establish a connection between the coordinate systems of all image collection devices, respectively calibrate the corresponding positions in each image collection device; S3: construct a non-motor vehicle target detection model, detect the monitoring pictures, and mark the found non-motor vehicle as: target vehicle; In the image collection device of the collection device set, the front, rear, left and right four angle pictures of the target vehicle are stored, and the vehicle features extracted by the non-motor vehicle target detection model when detecting the target vehicle are stored according to the front, rear, left and right four angles respectively; S4: obtain the stored four angle pictures of the target vehicle, denoted as: to be tracked pictures; Color correction is performed on the non-motor vehicle image area in each to-be-tracked picture, and contrast stretching is performed on the non-motor vehicle image area and the background to obtain the corrected to-be-tracked picture; S5: construct a vehicle driver and passenger portrait detection model; S6: based on the vehicle driver and passenger portrait detection model, respectively input four corrected to-be-tracked pictures, and calculate the driver and passenger corresponding to the target vehicle according to the model output result; At the same time, the portrait features extracted by the vehicle driver and passenger portrait detection model when detecting the driver and passenger are stored according to the front, rear, left and right four angles respectively; S7: establish a file for the target vehicle and the corresponding driver and passenger; Each target vehicle and each person is respectively assigned a unique ID; at the same time, the target vehicle and the person are related; The file form is: vehicle feature ID_driver and passenger mark_personnel feature ID; The vehicle feature ID is the unique ID of the non-motor vehicle; The personnel feature ID is the unique ID of the personnel in the system; The driver and passenger mark is used to distinguish the driver and passenger; The vehicle feature ID is related to the four angle monitoring pictures of the target vehicle and the vehicle feature; The personnel feature ID is related to the four angle monitoring pictures of the driver and passenger and the portrait feature; S8: when other image collection devices collect non-motor vehicle monitoring pictures, extract vehicle features through the non-motor vehicle target detection model, extract personnel features through the vehicle driver and passenger portrait detection model, and then compare the features with all target vehicles and driver and passenger stored in the system to trace the vehicle and personnel.

2. The method as claimed in claim 1, wherein: In step S1, the intersection includes: three-way intersection, cross intersection, T-shaped intersection and roundabout; When selecting the collection intersection, the cross intersection or T-shaped intersection is preferred.

3. The method as claimed in claim 1, wherein the non-motor vehicle is traced by: In step S2, the connection between the coordinate systems of all image collection devices is established, which specifically comprises the following steps: a1: constructing a grid on the road surface of the intersection for image collection; the grid is a square grid, and the side length of the grid is adapted to the length of the non-motor vehicle to be tracked; a2: finding an overlapping area of the shooting areas of all image collection devices in the grid, denoted as a shooting overlapping area; a3: based on the shooting overlapping area, establishing a connection between the physical world coordinates and the imaging picture coordinates of the four-way camera, mapping the physical world grid into a coordinate space grid, and establishing a connection for the coordinate systems of all image collection devices.

4. The method of claim 1, wherein: In step S3, the monitoring picture is detected based on the non-motor vehicle target detection model, specifically including the following steps: b1: in the monitoring picture, marking the detection box of the non-motor vehicle based on the non-motor vehicle target detection model, denoted as a to-be-confirmed vehicle detection box; b2: finding all grids that have intersections with the to-be-confirmed vehicle detection box in the monitoring picture, denoted as to-be-calculated grids; assuming that the to-be-confirmed vehicle detection box has intersections with n to-be-calculated grids, n≥1; confirming the shooting angle of the current monitoring picture and respectively calculating the positional relationship between the to-be-confirmed vehicle detection box coverage area and the n to-be-calculated grids; if it is a front and rear angle, step b3 is executed; if it is a left and right angle, step b4 is executed; b3: respectively calculating the intersection over union (IOU) of the to-be-confirmed vehicle detection box and the n to-be-calculated grids; comparing each IOU with a preset intersection over threshold value; if there is any IOU>intersection over threshold value, it indicates that the to-be-confirmed vehicle is completely in the corresponding grid, the grid corresponding to the IOU is denoted as a to-be-confirmed grid, and step b5 is executed; otherwise, it indicates that the to-be-confirmed vehicle is not completely in any grid, and b1 is executed; b4: finding the positions of the centers of the two wheels in the to-be-confirmed vehicle detection box, denoted as non-motor vehicle wheel centers; confirming whether the two non-motor vehicle wheel centers fall in any of the to-be-calculated grids; if yes, it indicates that the to-be-confirmed vehicle is completely in the corresponding grid, the corresponding grid is denoted as a to-be-confirmed grid, and step b5 is executed; otherwise, it indicates that the to-be-confirmed vehicle is not completely in any grid, and b1 is executed; b5: confirming whether there is only one non-motor vehicle in the to-be-confirmed grid; if yes, the to-be-confirmed vehicle is denoted as a target vehicle, and step b6 is executed; otherwise, if there are multiple non-motor vehicles in a grid, the current judgment is ended, and steps b1-b4 are executed in a loop; b6: based on the shooting time of the monitoring picture and the position of the to-be-confirmed grid, finding the pictures of the front, rear, left and right angles of the target vehicle in the image collection devices of the image collection device group, and storing the vehicle features extracted by the non-motor vehicle target detection model when detecting the target vehicle according to the front, rear, left and right angles.

5. The method of claim 4, wherein: The vehicle features include visual features and physical features.

6. The method of claim 1, wherein: In step S4, each of the to-be-tracked pictures is color corrected by a correction function, and the color correction function is: ； Wherein, C out is the output image pixel RGB value after correction, C in is the input image pixel RGB value before correction, M corresponds to the median value of a specific color component value in the RGB color space; C out is calculated for R, G, B three color branches respectively.

7. The method of claim 1, wherein: In step S6, the driver and passenger corresponding to the target vehicle are calculated according to the model output result, specifically including the following steps: c1: randomly select one of the rectified to-be-traced pictures, detect all the portrait images included in the non-motor vehicle image region based on the vehicle occupant portrait detection model, and mark as: to-be-confirmed portrait; construct a portrait region frame for each of the to-be-confirmed portraits; c2: establish an image coordinate system with the top-left corner of the detected target vehicle region as the coordinate origin; in the image coordinate system, confirm the coordinates corresponding to the portrait region frames corresponding to all the to-be-confirmed portraits; c3: based on the corresponding angle of the rectified to-be-traced picture when it is stored, determine the direction of the target vehicle in the rectified to-be-traced picture, and determine whether the target vehicle is photographed from the front, the back, or the side; if it is the front, execute step c4; if it is the back, execute step c5; otherwise, execute step c6; c4: the center point of the portrait region frame of the to-be-confirmed portrait is the maximum horizontal coordinate and the maximum vertical coordinate, which is the vehicle driver, and the others are the passengers; c5: the center point of the portrait region rectangular frame is the minimum horizontal coordinate and the minimum vertical coordinate, which is the vehicle driver, and the others are the passengers; c6: find the position of the vehicle head, and confirm the driver and the passenger from the vehicle head to the vehicle tail in turn, with the first one being the driver and the others being the passengers.

8. The method of claim 1, wherein: In step S8, the following steps are specifically included: d1: collect a monitoring picture of a non-motor vehicle by other image collection devices, and mark as: to-be-traced picture; mark the image collection device corresponding to the to-be-traced picture as: tracking collection device; d2: confirm the deviation angle of the shooting angle of the tracking collection device and the road it shoots, and determine the shooting direction of the to-be-traced picture; find the image collection device with the closest shooting angle to the to-be-traced picture in the set of collection devices according to the determined shooting direction of the to-be-traced picture, and mark as: comparison reference device; d3: rectify the image of the to-be-traced picture to the direction of the monitoring image shot by the comparison reference device, and obtain: rectified to-be-traced image; d4: extract the vehicle features in the rectified to-be-traced image by the non-motor vehicle target detection model, and mark as: to-be-traced vehicle features; extract the personnel features in the rectified to-be-traced image by the vehicle occupant portrait detection model, and mark as: to-be-traced personnel features; compare the to-be-traced vehicle features and the to-be-traced personnel features with the vehicle features and the personnel features corresponding to the image shot by the comparison reference device stored in the system, respectively; if the features with a similarity satisfying a preset threshold can be found, it means that the traced object is found, and the to-be-traced vehicle features or the to-be-traced personnel features of the traced object are added to the archives of the traced object.

Images