Traffic flow track generation method and system based on artificial intelligence

Abstract

The invention discloses a traffic flow track generation method and system based on artificial intelligence. The method comprises the following steps: acquiring a road traffic video; preprocessing the road traffic video; a target detection algorithm and a multi-target tracking algorithm are adopted to identify traffic participants in the road traffic video after tracking preprocessing; extracting historical track information of each traffic participation object, and generating a track of each traffic participation object on the road traffic video; wherein for the traffic participation object with the abnormal identification and tracking, a track editing algorithm is adopted to generate a track of the traffic participation object with the abnormal identification and tracking. According to the method, abnormal conditions are processed through a track editing algorithm, the complex conditions can be effectively handled, and the adaptability of the traffic flow track generation method based on artificial intelligence in different traffic scenes is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic scene generation, and more specifically, to a method and system for generating traffic flow trajectories based on artificial intelligence. Background Art

[0002] The closed-field test of autonomous vehicles is designed to verify the basic functions of autonomous vehicles and the stability of their operating systems. In a closed field, a variety of roads and traffic scenarios can be simulated to conduct a comprehensive and accurate performance evaluation of the autonomous driving system. In order to more realistically restore the traffic scenes on the road, it is necessary to collect traffic participants and their trajectories on the real road.

[0003] In the prior art, there are already technologies for identifying and tracking target objects. For example, the patent with the authorization announcement number CN118349694B discloses a method and system for generating a vehicle trajectory database in a ramp merging area. The method uses the laser radar point cloud data taken in the ramp merging area to identify vehicle targets, and then tracks the trajectories of all identified vehicle targets separately. The driving data of each vehicle target is calculated based on the trajectory tracking results of each vehicle target, and the vehicle trajectory database is generated based on the trajectory tracking results and driving data of each vehicle target. However, due to the complexity of real traffic scenes, there are usually traffic participants with abnormal identification and tracking, such as continuous or discontinuous occlusion by other objects, resulting in interruptions in the trajectory. Therefore, for such traffic participants with abnormal identification and tracking, how to generate their traffic trajectories is a technical problem that urgently needs to be solved. Summary of the invention

[0004] An object of the present invention is to solve at least the above problems and to provide at least the advantages which will be described hereinafter.

[0005] In order to achieve these purposes and other advantages according to the present invention, a method for generating traffic flow trajectories based on artificial intelligence is provided, comprising: Obtain road traffic videos; Preprocess road traffic videos; The YOLOv4 target detection algorithm and DeepSort multi-target tracking algorithm are used to identify and track traffic participants in pre-processed road traffic videos. Extract the historical trajectory information of each traffic participant and generate the trajectory of each traffic participant on the road traffic video; For a traffic participant object with abnormal identification and tracking, a trajectory editing algorithm is used to generate a trajectory of the traffic participant object with abnormal identification and tracking; The trajectory editing algorithm includes: Drawing a static object area in the preprocessed road traffic video, and removing traffic participants that are always in the static object area; When identifying and tracking traffic participants in road traffic videos, for different traffic participants identified under the same ID, the traffic participant with the highest appearance frequency is selected as the traffic participant of the ID; Eliminate traffic participants whose historical trajectory information is less than the preset number; Based on the historical trajectory information of the remaining traffic participants in the road traffic video, the trajectory of each traffic participant is regenerated.

[0006] Preferably, the method of regenerating the trajectory of each traffic participant object comprises: Crop the edges of road traffic videos; Using a trajectory smoothing algorithm to generate the trajectory of the traffic participant object in multiple adjacent frames of the road traffic video; Calculating the size of the recognition and tracking detection frame of the traffic participant object in each frame of the road traffic video, and then normalizing the recognition and tracking detection frame of the traffic participant object to the same size; A cubic B-spline fitting algorithm is used to merge all the generated trajectories of the traffic participant objects.

[0007] Preferably, the method for regenerating the trajectory of each traffic participant object further comprises: Verify the merged trajectories to find out the trajectories and traffic participants whose starting point and end point are at least one not at the edge of the road traffic video image; For the trajectories and their traffic participants whose at least one starting point and end point is not at the edge of the road traffic video image, find at least one pair of traffic participants of the same type in chronological order, in which the end point of one traffic participant object is at the shortest distance from the starting point of another traffic participant object, and merge the trajectories of the pair of traffic participants of the same type.

[0008] Preferably, the trajectory smoothing algorithm is a Catmull-Rom algorithm, wherein a tension parameter tension is dynamically calculated according to the speed of the traffic participant object, and the value of the tension parameter tension is inversely proportional to the speed of the traffic participant object.

[0009] Preferably, after using the target detection algorithm and the multi-target tracking algorithm to identify and track the traffic participants in the preprocessed road traffic video, the recognition results are also verified to identify and track abnormal traffic participants, and the verification methods include: identification object quantity comparison method, identification object coordinate analysis method, and visualization comparison method.

[0010] Preferably, preprocessing the road traffic video includes: Video stabilization: Video stabilization based on multi-scale feature point matching. The SIFT algorithm is used to extract feature points in the video frame and calculate the homography matrix between adjacent frames. The video frame is affine transformed to eliminate dynamic jitter. The threshold for dynamic jitter is adaptively adjusted according to the flight altitude of the drone. Distortion correction processing: Adopting adaptive fisheye distortion correction algorithm, dynamically generating correction grid according to lens focal length and field of view parameters, and correcting video frames pixel by pixel through bilinear interpolation, where the correction strength is inversely proportional to the square of the radius of the pixel from the center of the image; and Noise reduction based on spatiotemporal hybrid filtering: Multi-scale Retinex algorithm is used to separate the rain line noise layer of rainy and snowy weather video frames, and the noise layer is suppressed by non-local mean filtering; dynamic histogram equalization based on optical flow method is used for areas with sudden changes in illumination to balance the brightness differences between adjacent frames; The preprocessed video frames are verified by edge gradient histograms. If the inter-frame gradient change rate exceeds a preset threshold value T1, 0.12≤T1≤0.18, iterative optimization of video stabilization and distortion correction is triggered.

[0011] Preferably, the step of extracting feature points in the video frame by using the SIFT algorithm includes: extracting feature points of the video frame by using the SIFT algorithm, setting the number of Gaussian pyramid layers to 4, the scale factor of each layer to 1.6, and the threshold value of the number of feature points extracted per frame to 500; The steps of calculating the homography matrix between adjacent frames include: calculating the homography matrix between adjacent frames based on the RANSAC algorithm, setting the number of iterations to 2000, the reprojection error threshold to 1.5 pixels, and screening feature point pairs with a matching degree ≥ 85%; The calculation formula of the dynamic jitter judgment threshold T satisfies the following formula: T=0.025×H+3, where H is the flight altitude of the drone, in meters, and T is the maximum pixel offset allowed for jitter, in pixels. When the translation component of the homography matrix between adjacent frames exceeds the threshold T, affine transformation compensation is triggered. Among them, after the video frame is affine transformed, the effective area of ​​the cropping center is 80% of the original frame width and 80% of the original frame height, and the coordinates of the retained area are [0.1W: 0.9W, 0.1H: 0.9H], where W and H are the width and height of the original frame respectively; the video frame after stabilization is output, the feature point matching error after stabilization is ≤1.5 pixels, and the processing speed is ≥25 frames / second. Preferably, the method for generating a correction grid comprises: generating a dynamic correction grid according to the focal length f of the lens, in units of mm, and the field of view angle θ, in units of degrees, wherein the grid node spacing d, in units of mm, is calculated according to the formula d=0.12f×tan(θ / 2), wherein the value range of θ is 120°≤θ≤180°, and the value range of f is 2.8mm≤f≤8mm; The method for correcting the video frame pixel by pixel by bilinear interpolation includes: calculating the normalized center distance r of each pixel, satisfying , where (x c ,y c ) is the pixel coordinate, W and H are the width and height of the video frame respectively; the correction strength coefficient is generated according to the distance r , bilinear interpolation is used to calculate the offset at the correction grid node pixel by pixel, the offset Δ x , Δ y satisfy , ; Map the corrected pixel coordinates to (x c +Δ x ,y c +Δ y ), and bilinear interpolation sampling is used for non-integer coordinate pixels, and the interpolation weight is allocated according to the inverse of the coordinate distance of the four adjacent pixels.

[0012] Preferably, the method of separating the rain line noise layer using the multi-scale Retinex algorithm includes: Set the Gaussian kernel size to [15, 30, 45], corresponding to the standard deviation σ = [15, 30, 45], and separate the noise layer I by the following formula noise : in, is the sth Gaussian kernel, σ s is the corresponding standard deviation, x and y represent pixel coordinates; Suppressing the noise floor includes the following steps: Calculate the noise layer pixel suppression weight W as , perform non-local mean filtering on the noise layer, the search window size is 7×7, unit: pixel, the similarity block size is 3×3, unit: pixel, the filtering parameter h=0.8×σ_n, σ_n is the noise variance; The illumination mutation area is detected by the following method: the motion vectors Δx and Δy of adjacent frames are calculated based on the optical flow method. Pixels are determined as areas of sudden illumination changes; The dynamic histogram equalization method includes: dividing the illumination mutation area into local windows, and the window size is , unit: pixel, contrast enhancement coefficient within the window , the brightness constraint after equalization is , where μ is the mean brightness in the window, L in Represents the brightness value of the original input pixel, L out Represents the output pixel brightness value after equalization.

[0013] The present invention also provides a traffic flow trajectory generation system based on artificial intelligence, comprising: A video acquisition module, which is used to acquire road traffic videos; A preprocessing module, which is used to preprocess the road traffic video; The recognition and tracking module is used to recognize and track traffic participants in the pre-processed road traffic video using the YOLOv4 target detection algorithm and the DeepSort multi-target tracking algorithm; A trajectory generation module, which is used to extract the historical trajectory information of each traffic participant and generate the trajectory of each traffic participant on the road traffic video; A trajectory editing module, which is used to generate a trajectory of the traffic participant object with abnormal identification and tracking by using a trajectory editing algorithm; The trajectory editing algorithm includes: Drawing a static object area in the preprocessed road traffic video, and removing traffic participants that are always in the static object area; When identifying and tracking traffic participants in road traffic videos, for different traffic participants identified under the same ID, the traffic participant with the highest appearance frequency is selected as the traffic participant of the ID; Eliminate traffic participants whose historical trajectory information is less than the preset number; Based on the historical trajectory information of the remaining traffic participants in the road traffic video, the trajectory of each traffic participant is regenerated.

[0014] The present invention includes at least the following beneficial effects: drawing a static object area in the pre-processed road traffic video and eliminating the traffic participants who are always in the area, which can effectively remove static interference factors such as roadside fixed facilities and stationary parked vehicles. In the identification and tracking process, different traffic participants may be identified under the same ID due to detection errors and other reasons. Selecting the traffic participant with the highest frequency as the object corresponding to the ID can effectively solve the ID confusion problem. Eliminating the traffic participant whose historical trajectory information is less than the preset number avoids the unreliable trajectory due to too little data from entering the subsequent analysis. After processing the traffic participant objects identified and tracked abnormally by the above-mentioned trajectory editing algorithm, the trajectory is regenerated based on the historical trajectory information of the remaining traffic participant objects, providing a more accurate and reliable data basis for traffic flow analysis. Accurate trajectory generation helps to explore the potential movement patterns and relationships between traffic participants. The road traffic scenes in reality are complex and changeable, and there are various interference factors and detection errors. The scheme processes abnormal situations through the trajectory editing algorithm, which can effectively deal with these complex situations and enhance the adaptability of the traffic flow trajectory generation method based on artificial intelligence in different traffic scenes. By rationally handling the traffic participants that are tracked through abnormal identification, the possibility of system errors or instability due to data anomalies is reduced.

[0015] Other advantages, objectives and features of the present invention will be embodied in part through the following description, and in part will be understood by those skilled in the art through study and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flow chart of the method for generating traffic flow trajectories based on artificial intelligence according to the present invention; Figure 2 A schematic diagram of the flow of the target detection algorithm of the present invention; Figure 3 It is a structural schematic diagram of the traffic flow trajectory generation system based on artificial intelligence described in the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] like Figure 1 As shown, the present invention provides a method for generating traffic flow trajectories based on artificial intelligence, comprising: S1. Obtain road traffic video; Specifically, the road traffic video can be shot by a drone at high altitude, the shooting height can be set at 100m and / or 300m, the shooting frequency can be set at 25Hz and / or 30Hz, and the resolution can be set to 3840×2160 and / or 4096×2160.

[0019] Road traffic videos taken by drones at high altitudes can be sent to mobile terminals or ground stations via wireless transmission methods, including WiFi, Bluetooth, 4G / 5G, etc.

[0020] S2, preprocessing the road traffic video; Specifically, the preprocessing of the road traffic video includes: video stabilization processing and distortion correction processing.

[0021] The main purpose of video stabilization is to eliminate jitter in the video and improve the video image presentation effect, which is conducive to subsequent target detection, recognition and tracking.

[0022] The video stabilization process can adopt the method in "Eliminating the shake of high-altitude video based on OpenCV".

[0023] The main purpose of distortion correction processing is to eliminate the image geometric distortion caused by lens characteristics and further improve the video stabilization effect.

[0024] Distortion correction processing can use an adaptive fisheye correction algorithm to eliminate barrel distortion caused by wide-angle lenses.

[0025] Of course, the impact of rainy and snowy weather and lighting changes on video quality can also be reduced based on the spatiotemporal filtering algorithm, further improving the video quality.

[0026] S3, using target detection algorithm and multi-target tracking algorithm to identify and track traffic participants in the pre-processed road traffic video; Specifically, the Darknet framework can be used to train the YOLOv4 model and the DeepSort model.

[0027] Darknet is mainly written in C and CUDA (Compute Unified Devices Architecture). CUDA can be used for inference acceleration computing operations on ordinary NVIDIA graphics cards, which can reduce training time and inference speed and is a widely used operation in deep learning. The model trained using darknet is a binary model file with the suffix .weight. The models of the commonly used YOLOv1-YOLOv4 series official algorithms are all trained and generated using the darknet framework. When parsing the .weight model, the deep neural network (DNN) needs to parse the .weight model and the .cfg model structure file at the same time. .cfg is the structure file corresponding to the .weight model, which contains information such as the number of layers, depth, and dimensions of each layer of convolution of the network structure. In addition to the above two files, the corresponding border confidence and IOU_threshold need to be set during parsing to filter the target box through NMS non-maximum suppression.

[0028] In this embodiment, the YOLOv4 model and the DeepSort model are selected to build a target recognition and tracking framework according to the evaluation results of the target detection and tracking model, and the confidence threshold is set to 0.5. The model can be input into the DNN for parsing, and then the video can be input into the network to complete the tracking of the target in the video.

[0029] The YOLOv4 model and DeepSort model use labelImage to prepare the VOC format training set.

[0030] The main task of target detection is to find the target to be found in an image, and the algorithm outputs the location and confidence of the target in the image.

[0031] Target detection uses the DNN library of opencv to complete model loading and reasoning. Opencv can be used directly to complete the reasoning process using the CPU. NVIDIA series graphics cards can use the opencv compiled by CUDA for gpu reasoning. Its working process includes: loading the model-obtaining frames-image preprocessing-model reasoning-model post-processing. Loading the model is also called model initialization, which is to load the weights of each layer of the model. The trained neural network model will have a fixed image input resolution and a fixed data format. Image preprocessing is to convert the obtained image into the format required by the model. The preprocessing results are output to the model, and the model is used for reasoning calculation to obtain the model output. The model output results are analyzed and sorted to obtain the required results. The model output results often have different structures due to different model characteristics. Post-processing first requires parsing and screening the output results to obtain the required results.

[0032] For example, Figure 2 As shown in the figure, when there is an object in a picture, the YOLOv4 model detects the object through the following framework: the input picture is passed through a CNN network to obtain a feature map, and then the feature map is connected to two fully connected layers, one is a fully connected layer that outputs object classification (as many outputs as there are objects), and the other is a fully connected layer that outputs location information. In addition, the feature map obtained by the CNN network can be used as the input of the DeepSort model. The DeepSort model uses these features for target matching, uses the Kalman filter to predict the motion state of the target, and combines the appearance features for data association. The Hungarian algorithm is used to solve the optimal matching problem between the target and the detection box, and finally outputs the tracking results of each target, including the target ID and location information.

[0033] S4, extracting historical trajectory information of each traffic participant, and generating a trajectory of each traffic participant on the road traffic video; Specifically, in the above steps, the traffic participants in the pre-processed road traffic video have been identified and tracked by the target detection algorithm and the multi-target tracking algorithm, that is, the traffic participants in the road traffic video have been identified and tracked, and the ID, classification result and position result of each traffic participant have been obtained. Therefore, in this step, the position information of the traffic participants in each frame of the road traffic video is extracted in sequence based on the time sequence. The position information of the traffic participants with the same ID is its historical trajectory information. The historical trajectory information of the traffic participants is connected in the image of the road traffic video according to the time sequence to obtain the trajectory of the traffic participants.

[0034] S5. For the traffic participant object with abnormal identification and tracking, a trajectory editing algorithm is used to generate the trajectory of the traffic participant object with abnormal identification and tracking.

[0035] The trajectory editing algorithm includes: Drawing a static object area in the preprocessed road traffic video, and removing traffic participants that are always in the static object area; When identifying and tracking traffic participants in road traffic videos, for different traffic participants identified under the same ID, the traffic participant with the highest appearance frequency is selected as the traffic participant of the ID; Eliminate traffic participants whose historical trajectory information is less than the preset number; Based on the historical trajectory information of the remaining traffic participants in the road traffic video, the trajectory of each traffic participant is regenerated.

[0036] Specifically, the above-mentioned trajectory editing algorithm can be used to correct and improve the recognition and tracking anomalies caused by algorithm errors or other reasons.

[0037] Draw a static object area in the pre-processed road traffic video. The static object area is drawn by manual annotation or automatic identification based on video frame difference analysis, and then the traffic participants who are always in the area are eliminated, which can effectively eliminate interference. For example, the static object area set in this embodiment is a bicycle lane and a roadside green belt. The entire space and time (time, position) of the target object is within the static object area, and it is considered to be a static object. For example: fixed signs beside the road, vehicles that are parked and do not move during monitoring, etc. If these static objects are not eliminated, they will be mistakenly identified as traffic participants, increasing the amount of data processing and interfering with normal analysis. After the static objects in the static object area are eliminated, there are no box annotations for objects outside the road area. The road area is displayed using a high-precision map, and the box annotations of the remaining traffic participants are superimposed.

[0038] In the case of identifying different traffic participants under the same ID, the traffic participant with the highest frequency is selected as the traffic participant corresponding to the ID, ensuring the consistency of the identity of the traffic participant and the accuracy of the data. The highest frequency means that under the same ID, the target object appears the most times in all video frames. In actual scenarios, due to detection errors or occlusions and other factors, the same real traffic participant may be misjudged as a different object and assigned the same ID. This method can correct such errors. Traffic participants whose historical trajectory information is lower than the preset number are eliminated to further ensure data reliability. The preset number can be set to 10 trajectory points. Too little trajectory information may be due to detection errors or short-term appearance, which is of little value to the overall traffic analysis. After elimination, noise data can be reduced, so that subsequent analysis can be based on more effective data.

[0039] The trajectory is regenerated based on the historical trajectory information of the remaining traffic participants, which improves the accuracy and availability of the trajectory of abnormal traffic participants. The optimized trajectory can more realistically reflect its actual movement and provide reliable data support for subsequent abnormal behavior analysis and accident cause tracing.

[0040] Furthermore, the method of regenerating the trajectory of each traffic participant object includes: Crop the edges of road traffic videos; Using a trajectory smoothing algorithm to generate the trajectory of the traffic participant object in multiple adjacent frames of the road traffic video; Calculating the size of the recognition and tracking detection frame of the traffic participant object in each frame of the road traffic video, and then normalizing the recognition and tracking detection frame of the traffic participant object to the same size; A fitting algorithm is used to merge all the generated trajectories of the traffic participant objects.

[0041] Specifically, since the video edge will be distorted or pixel-filled after video stabilization, making the area near the edge unusable, a certain amount of padding can be removed, such as removing the absolute distance of padding = 100px. When performing tasks involving target detection and recognition, such as traffic scene analysis, the distortion and filling areas at the edge of the video may interfere with the algorithm's accurate recognition of the target. Removing these unreliable areas can improve the accuracy and stability of the target detection algorithm. For example, when detecting vehicles and pedestrians in traffic videos, it can avoid false detections or missed detections caused by abnormal edge areas, making the detection results more accurate. Removing the padding area at the edge reduces the amount of data processed by the video. Whether in storage, transmission or subsequent analysis and processing, it can reduce the burden on the system and improve processing efficiency.

[0042] The Catmull-Rom algorithm can be used for trajectory smoothing, in which the tension parameter tension is dynamically calculated according to the speed of the traffic participant. The value of the tension parameter tension is inversely proportional to the speed of the traffic participant. The speed of the traffic participant can be calculated by the pixel displacement of the target center point of the continuous frame and the video frame rate. Since the motion of the traffic participant is not an idealized linear or regular curve, the motion trend at different speeds is different. For example, when a high-speed vehicle turns, the curvature change of its trajectory is relatively small, while the trajectory of a pedestrian or bicycle driving at a low speed is more flexible and changeable when turning. By making the tension parameter inversely proportional to the speed, the smooth trajectory generated by the Catmull-Rom algorithm can better simulate this actual motion characteristic. When the speed of the traffic participant is fast, the tension parameter is small, and the generated trajectory is smoother and the curvature change is small, which is similar to the relatively stable trajectory of a high-speed vehicle; when the speed is slow, the tension parameter is large, and the trajectory can better reflect the flexible and changeable characteristics, which is in line with the actual motion of low-speed traffic participants. The smooth trajectory generated based on this algorithm can accurately restore the motion path of vehicles and pedestrians before the accident.

[0043] BOX normalization: Calculate the size of all frame boxes, calculate their mean, and normalize the tracking target bound to the same size. The size of the tracking target box detected in different frames may vary due to factors such as the distance of the target from the camera and the change in shooting angle. By normalizing to a unified size, a unified standard is established for all target boxes, so that the data between different frames and different targets is consistent. The normalization operation reduces the interference caused by box size fluctuations, making the algorithm more stable when processing data from different frames. In complex traffic scenes, factors such as light changes and target occlusion have brought challenges to the algorithm. Unifying the box size can avoid algorithm fluctuations caused by unstable target box size and ensure that the algorithm can run reliably in various situations. In addition, the normalized box data structure is more regular. When storing, transmitting and processing data, there is no need to perform special processing for boxes of different sizes, which greatly simplifies the data processing process.

[0044] The cubic B-spline fitting algorithm can be used for trajectory merging and frame supplementation. The cubic B-spline fitting algorithm generates a very smooth trajectory curve by fitting the given data points. In traffic scenes, the actual movement of traffic participants is usually continuous and smooth, and the algorithm can simulate this characteristic well. For example, when a vehicle is driving on the road, its trajectory will not have sudden turns or jumps. The frame supplementation trajectory generated by the cubic B-spline fitting algorithm can accurately reflect this smooth motion state, making the entire trajectory look more natural and reasonable. In practical applications, due to detection errors, occlusions and other reasons, trajectory data may be missing or incomplete. The cubic B-spline fitting algorithm can use existing data points to infer the missing part of the trajectory through fitting, thereby completing the entire trajectory. For example, in a surveillance video, a vehicle may be temporarily blocked by a large truck, resulting in the loss of part of the trajectory data. The cubic B-spline fitting algorithm can reasonably complete the trajectory during the blocked period based on the visible trajectory information before and after, ensuring a complete analysis of the vehicle's movement. Compared with some complex fitting algorithms, the cubic B-spline fitting algorithm has certain advantages in calculation. It can complete the fitting and frame supplementation operations of a large amount of trajectory data in a relatively short time. The algorithm is robust to noise and data fluctuations. In the actual collected trajectory data, there may be some small fluctuations caused by detection errors or environmental interference. The cubic B-spline fitting algorithm will not be affected by these small noises and can still stably generate accurate fitting trajectories. This stability ensures that trajectory merging and frame filling can be reliably achieved in various complex actual environments.

[0045] Furthermore, the method for regenerating the trajectory of each traffic participant object also includes: Verify the merged trajectories to find out the trajectories and traffic participants whose starting point and end point are at least one not at the edge of the road traffic video image; For the trajectories and their traffic participants whose at least one starting point and end point is not at the edge of the road traffic video image, find at least one pair of traffic participants of the same type in chronological order, in which the end point of one traffic participant object is at the shortest distance from the starting point of another traffic participant object, and merge the trajectories of the pair of traffic participants of the same type.

[0046] During the verification process, find out the tracks whose starting point and end point are not at the edge of the video image at least. These tracks may be incomplete or abnormal. For example, in traffic monitoring videos, normally moving vehicles usually enter from the edge of the screen and leave from the other edge. If the starting point or end point of a vehicle track is inside the screen, it may mean that the track record is missing or wrong. Excluding such abnormal tracks can improve the accuracy of the overall track data and avoid misleading subsequent analysis.

[0047] Finding a pair of the same type of traffic participants with the smallest distance between the end and the starting point in chronological order and merging their trajectories will help further improve the continuity of the trajectories. In actual traffic scenarios, the trajectory of the same traffic participant may be divided into multiple segments due to detection errors or temporary occlusions. Through this merging method, these trajectories that should be continuous can be reasonably connected to make the movement path of the traffic participant more in line with the actual situation. For application scenarios such as traffic flow analysis and research on the causes of traffic congestion, accurate and complete trajectory data is crucial. The optimized trajectory can more accurately reflect the actual driving path and rules of traffic participants, helping analysts better understand the operation mode of traffic flow.

[0048] For example, the object with ID 38 disappears in frame 3965. In the list of objects that may be merged with object ID 38, object 1158 is of the same type as object 38, both are bicycles, and its appearance frame is frame 3974, which is closest to the frame where the trajectory of object 38 disappears. The starting point of the trajectory of object 1158 is also closest to the vanishing point of the trajectory of object 38, with a distance of only 19. Therefore, the two are most likely to be the same target object. By checking, it is found that the trajectory and box can be connected, so the trajectories of the two target objects can be merged.

[0049] Furthermore, after using the target detection algorithm and the multi-target tracking algorithm to identify and track the traffic participants in the road traffic video after preprocessing, the recognition results are also verified to identify and track abnormal traffic participants. The verification methods include: identification object quantity comparison method, identification object coordinate analysis method, and visualization comparison method.

[0050] For example, a 300 s video at an altitude of 100 m and a 300 s video at an altitude of 300 m of a certain road section are taken as data samples.

[0051] Identification object quantity comparison method: count all identified objects and their types; parse and count the objects and types annotated in all frames within 300 seconds by the annotation tool for comparison (static objects are not excluded in Table 1 below, and static objects are excluded in Table 2 below).

[0052] Table 1 Table 2 Coordinate analysis method for identifying objects: Calculate the IOU based on the labeled box and the identified box. If it is greater than 0.5, it is considered to be correctly identified. Exemplarily, three verification frames are used for verification, and the results are shown in Tables 3 to 5 below. In the table, src represents the original labeled correct data, and det represents the data after detection and identification. The annotation of the recognition results can be divided into four situations: annotation 1 represents correct recognition, annotation 2 represents target loss, annotation 3 represents IOU error, and annotation 4 represents detection type error. Table 6 is a part of the intercepted recognition result table to illustrate the verification comparison details.

[0053] Table 3 Detection results of check frame 1 Table 4 Detection results of check frame 2 Table 5 Detection results of check frame 3 Table 6 Visual comparison method: Draw the operator output results and the annotation tool output results into the same frame. Boxes of different colors represent different object types. Red represents cars, blue represents tricycles, yellow represents bicycles, and orange represents pedestrians.

[0054] Furthermore, preprocessing the road traffic video includes: Video stabilization: Video stabilization based on multi-scale feature point matching. The SIFT algorithm is used to extract feature points in the video frame and calculate the homography matrix between adjacent frames. The video frame is affine transformed to eliminate dynamic jitter. The threshold for dynamic jitter is adaptively adjusted according to the flight altitude of the drone. Distortion correction processing: Adopting adaptive fisheye distortion correction algorithm, dynamically generating correction grid according to lens focal length and field of view parameters, and correcting video frames pixel by pixel through bilinear interpolation, where the correction strength is inversely proportional to the square of the radius of the pixel from the center of the image; and Noise reduction based on spatiotemporal hybrid filtering: Multi-scale Retinex algorithm is used to separate the rain line noise layer of rainy and snowy weather video frames, and the noise layer is suppressed by non-local mean filtering; dynamic histogram equalization based on optical flow method is used for areas with sudden changes in illumination to balance the brightness differences between adjacent frames; The preprocessed video frames are verified by edge gradient histograms. If the inter-frame gradient change rate exceeds a preset threshold value T1, 0.12≤T1≤0.18, iterative optimization of video stabilization and distortion correction is triggered.

[0055] In the above embodiment, when preprocessing the road traffic video, the video stabilization process adopts a method based on multi-scale feature point matching. The feature points in the video frame are extracted by the SIFT algorithm, and then the homography matrix between adjacent frames is calculated, and the dynamic jitter is eliminated by affine transformation. In addition, the dynamic jitter determination threshold can be adaptively adjusted according to the flight altitude of the drone, so as to ensure that the dynamic jitter in the video can be accurately and effectively eliminated at different flight altitudes, providing a stable video basis for subsequent analysis.

[0056] Distortion correction uses an adaptive fisheye distortion correction algorithm to dynamically generate a correction grid based on the lens focal length and field of view parameters. Video frames are corrected pixel by pixel using bilinear interpolation, where the correction strength is inversely proportional to the square of the radius of the pixel from the center of the image. This method can accurately correct distortion for different lens parameters, ensuring that both the center and edge areas of the image can be reasonably corrected, improving the realism and overall quality of the image.

[0057] Different methods are used for noise reduction processing in different situations. In the processing of video frames in rainy and snowy weather, the multi-scale Retinex algorithm is used to separate the rain line noise layer, and then the non-local mean filter is used to suppress the noise layer, effectively removing the noise interference in bad weather. For areas with sudden changes in illumination, dynamic histogram equalization based on the optical flow method is used to balance the brightness differences between adjacent frames and improve the video picture quality problems caused by changes in illumination.

[0058] In terms of overall optimization, the pre-processed video frames are verified by edge gradient histogram. If the inter-frame gradient change rate exceeds the preset threshold T1 (0.12≤T1≤0.18), it will trigger iterative optimization of video stabilization and distortion correction. This feedback mechanism can continuously adjust processing parameters to adapt to different video content and scene changes, ensure that the output video frames achieve the best processing effect, and provide reliable data support for subsequent intelligent traffic analysis.

[0059] Further, the step of extracting feature points in the video frame by using the SIFT algorithm includes: extracting feature points of the video frame by using the SIFT algorithm, setting the number of Gaussian pyramid layers to 4, the scale factor of each layer to 1.6, and the threshold value of the number of feature points extracted per frame to 500; The steps of calculating the homography matrix between adjacent frames include: calculating the homography matrix between adjacent frames based on the RANSAC algorithm, setting the number of iterations to 2000, the reprojection error threshold to 1.5 pixels, and screening feature point pairs with a matching degree ≥ 85%; The calculation formula of the dynamic jitter judgment threshold T satisfies the following formula: T=0.025×H+3, where H is the flight altitude of the drone, in meters, and T is the maximum pixel offset allowed for jitter, in pixels. When the translation component of the homography matrix between adjacent frames exceeds the threshold T, affine transformation compensation is triggered. Among them, after the video frame is affine transformed, the effective area of ​​the cropping center is 80% of the original frame width and 80% of the original frame height, and the coordinates of the retained area are [0.1W: 0.9W, 0.1H: 0.9H], where W and H are the width and height of the original frame respectively; the video frame after stabilization is output, the feature point matching error after stabilization is ≤1.5 pixels, and the processing speed is ≥25 frames / second.

[0060] In the above embodiment, in terms of feature point extraction, the number of Gaussian pyramid layers is set to 4 and the scale factor of each layer is set to 1.6, and the threshold of the number of feature points extracted per frame is set to 500, which can effectively control the amount of calculation while accurately and comprehensively capturing the key feature information in the video frame. The appropriate number of pyramid layers and scale factors ensure that feature points can be found at different scales, and the number threshold of 500 ensures the sufficiency of feature points, laying a solid foundation for the subsequent calculation of the relationship between adjacent frames, and improving the accuracy and reliability of feature matching.

[0061] When calculating the homography matrix between adjacent frames, the RANSAC algorithm is used with 2000 iterations, a reprojection error threshold of 1.5 pixels, and feature point pairs with a matching degree of ≥85% to effectively remove mismatched points and greatly improve the accuracy of homography matrix calculation. A large number of iterations gives the algorithm enough opportunities to find the optimal solution, and strict reprojection error thresholds and matching degree requirements ensure the quality of matching point pairs, thereby ensuring that the transformation relationship between adjacent frames can be accurately determined, providing a reliable basis for eliminating dynamic jitter.

[0062] The dynamic jitter determination threshold T is calculated according to the formula T = 0.025×H + 3. This adaptive adjustment method based on the flight altitude of the drone fully considers the differences in video jitter characteristics at different flight altitudes. The higher the flight altitude, the greater the allowed jitter pixel offset, which is in line with the actual situation. It enables accurate determination of jitter at various flight altitudes and timely triggering of affine transformation compensation, significantly improving the adaptability and robustness of video stabilization processing.

[0063] After the video frame is affine transformed, the effective area of ​​the crop center is 80% of the width and height of the original frame, and the coordinates of the retained area are [0.1W: 0.9W, 0.1H: 0.9H]. This operation effectively removes invalid or interfering areas such as edge distortion that may be caused by image stabilization processing, highlights the key information in the video, and improves the effectiveness and accuracy of video analysis. At the same time, the feature point matching error after stabilization is ≤1.5 pixels and the processing speed is ≥25 frames / second, which ensures the high quality and real-time performance of the video frame after stabilization. Low error ensures the stability and continuity of the video picture, and high processing speed meets the requirements of video processing efficiency in application scenarios such as real-time traffic monitoring, providing stable, efficient and reliable video data support for subsequent intelligent traffic analysis.

[0064] Further, the method for generating a correction grid includes: generating a dynamic correction grid according to a lens focal length f, in units of mm, and a field of view angle θ, in units of degrees, wherein a grid node spacing d, in units of mm, is calculated according to a formula d=0.12f×tan(θ / 2), wherein a value range of θ is 120°≤θ≤180°, and a value range of f is 2.8mm≤f≤8mm; The method for correcting the video frame pixel by pixel by bilinear interpolation includes: calculating the normalized center distance r of each pixel, satisfying , where (x c ,y c ) is the pixel coordinate, W and H are the width and height of the video frame respectively; the correction strength coefficient is generated according to the distance r , bilinear interpolation is used to calculate the offset at the correction grid node pixel by pixel, the offset Δ x , Δ y satisfy , ; Map the corrected pixel coordinates to (x c +Δ x ,y c +Δ y ), and bilinear interpolation sampling is used for non-integer coordinate pixels, and the interpolation weight is allocated according to the inverse of the coordinate distance of the four adjacent pixels.

[0065] In the above embodiment, the method of dynamically generating the correction grid can closely fit the parameter characteristics of different types of fisheye lenses. For lenses with different focal lengths and field of view angles, an adaptive grid can be generated, providing a basic framework for accurate distortion correction, ensuring that the correction grid can fully reflect the actual situation of lens distortion, and greatly improving the pertinence and accuracy of the correction.

[0066] The advantage of the method of correcting the video frame pixel by pixel through bilinear interpolation is that the correction strength coefficient is related to the distance from the pixel to the center of the image, which conforms to the law of fisheye lens distortion, that is, the closer to the edge of the image, the more serious the distortion. The coefficient can be used to reasonably correct pixels at different positions. The use of bilinear interpolation ensures that when calculating the offset at the correction grid node pixel by pixel, the information of adjacent pixels can be fully utilized, so that the corrected pixel transition is natural and smooth, avoiding the jagged or blocky effects in the correction process. The processing method for non-integer coordinate pixels further improves the accuracy of the correction, ensuring that each pixel can be accurately corrected, and ultimately significantly improves the quality of video frame distortion correction, making the video picture more realistic and distortion-free, and providing high-quality image data for subsequent tasks such as target recognition in road traffic video analysis.

[0067] Furthermore, the method of separating the rain line noise layer using the multi-scale Retinex algorithm includes: Set the Gaussian kernel size to [15, 30, 45], corresponding to the standard deviation σ = [15, 30, 45], and separate the noise layer I by the following formula noise : in, is the sth Gaussian kernel, σ s is the corresponding standard deviation, x and y represent pixel coordinates; Suppressing the noise floor includes the following steps: Calculate the noise layer pixel suppression weight W as , perform non-local mean filtering on the noise layer, the search window size is 7×7, unit: pixel, the similarity block size is 3×3, unit: pixel, the filtering parameter h=0.8×σ_n, σ_n is the noise variance; The illumination mutation area is detected by the following method: the motion vectors Δx and Δy of adjacent frames are calculated based on the optical flow method. Pixels are determined as areas of sudden illumination changes; The dynamic histogram equalization method includes: dividing the illumination mutation area into local windows, and the window size is , unit: pixel, contrast enhancement coefficient within the window , the brightness constraint after equalization is , where μ is the mean brightness in the window, L inRepresents the brightness value of the original input pixel, L out Represents the output pixel brightness value after equalization.

[0068] In the above embodiment, the method of separating the rain line noise layer by the multi-scale Retinex algorithm can effectively capture the rain line noise characteristics at different scales. The combination of multiple Gaussian kernels of different scales can adapt to the complex and changeable distribution of rain lines, comprehensively and accurately separate the rain line noise from the video frame, lay a good foundation for subsequent noise suppression, and significantly improve the accuracy and comprehensiveness of noise separation. The method of suppressing the noise layer generates weights by using the characteristics of the noise layer pixels themselves, which can suppress noise in a targeted manner. At the same time, the non-local mean filter is processed based on the similarity of image blocks, which can effectively remove noise while better retaining image detail information. The appropriate window size and filter parameter settings balance the noise suppression effect and image detail retention, so that the video picture is still clear after noise reduction, and the visual quality of the video is improved. The illumination mutation area detection method can accurately capture the pixel motion anomaly caused by illumination changes. The optical flow method can effectively reflect the motion of objects in the image. Using it for illumination mutation detection can quickly and accurately locate the illumination mutation area, provide a basis for subsequent targeted processing, avoid unnecessary processing of normal areas, and improve processing efficiency and accuracy. The dynamic histogram equalization method can dynamically adjust the processing parameters according to the actual motion of the illumination mutation area, and realize adaptive brightness equalization of the local area. The window size and contrast enhancement coefficient are related to the motion vector, so that the processing is more in line with the characteristics of the illumination mutation area, while enhancing the contrast and ensuring the rationality of the brightness adjustment, effectively balancing the brightness difference between adjacent frames, improving the visual effect of the video in the scene of illumination change, and providing more stable and clear image data for road traffic video analysis.

[0069] Based on the same inventive concept, the present invention also provides an artificial intelligence-based traffic flow trajectory generation system, which can be a personal computer, a server, or other devices that implement the aforementioned artificial intelligence-based traffic flow trajectory generation method.

[0070] See also Figure 3 As shown, the traffic flow trajectory generation system based on artificial intelligence includes: A video acquisition module, which is used to acquire road traffic videos; A preprocessing module, which is used to preprocess the road traffic video; An identification and tracking module, which is used to identify and track traffic participants in the pre-processed road traffic video using a target detection algorithm and a multi-target tracking algorithm; A trajectory generation module extracts the historical trajectory information of each traffic participant and generates the trajectory of each traffic participant on the road traffic video; A trajectory editing module, which is used to generate a trajectory of the traffic participant object with abnormal identification and tracking by using a trajectory editing algorithm; The trajectory editing algorithm includes: Drawing a static object area in the preprocessed road traffic video, and removing traffic participants that are always in the static object area; When identifying and tracking traffic participants in road traffic videos, for different traffic participants identified under the same ID, the traffic participant with the highest appearance frequency is selected as the traffic participant of the ID; Eliminate traffic participants whose historical trajectory information is less than the preset number; Based on the historical trajectory information of the remaining traffic participants in the road traffic video, the trajectory of each traffic participant is regenerated.

[0071] All relevant contents of each step involved in the embodiment of the aforementioned artificial intelligence-based traffic flow trajectory generation method can be referred to the functional description of the functional modules corresponding to the system in the embodiment of the present application, and will not be repeated here.

[0072] The division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0073] In the drawings of the system embodiment provided by the present invention, the connection relationship between modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0074] The present invention also provides an electronic device, comprising: at least one processor, and a memory connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the above-mentioned traffic flow trajectory generation method based on artificial intelligence. The electronic device can be any terminal device including a mobile phone, a laptop computer, a desktop computer, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), a vehicle-mounted computer, etc.

[0075] The present invention also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned traffic flow trajectory generation method based on artificial intelligence is implemented.

[0076] Through the description of the above implementation mode, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present invention, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, RandomAccess Memory), a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0077] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A method for generating traffic flow trajectories based on artificial intelligence, characterized in that: include: Obtain road traffic videos; Preprocess road traffic videos; The YOLOv4 target detection algorithm and DeepSort multi-target tracking algorithm are used to identify and track traffic participants in pre-processed road traffic videos. Extract the historical trajectory information of each traffic participant and generate the trajectory of each traffic participant on the road traffic video; For a traffic participant object with abnormal identification and tracking, a trajectory editing algorithm is used to generate a trajectory of the traffic participant object with abnormal identification and tracking; The trajectory editing algorithm includes: Drawing a static object area in the preprocessed road traffic video, and removing traffic participants that are always in the static object area; When identifying and tracking traffic participants in road traffic videos, for different traffic participants identified under the same ID, the traffic participant with the highest appearance frequency is selected as the traffic participant of the ID; Eliminate traffic participants whose historical trajectory information is less than the preset number; Based on the historical trajectory information of the remaining traffic participants in the road traffic video, the trajectory of each traffic participant is regenerated.

2. The method for generating traffic flow trajectories based on artificial intelligence according to claim 1, characterized in that: The method of regenerating the trajectory of each traffic participant object includes: Crop the edges of road traffic videos; Using a trajectory smoothing algorithm to generate the trajectory of the traffic participant object in multiple adjacent frames of the road traffic video; Calculating the size of the recognition and tracking detection frame of the traffic participant object in each frame of the road traffic video, and then normalizing the recognition and tracking detection frame of the traffic participant object to the same size; A cubic B-spline fitting algorithm is used to merge all the generated trajectories of the traffic participant objects.

3. The method for generating traffic flow trajectories based on artificial intelligence as claimed in claim 2, characterized in that: The method for regenerating the trajectory of each traffic participant object also includes: Verify the merged trajectories to find out the trajectories and traffic participants whose starting point and end point are at least one not at the edge of the road traffic video image; For the trajectories and their traffic participants whose at least one starting point and end point is not at the edge of the road traffic video image, find at least one pair of traffic participants of the same type in chronological order, in which the end point of one traffic participant object is at the shortest distance from the starting point of another traffic participant object, and merge the trajectories of the pair of traffic participants of the same type.

4. The method for generating traffic flow trajectories based on artificial intelligence as claimed in claim 2, characterized in that: The trajectory smoothing algorithm is a Catmull-Rom algorithm, wherein a tension parameter tension is dynamically calculated according to the speed of the traffic participant object, and the value of the tension parameter tension is inversely proportional to the speed of the traffic participant object.

5. The method for generating traffic flow trajectories based on artificial intelligence according to claim 1, characterized in that: After using the target detection algorithm and the multi-target tracking algorithm to identify and track the traffic participants in the road traffic video after preprocessing, the recognition results are also verified to identify and track abnormal traffic participants. The verification methods include: identification object quantity comparison method, identification object coordinate analysis method, and visualization comparison method.

6. The method for generating traffic flow trajectories based on artificial intelligence according to claim 1, characterized in that: Preprocessing of road traffic videos includes: Video stabilization: Video stabilization based on multi-scale feature point matching. The SIFT algorithm is used to extract feature points in the video frame and calculate the homography matrix between adjacent frames. The video frame is affine transformed to eliminate dynamic jitter. The threshold for dynamic jitter is adaptively adjusted according to the flight altitude of the drone. Distortion correction processing: Adopting adaptive fisheye distortion correction algorithm, dynamically generating correction grid according to lens focal length and field of view parameters, and correcting video frames pixel by pixel through bilinear interpolation, where the correction strength is inversely proportional to the square of the radius of the pixel from the center of the image; and Noise reduction based on spatiotemporal hybrid filtering: Multi-scale Retinex algorithm is used to separate the rain line noise layer of rainy and snowy weather video frames, and the noise layer is suppressed by non-local mean filtering; dynamic histogram equalization based on optical flow method is used for areas with sudden changes in illumination to balance the brightness differences between adjacent frames; The preprocessed video frames are verified by edge gradient histograms. If the inter-frame gradient change rate exceeds a preset threshold value T1, 0.12≤T1≤0.18, iterative optimization of video stabilization and distortion correction is triggered.

7. The method for generating traffic flow trajectories based on artificial intelligence as claimed in claim 6, characterized in that: The step of extracting feature points in the video frame by using the SIFT algorithm includes: extracting feature points of the video frame by using the SIFT algorithm, setting the number of Gaussian pyramid layers to 4, the scale factor of each layer to 1.6, and the threshold value of the number of feature points extracted per frame to 500; The steps of calculating the homography matrix between adjacent frames include: calculating the homography matrix between adjacent frames based on the RANSAC algorithm, setting the number of iterations to 2000, the reprojection error threshold to 1.5 pixels, and screening feature point pairs with a matching degree ≥ 85%; The calculation formula of the dynamic jitter judgment threshold T satisfies the following formula: T=0.025×H+3, where H is the flight altitude of the drone, in meters, and T is the maximum pixel offset allowed for jitter, in pixels. When the translation component of the homography matrix between adjacent frames exceeds the threshold T, affine transformation compensation is triggered. Among them, after the video frame is affine transformed, the effective area of ​​the cropping center is 80% of the original frame width and 80% of the original frame height, and the coordinates of the retained area are [0.1W: 0.9W, 0.1H: 0.9H], where W and H are the width and height of the original frame respectively; the video frame after stabilization is output, the feature point matching error after stabilization is ≤1.5 pixels, and the processing speed is ≥25 frames / second.

8. The method for generating traffic flow trajectories based on artificial intelligence as claimed in claim 6, characterized in that: The method for generating a correction grid includes: generating a dynamic correction grid according to a lens focal length f, in units of mm, and a field angle θ, in units of degrees, wherein a grid node spacing d, in units of mm, is calculated according to a formula d=0.12f×tan(θ / 2), wherein a value range of θ is 120°≤θ≤180°, and a value range of f is 2.8mm≤f≤8mm; The method for correcting the video frame pixel by pixel by bilinear interpolation includes: calculating the normalized center distance r of each pixel, satisfying , where (x c ,y c ) is the pixel coordinate, W and H are the width and height of the video frame respectively; the correction strength coefficient is generated according to the distance r , bilinear interpolation is used to calculate the offset at the correction grid node pixel by pixel, the offset Δ x , Δ y satisfy ; Map the corrected pixel coordinates to (x c +Δ x ,y c +Δ y ), and bilinear interpolation sampling is used for non-integer coordinate pixels, and the interpolation weights are allocated according to the inverse of the coordinate distances of the four adjacent pixels.

9. The method for generating traffic flow trajectories based on artificial intelligence as claimed in claim 6, characterized in that: The method of separating the rain line noise layer by the multi-scale Retinex algorithm includes: Set the Gaussian kernel size to [15, 30, 45], corresponding to the standard deviation σ = [15, 30, 45], and separate the noise layer I by the following formula noise : in, is the sth Gaussian kernel, σ s is the corresponding standard deviation, x and y represent pixel coordinates; Suppressing the noise floor includes the following steps: Calculate the noise layer pixel suppression weight W as , perform non-local mean filtering on the noise layer, the search window size is 7×7, unit: pixel, the similarity block size is 3×3, unit: pixel, the filtering parameter h=0.8×σ_n, σ_n is the noise variance; The illumination mutation area is detected by the following method: the motion vectors Δx and Δy of adjacent frames are calculated based on the optical flow method. Pixels are determined as areas of sudden illumination changes; The dynamic histogram equalization method includes: dividing the illumination mutation area into local windows, and the window size is , unit: pixel, contrast enhancement coefficient within the window , the brightness constraint after equalization is , where μ is the mean brightness in the window, L in Represents the brightness value of the original input pixel, L out Represents the output pixel brightness value after equalization.

10. A traffic flow trajectory generation system based on artificial intelligence, characterized in that: include: A video acquisition module, which is used to acquire road traffic videos; A preprocessing module, which is used to preprocess the road traffic video; The recognition and tracking module is used to identify and track traffic participants in the pre-processed road traffic video using the YOLOv4 target detection algorithm and the DeepSort multi-target tracking algorithm; A trajectory generation module, which is used to extract the historical trajectory information of each traffic participant and generate the trajectory of each traffic participant on the road traffic video; A trajectory editing module, which is used to generate a trajectory of the traffic participant object with abnormal identification and tracking by using a trajectory editing algorithm; The trajectory editing algorithm includes: Drawing a static object area in the preprocessed road traffic video, and removing traffic participants that are always in the static object area; When identifying and tracking traffic participants in road traffic videos, for different traffic participants identified under the same ID, the traffic participant with the highest appearance frequency is selected as the traffic participant of the ID; Eliminate traffic participants whose historical trajectory information is less than the preset number; Based on the historical trajectory information of the remaining traffic participants in the road traffic video, the trajectory of each traffic participant is regenerated.

Images