A method for calculating and restoring vehicle driving trajectory

Abstract

The present invention discloses a method for calculating and restoring a vehicle driving trajectory, and belongs to the technical field of vehicle driving trajectory restoration. The method solves the problem that the traditional vehicle driving trajectory restoration method in the prior art needs to manually call historical data analysis and the amount of analysis data is large when performing traffic road condition analysis; the present invention uses a common camera on the side of the road to perform multi-attribute data recognition and calculation on vehicles traveling on the road to obtain driving trajectory point data; according to the vehicle driving direction and driving trajectory point data, the driving angle of the vehicle head is calculated, and the two-dimensional coordinate system and the real-world coordinate system are converted to obtain the vehicle's driving speed; the collected driving trajectory point data is packaged and uploaded to the database, and finally the vehicle trajectory point data of a certain time period in the database is subjected to curve fitting processing to draw the restored vehicle driving trajectory. The present invention effectively improves the accuracy and efficiency of vehicle driving trajectory restoration, and can be applied to traffic road condition analysis.

Description

Technical Field

[0001] The invention relates to a vehicle driving track calculation and restoration method, and belongs to the technical field of vehicle driving track restoration. Background Art

[0002] At present, vehicle driving trajectories are often used in traffic supervision scenarios such as vehicle driving analysis, accident analysis, and road condition analysis. The vehicle driving trajectory is generally collected by saving the video of the camera on the road under traffic supervision, and collecting multi-attribute data of passing vehicles through various sensor devices. When a traffic accident occurs or the traffic conditions need to be analyzed, the saved historical data is retrieved for manual analysis, but the above method undoubtedly has a large amount of data and high cost, and cannot provide detailed and intuitive feedback on traffic conditions and the specific trajectories of all vehicles.

[0003] In summary, a method is needed that uses a low-data volume, ordinary road-side camera to perform multi-attribute data recognition on vehicles traveling on the road to restore the subsequent vehicle driving trajectory. Summary of the invention

[0004] A brief overview of the present invention is provided below in order to provide a basic understanding of certain aspects of the present invention. It should be understood that this overview is not an exhaustive overview of the present invention. It is not intended to identify key or important parts of the present invention, nor is it intended to limit the scope of the present invention. Its purpose is merely to present certain concepts in a simplified form as a prelude to a more detailed description discussed later.

[0005] In view of this, in order to solve the problem that the traditional vehicle driving trajectory restoration method in the prior art needs to manually call historical data analysis and the amount of analysis data is large when performing traffic road condition analysis, the present invention provides a vehicle driving trajectory calculation and restoration method.

[0006] The technical solution is as follows: A method for calculating and restoring a vehicle driving trajectory comprises the following steps:

[0007] S1. Connect to the ordinary camera on the road side, identify and collect multi-attribute data of the moving vehicles, use the improved YOLO V8 network framework to detect vehicles, use the improved B-CNN network model to classify the vehicle model and color, use the scaling factor to correct the tilt of the vehicle image, use the convolutional neural network to extract the license plate image, use the convolutional neural network to recognize the license plate characters on the license plate image, and obtain the output result of vehicle detection;

[0008] Specifically: the output results of vehicle detection include vehicle position coordinates, vehicle model classification, vehicle color classification and license plate number detection results and their confidence levels;

[0009] S2. Use DeepSort algorithm to build a vehicle tracking model, input the output result of vehicle detection, and output the target vehicle tracking ID;

[0010] S3. Set a vehicle driving detection area on the road driving area in the camera video screen, calculate the coordinates of the vehicle center point, use the ray method to determine whether the vehicle is in the vehicle driving detection area according to the coordinates of the vehicle center point, and eliminate vehicles outside the driving detection area;

[0011] S4. According to the two-dimensional moving coordinate points of the vehicle and the time difference, the front driving angle is calculated, and then according to the parameters of the ordinary camera on the road side, the two-dimensional moving coordinate points of the vehicle are converted into real-world coordinate points to calculate the driving speed of the vehicle;

[0012] S5. The driving trajectory point data is structured and packaged into a JSON message, i.e., structured data, and the structured data is sent and saved to a database on a remote server;

[0013] Specifically: the driving trajectory point data includes time, camera ID number, camera resolution, vehicle ID number, vehicle location coordinates, vehicle model, vehicle head driving angle, vehicle speed, vehicle body color and license plate number;

[0014] S6. Select the driving trajectory point data in the database, and fit the driving trajectory point data with the same target vehicle tracking ID to the upper left corner coordinate set of the corresponding vehicle on the rectangular detection box to obtain the fitting curve function of the vehicle driving trajectory;

[0015] S7. Select a video screen of a certain camera as a canvas, draw a fitting curve of the vehicle's driving trajectory on the canvas, mark and display the data of each driving trajectory point of the vehicle, the driving angle and the driving speed of the vehicle head, and obtain the restored vehicle driving trajectory.

[0016] Furthermore, the step S1 includes the following steps:

[0017] S11. Use the improved YOLO V8 network framework to detect vehicles through the target detection algorithm to obtain vehicle detection images and vehicle position coordinates;

[0018] S12. According to the vehicle detection image, the improved B-CNN network model is used to perform vehicle model classification and vehicle color classification to obtain vehicle model classification and vehicle color classification;

[0019] S13. According to the vehicle detection image, the scaling factor is corrected, and then the license plate image is extracted by the convolutional neural network for license plate location, and then the convolutional neural network is used to recognize the license plate image, and the license plate number detection result and its confidence are output;

[0020] In the S11, the YOLO V8 network framework is improved, the number of detection target categories of the YOLO V8 network framework is set to 1, the YOLO V8 network framework is trained and adjusted, the real-time video stream of the camera device is connected, the video decoding operation is performed, the video is divided into frame-by-frame pictures, that is, the vehicle detection image is input into the YOLO V8 network framework, and the vehicle detection is performed on each frame of the picture through the target detection algorithm of the YOLO V8 network framework to obtain the vehicle position coordinates, that is, the rectangular detection frame coordinates [left, top, width, height], wherein left is the upper left corner horizontal coordinate of the rectangular detection frame, top is the upper left corner vertical coordinate of the rectangular detection frame, width is the width of the rectangular detection frame, and height is the height of the rectangular detection frame;

[0021] In the S12, the B-CNN network model is improved, the sub-network B of the B-CNN network model is replaced with the ResNet network, the sub-network A uses the VGGNet-16 network, an outer product operation is added, the features of the Conv4_3 network layer of the B-CNN network model are outer-producted with the features of the Conv5_3 network layer, so that the shallow features and the deep features are combined, and the vehicle model and vehicle color are input to obtain the vehicle model classification and vehicle color classification;

[0022] In S13, according to the aspect ratio of the license plate bounding box, the target vehicle image is tilt-corrected using a scaling factor, and then the license plate is recognized using a convolutional neural network, and the license plate position and scale information are output to obtain a license plate image;

[0023] A convolutional neural network model is used to perform license plate character recognition on license plate images. The number of prediction boxes of the convolutional neural network model is set to 2, and the aspect ratio of the license plate bounding box is 3:1. The input size preprocessing of the convolutional neural network model is set to 240*80. The license plate image is input into the convolutional neural network model for recognition, and the license plate number detection result and its confidence are output. The confidence threshold of the license plate number detection result is set. If the confidence of the detection result is higher than the confidence threshold, the detection result is directly output. If the confidence of the detection result is lower than the confidence threshold, the detection result is discarded.

[0024] Furthermore, in S2, the DeepSort algorithm is used to construct a vehicle tracking model, and the output result of the vehicle detection is used as the input of the vehicle tracking model, and the target vehicle tracking ID and the predicted target tracking frame are output. Cascade matching is performed through the DeepSort algorithm, and the intersection and union ratio of the rectangular detection frame coordinates and the predicted target tracking frame are input and combined to obtain the cascade matching result, which is input into the Hungarian algorithm for linear assignment to directly associate the rectangular detection frame coordinates with the predicted target tracking frame The target vehicle tracking ID in the current frame is matched between frames.

[0025] Furthermore, in S3, a two-dimensional coordinate system is established to determine whether the vehicle is within the vehicle driving detection area by the coordinates (x, y) of the center point of the vehicle;

[0026] Get the number of intersections between the horizontal line where the center point coordinates (x, y) of the vehicle are located and the edges of the polygon of the vehicle driving detection area. If the number of intersections is an odd number, the vehicle is within the vehicle driving detection area, otherwise the vehicle is outside the vehicle driving detection area.

[0027] Furthermore, the step S4 includes the following steps:

[0028] S41. Calculate the vehicle's front travel angle angle according to the vehicle's two-dimensional moving coordinates and the time difference t;

[0029] S42. According to the parameters of the ordinary camera on the road side, the two-dimensional moving coordinate points of the vehicle are converted into real-world coordinate points to calculate the vehicle's speed;

[0030] In the above S41, the two-dimensional moving coordinate points of the vehicle include the coordinates of the first moving point of the vehicle (x1, y1) and the coordinates of the second moving point of the vehicle (x2, y2);

[0031] The front driving angle angle is the clockwise angle between the vehicle driving direction and the negative direction of the y-axis;

[0032] The negative direction of the y-axis is represented by vector a (v1x, v1y), and the forward and backward movement points of the vehicle in the direction of travel of the vehicle are represented by vector b (v2x, v2y):

[0033] If the absolute value of vector a |a| or the absolute value of vector b |b| is equal to 0, the front driving angle angle is 0, otherwise the front driving angle angle is expressed as: angle = 180*acos(ab / (|a|*|b|)) / 3.1415926, where acos is the arc cosine function, ab is the cross product ab of vector a and vector b, and if the horizontal coordinate x2 of the first moving point of the vehicle is less than the horizontal coordinate x1 of the second moving point of the vehicle, the front driving angle angle is expressed as: angle = 360-angle;

[0034] In the S42, the two-dimensional moving coordinate points of the vehicle are converted into real-world coordinate points, and a real-world coordinate system is established. According to the time difference t between the first moving point coordinates (x1, y1) of the vehicle and the second moving point coordinates (x2, y2), the camera resolution width camera_width, the camera resolution height camera_height, the camera installation height h, the camera overhead angle (alpha), the horizontal field of view (hFoV) and the vertical field of view (vFoV), the first moving point coordinates (x1, y1) of the vehicle in the image are converted into first normalized coordinates (x1_normalized, y1_normalized) relative to the center of the image, and the second moving point coordinates (x2, y2) of the vehicle in the image are converted into second normalized coordinates (x2_normalized, y2_normalized) relative to the center of the image;

[0035] According to the first normalized coordinates (x1_normalized, y1_normalized) and the second normalized coordinates (x2_normalized, y2_normalized), the first real-world coordinates (X1, Y1) corresponding to the first moving point coordinates (x1, y1) of the vehicle and the second real-world coordinates (X2, Y2) corresponding to the second moving point coordinates (x2, y2) of the vehicle are calculated;

[0036] The driving speed speed is calculated according to the first real world coordinate (X1, Y1), the second real world coordinate (X2, Y2) and the time difference t.

[0037] Furthermore, in S6, the driving trajectory point data within a certain camera in a certain time period is selected from the database, and for the driving trajectory point data with the same target vehicle tracking ID, the upper left corner coordinate set {(x1, y1), (x2, y2), ..., (x n ,y n )} Perform k-order polynomial fitting to obtain the fitting curve function f(x) of the vehicle's driving trajectory.

[0038] The beneficial effects of the present invention are as follows: the present invention provides a method for calculating and restoring a vehicle driving trajectory, which specifically includes two parts: vehicle trajectory point data collection and vehicle trajectory point data restoration. For vehicle trajectory point data collection, an ordinary camera on the road side is connected to perform multi-attribute data recognition and calculation on vehicles traveling on the road to obtain driving trajectory point data, and then the calculated and collected vehicle driving trajectory point data in the current camera is packaged into a JSON message, and uploaded and saved to the database of a cloud platform server at regular intervals; for vehicle trajectory point data simulation and restoration, the vehicle trajectory point data of a certain time period in the database is subjected to curve fitting processing, and then the driving trajectory is drawn and restored on the video screen of the camera, thereby realizing a detailed analysis of the vehicle driving trajectory in a low-cost and low-data-volume manner, thereby facilitating real-time vehicle supervision. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0040] Figure 1 A schematic diagram of a process for calculating and restoring a vehicle driving trajectory;

[0041] Figure 2 A schematic diagram for restoring the vehicle's driving trajectory;

[0042] Figure 3 A schematic diagram of an embodiment flow of using the improved B-CNN network model to detect vehicles;

[0043] Figure 4 The figure is a flow chart of an embodiment of license plate recognition. DETAILED DESCRIPTION

[0044] In order to make the technical solutions and advantages of the embodiments of the present invention more clearly understood, the exemplary embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0045] refer to Figure 1-Figure 4 The present embodiment is described in detail, a method for calculating and restoring a vehicle driving trajectory, specifically comprising the following steps:

[0046] S1. Connect to the ordinary camera on the road side, identify and collect multi-attribute data of the moving vehicles, use the improved YOLO V8 network framework to detect vehicles, use the improved B-CNN network model to classify the vehicle model and color, use the scaling factor to correct the tilt of the vehicle image, use the convolutional neural network to extract the license plate image, use the convolutional neural network to recognize the license plate characters on the license plate image, and obtain the output result of vehicle detection;

[0047] Specifically: the output results of vehicle detection include vehicle position coordinates, vehicle model classification, vehicle color classification and license plate number detection results and their confidence levels;

[0048] S2. Use DeepSort algorithm to build a vehicle tracking model, input the output result of vehicle detection, and output the target vehicle tracking ID;

[0049] S3. Set a vehicle driving detection area on the road driving area in the camera video screen, calculate the coordinates of the vehicle center point, use the ray method to determine whether the vehicle is in the vehicle driving detection area according to the coordinates of the vehicle center point, and eliminate vehicles outside the driving detection area;

[0050] S4. According to the two-dimensional moving coordinate points of the vehicle and the time difference, the front driving angle is calculated, and then according to the parameters of the ordinary camera on the road side, the two-dimensional moving coordinate points of the vehicle are converted into real-world coordinate points to calculate the driving speed of the vehicle;

[0051] S5. The driving trajectory point data is structured and packaged into a JSON message, i.e., structured data, and the structured data is sent and saved to a database on a remote server;

[0052] Specifically: the driving trajectory point data includes time, camera ID number, camera resolution, vehicle ID number, vehicle location coordinates, vehicle model, vehicle head driving angle, vehicle speed, vehicle body color and license plate number;

[0053] S6. Select the driving trajectory point data in the database, and fit the driving trajectory point data with the same target vehicle tracking ID to the upper left corner coordinate set of the corresponding vehicle on the rectangular detection box to obtain the fitting curve function of the vehicle driving trajectory;

[0054] S7. Select a video screen of a certain camera as a canvas, draw a fitting curve of the vehicle's driving trajectory on the canvas, mark and display the data of each driving trajectory point of the vehicle, the driving angle and the driving speed of the vehicle head, and obtain the restored vehicle driving trajectory.

[0055] Furthermore, the step S1 includes the following steps:

[0056] S11. Use the improved YOLO V8 network framework to detect vehicles through the target detection algorithm to obtain vehicle detection images and vehicle position coordinates;

[0057] S12. According to the vehicle detection image, the improved B-CNN network model is used to perform vehicle model classification and vehicle color classification to obtain vehicle model classification and vehicle color classification;

[0058] S13. According to the vehicle detection image, the scaling factor is corrected, and then the license plate image is extracted by the convolutional neural network for license plate location, and then the convolutional neural network is used to recognize the license plate image, and the license plate number detection result and its confidence are output;

[0059] In the S11, the YOLO V8 network framework is improved, the number of detection target categories of the YOLO V8 network framework is set to 1, the YOLO V8 network framework is trained and adjusted, the real-time video stream of the camera device is accessed, a video decoding operation is performed, the video is divided into frame-by-frame pictures, i.e., vehicle detection images are input into the YOLO V8 network framework, and the vehicle detection is performed on each frame of the picture through the target detection algorithm of the YOLO V8 network framework to obtain the vehicle position coordinates, i.e., the rectangular detection frame coordinates [left, top, width, height], wherein left is the upper left corner horizontal coordinate of the rectangular detection frame, top is the upper left corner vertical coordinate of the rectangular detection frame, width is the width of the rectangular detection frame, and height is the height of the rectangular detection frame;

[0060] In the S12, the B-CNN network model is improved, the sub-network B of the B-CNN network model is replaced with the ResNet network, the sub-network A uses the VGGNet-16 network, an outer product operation is added, the features of the Conv4_3 network layer of the B-CNN network model are outer-producted with the features of the Conv5_3 network layer, so that the shallow features and the deep features are combined, and the vehicle model and vehicle color are input to obtain the vehicle model classification and vehicle color classification;

[0061] In S13, according to the aspect ratio of the license plate boundary frame, the target vehicle image is tilt-corrected using a scaling factor, and the license plate image is adjusted to the front view to obtain the license plate image;

[0062] Scaling factor f sc It is expressed as:

[0063]

[0064] Among them, W V is the width of the license plate bounding box, H V is the height of the license plate bounding box, D min and D maxrepresent constants, which are 288 and 608 respectively;

[0065] The CR-NET convolutional neural network model is used to perform license plate character recognition on license plate images. The number of prediction boxes of the convolutional neural network model is set to 2, and the aspect ratio of the license plate bounding box is 3:1. The input size preprocessing of the convolutional neural network model is set to 240*80. The license plate image is input into the convolutional neural network model for recognition, and the license plate number detection result and its confidence are output. The confidence threshold of the license plate number detection result is set. If the confidence of the detection result is higher than the confidence threshold, the detection result is directly output. If the confidence of the detection result is lower than the confidence threshold, the detection result is discarded.

[0066] Specifically, in the target detection under the current traffic scene, it is only necessary to detect the vehicle to output its position scale information. The YOLO series algorithm has many iterative versions, such as V5, V6, V8, etc. Compared with other detection algorithms, such as SSD (single shot multi-frame detection) model and RetinaNet network structure, the network model framework of the YOLO series detection algorithm has better detection accuracy and speed. The input target image to be detected is extracted through the convolution layer feature capability, and finally the position, category and confidence of the detection target are regressed. Therefore, the present invention selects the YOLO V8 network framework for fine-tuning. The rectangular detection frame is the position range of the vehicle in the two-dimensional video screen;

[0067] For vehicle classification, the B-CNN network model has a very good effect on the fine-grained classification of targets. The two subnetworks of the B-CNN network model perform region detection and feature extraction respectively. The different choices of the two subnetworks determine different classification performances. The classification performance of the same subnetwork is better than that of the same subnetwork, the performance of the dual-line network is better than that of a single network, and the performance of the deep network is better than that of the shallow network. Figure 3 , Conv1_1 is the network layer, Max Pool 4 is the pooling operation;

[0068] The license plate detection result is the license plate position and scale information. At the same time, in order to prevent the license plate recognition accuracy from being affected by the actual traffic environment, only the planned vehicles close to the camera area are recognized. The subsequent trajectory point data can be matched front and back through the target vehicle tracking ID;

[0069] The traditional license plate recognition algorithm is divided into three steps: license plate positioning, license plate number character segmentation and license plate number character recognition. The image of a vehicle traveling on the road is small and is at a certain angle to the camera monitoring, which causes the license plate images obtained by license plate positioning to be mostly tilted. In addition, the quality of license plate number character segmentation greatly affects the results of character recognition. Therefore, it is necessary to optimize the traditional method, simplify the license plate recognition process, and improve the detection effect.

[0070] Further, in S2, a vehicle tracking model is constructed by using a DeepSort algorithm, the output result of the vehicle detection is used as the input of the vehicle tracking model, the target vehicle tracking ID and the predicted target tracking frame are output, cascade matching is performed by the DeepSort algorithm, the intersection and union ratio (IOU) of the rectangular detection frame coordinates and the predicted target tracking frame are input and combined to obtain a cascade matching result, which is input into the Hungarian algorithm for linear assignment to directly associate the rectangular detection frame coordinates with the predicted target tracking frame to perform inter-frame matching on the target vehicle tracking ID in the current frame;

[0071] Specifically, the main task of target tracking is to associate the detected targets between two frames, mark the target vehicle with an ID number, and obtain the target vehicle's motion trajectory;

[0072] The appearance position information of the target vehicle, namely the vehicle position coordinates, vehicle model and vehicle color, can be added to the calculation process of inter-frame matching. In this way, in the actual situation where the target vehicle is blocked by an object but the target vehicle appears again later, the target vehicle tracking ID between frames can be correctly matched. In the process of real-time detection and target tracking, the real-time target automatic tracking detection effect in the case of blocked targets can be improved. At the same time, the problem of frequent jumps between target vehicle tracking IDs is greatly reduced, thereby achieving the purpose of continuous tracking.

[0073] Furthermore, in S3, a two-dimensional coordinate system is established to determine whether the vehicle is within the vehicle driving detection area by the coordinates (x, y) of the center point of the vehicle;

[0074] The coordinates (x, y) of the vehicle center point are expressed as:

[0075] x=left+width / 2

[0076] y=top+height / 2

[0077] Get the number of intersections between the horizontal line where the center point coordinates (x, y) of the vehicle are located and the edges of the polygon of the vehicle driving detection area. If the number of intersections is an odd number, the vehicle is within the vehicle driving detection area, otherwise the vehicle is outside the vehicle driving detection area.

[0078] Furthermore, the step S4 includes the following steps:

[0079] S41. Calculate the vehicle's front travel angle angle according to the vehicle's two-dimensional moving coordinates and the time difference t;

[0080] S42. Convert the parameters of the ordinary camera on the road side, namely the given camera top-down angle alpha, horizontal field of view hFoV and vertical field of view vFoV, into radians, establish a real-world coordinate system, and calculate the vehicle's driving speed according to the front driving angle angle;

[0081] In the above S41, the two-dimensional moving coordinate points of the vehicle include the coordinates of the first moving point of the vehicle (x1, y1), the coordinates of the second moving point of the vehicle (x2, y2), and the front driving angle angle is the clockwise angle between the vehicle driving direction and the negative direction of the y-axis;

[0082] The negative direction of the y-axis is represented by vector a (v1x, v1y), and the forward and backward movement points of the vehicle in the direction of travel of the vehicle are represented by vector b (v2x, v2y):

[0083] The vector a(v1x, v1y) is represented as:

[0084] v1x=0

[0085] v1y=(-100)-y1

[0086] Among them, v1x is the horizontal coordinate of vector a, and v1y is the vertical coordinate of vector a;

[0087] The vector b(v2x, v2y) is represented as:

[0088] v2x=x2-x1

[0089] v2y=y2-y1

[0090] Wherein, (x1, y1) is the coordinate of the first moving point of the vehicle, and (x2, y2) is the coordinate of the second moving point of the vehicle;

[0091] The absolute value of vector a |a| is expressed as:

[0092] |a|=sqrt(v1x*v1x+v1y*v1y)

[0093] Among them, sqrt is the square root function;

[0094] The absolute value of vector b |b| is expressed as:

[0095] |b|=sqrt(v2x*v2x+v2y*v2y)

[0096] The cross product ab of vector a and vector b is expressed as:

[0097] ab=v1x*v2x+v1y*v2y

[0098] If the absolute value of vector a |a| or the absolute value of vector b |b| is equal to 0, the front driving angle angle is 0, otherwise the front driving angle angle is expressed as: angle = 180*acos(ab / (|a|*|b|)) / 3.1415926, where acos is the arc cosine function. If the horizontal coordinate x2 of the first moving point of the vehicle is less than the horizontal coordinate x1 of the second moving point of the vehicle, the front driving angle angle is expressed as: angle = 360-angle;

[0099] In the S42, the two-dimensional moving coordinate point of the vehicle is converted into a real-world coordinate point, and according to the time difference t between the coordinates of the first moving point (x1, y1) of the vehicle and the coordinates of the second moving point (x2, y2) of the vehicle, the camera resolution width camera_width, the camera resolution height camera_height, the camera installation height h, the camera overhead angle alpha, the horizontal field of view hFoV and the vertical field of view vFoV, the coordinates of the first moving point (x1, y1) of the vehicle in the image are converted into first normalized coordinates (x1_normalized, y1_normalized) relative to the center of the image, and the coordinates of the second moving point (x2, y2) of the vehicle in the image are converted into second normalized coordinates (x2_normalized, y2_normalized) relative to the center of the image;

[0100] The camera's top-down angle alpha is converted into radians as follows:

[0101] alpha=(alpha / 180)*3.1415926

[0102] The horizontal field of view angle hFoV is converted into radians as:

[0103] hFoV=(hFoV / 180)*3.1415926

[0104] The vertical field of view angle vFoV is converted into radians as:

[0105] vFoV=(vFoV / 180)*3.1415926

[0106] The first normalized coordinates (x1_normalized, y1_normalized) are expressed as:

[0107] x1_normalized=(2.0*x1-camera_width) / camera_width

[0108] y1_normalized=(camera_height-2.0*y1) / camera_height

[0109] The second normalized coordinates (x2_normalized, y2_normalized) are expressed as:

[0110] x2_normalized=(2.0*x2-camera_width) / camera_width

[0111] y2_normalized=(camera_height-2.0*y2) / camera_height

[0112] According to the first normalized coordinates (x1_normalized, y1_normalized) and the second normalized coordinates (x2_normalized, y2_normalized), the first real world coordinates (X1, Y1) corresponding to the first moving point coordinates (x1, y1) of the vehicle and the second real world coordinates (X2, Y2) corresponding to the second moving point coordinates (x2, y2) of the vehicle are obtained;

[0113] The first real-world coordinate (X1, Y1) is expressed as:

[0114] Y1=h / tan(alpha+y1_normalized*(vFoV / 2.0))

[0115] X1=Y1*tan(x1_normalized*(hFoV / 2.0))

[0116] Among them, tan is the sine function;

[0117] The second real-world coordinate (X2, Y2) is expressed as:

[0118] Y2=h / tan(alpha+y1_normalized*(vFoV / 2.0))

[0119] X2=Y2*tan(x1_normalized*(hFoV / 2.0))

[0120] The driving speed speed is calculated according to the first real world coordinate (X1, Y1), the second real world coordinate (X2, Y2) and the time difference t;

[0121] The driving speed is expressed as:

[0122] speed=sqrt((X2-X1)*(X2-X1)+(Y2-Y1)*(Y2-Y1)) / t.

[0123] Specifically, from the continuous multiple frames of the camera video, extract the current frame to record the time and all vehicle information, and then extract a frame after an interval of k frames to record the time and all vehicle information, and take out the vehicles with the same tracking ID in the two frames before and after. According to the coordinates of the first moving point of the vehicle on the two-dimensional screen of the video (x1, y1), the coordinates of the second moving point of the vehicle (x2, y2) and the time difference T, calculate the driving angle of the vehicle head angle, and then calculate the driving speed according to the camera parameters and the moving pixel points of the vehicle on the two-dimensional plane. The moving pixel points need to convert the coordinates in the two-dimensional coordinate system into the coordinates in the real world coordinate system according to the camera parameters;

[0124] In this embodiment, the specific content of the JSON message refers to Table 1, string indicates that the data type is a text type, object indicates that the data type is an object type, int indicates that the data type is an integer type, array[] indicates that the data type is an array type, and double indicates that the data type is a double floating point type;

[0125] Parameter name type Required illustrate deviceId string yes The device id where the software is located msgId string yes Message unique ID, 1 to 32 bytes type string yes Message Type time string yes Message time, date type time params object yes Specific parameter information sourceId string yes Data source id, camera device id camera_width int yes Camera image resolution wide camera_height int yes High resolution camera image roadId string no The id of the location segment list array[] yes List id string yes Target vehicle tracking id left int yes The horizontal coordinate of the upper left corner of the rectangular detection box top int yes The ordinate of the upper left corner of the rectangular detection box width int yes Width of the rectangular detection box height int yes Height of the rectangular detection box model int yes Vehicle type code speed double yes Driving speed, km / h angle double yes Front driving angle, range 0~360 color int no Body color coding licensePlate string no License plate number

[0126] Table 1

[0127] Furthermore, in S6, the driving trajectory point data within a certain camera in a certain time period is selected from the database, and for the driving trajectory point data with the same target vehicle tracking ID, the upper left corner coordinate set {(x1, y1), (x2, y2), ..., (x n ,y n )} Perform k-order polynomial fitting to obtain the fitting curve function of the vehicle's driving trajectory;

[0128] The fitting curve function f(x) of the vehicle's driving trajectory is expressed as:

[0129]

[0130] f(x)=a0+a1x+a2x 2 +...+a k x k

[0131] Among them, a k are k-order polynomial coefficients, k = 0, 1, 2, ...;

[0132] Minimize the error ε between the calculated value and the true value of the fitting curve function f(x) of the vehicle's driving trajectory;

[0133] The process of minimizing the error ε is expressed as:

[0134]

[0135] Among them, x i is the horizontal coordinate of the upper left corner of the vehicle in the rectangular detection box, y i The upper left corner ordinate of the vehicle in the rectangular detection frame;

[0136] For the k-order polynomial coefficient set {a0, a1, a2..., a k} respectively, and set the derivative result to 0 to obtain the final coefficient a;

[0137] The derivation process is expressed as:

[0138]

[0139]

[0140] make Get X T Xa=X T Y→a=(X T X) -1 X T Y

[0141] Among them, T represents the matrix transpose symbol, and finally all coefficients {a0, a1, a2..., a k}, a curve model that approximates all the test data can be obtained, thereby realizing curve fitting;

[0142] Specifically, whether the fitting curve function f(x) of the vehicle's driving trajectory has a solution depends on whether the ranks of the coefficient matrix and the augmented matrix of the equation system are equal. If they are not equal, the equation system has no solution. Therefore, in the actual calculation process, we can only minimize the error ε between the calculated value and the true value of the equation system.

[0143] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is selected primarily for readability and teaching purposes, rather than for explaining or defining the subject matter of the present invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is illustrative, not restrictive, with respect to the scope of the present invention, which is defined by the appended claims.

Claims

1. A method for calculating and restoring a vehicle driving trajectory, characterized in that: The following steps are involved: S1. Connect to the ordinary camera on the road side, identify and collect multi-attribute data of the moving vehicles, use the improved YOLO V8 network framework to detect vehicles, use the improved B-CNN network model to classify the vehicle model and color, use the scaling factor to correct the tilt of the vehicle image, use the convolutional neural network to extract the license plate image, use the convolutional neural network to recognize the license plate characters on the license plate image, and obtain the output result of vehicle detection; Specifically: the output results of vehicle detection include vehicle position coordinates, vehicle model classification, vehicle color classification and license plate number detection results and their confidence levels; S2. Use DeepSort algorithm to build a vehicle tracking model, input the output result of vehicle detection, and output the target vehicle tracking ID; S3. Set a vehicle driving detection area on the road driving area in the camera video screen, calculate the coordinates of the vehicle center point, use the ray method to determine whether the vehicle is in the vehicle driving detection area according to the coordinates of the vehicle center point, and eliminate vehicles outside the driving detection area; S4. According to the two-dimensional moving coordinate points of the vehicle and the time difference, the front driving angle is calculated, and then according to the parameters of the ordinary camera on the road side, the two-dimensional moving coordinate points of the vehicle are converted into real-world coordinate points to calculate the driving speed of the vehicle; S5. The driving trajectory point data is structured and packaged into a JSON message, i.e., structured data, and the structured data is sent and saved to a database on a remote server; Specifically: the driving trajectory point data includes time, camera ID number, camera resolution, vehicle ID number, vehicle location coordinates, vehicle model, vehicle head driving angle, vehicle speed, vehicle body color and license plate number; S6. Select the driving trajectory point data in the database, and fit the driving trajectory point data with the same target vehicle tracking ID to the upper left corner coordinate set of the corresponding vehicle on the rectangular detection box to obtain the fitting curve function of the vehicle driving trajectory; S7. Select a video screen of a camera as a canvas, draw the fitting curve of the vehicle's driving trajectory on the canvas, mark and display the data of each driving trajectory point of the vehicle, the driving angle and the driving speed of the vehicle, and obtain the restored vehicle driving trajectory; The S4 comprises the following steps: S41. Calculate the vehicle's front travel angle angle according to the vehicle's two-dimensional moving coordinates and the time difference t; S42. According to the parameters of the ordinary camera on the road side, the two-dimensional moving coordinate points of the vehicle are converted into real-world coordinate points to calculate the vehicle's speed; In the above S41, the two-dimensional moving coordinate points of the vehicle include the coordinates of the first moving point of the vehicle (x1, y1) and the coordinates of the second moving point of the vehicle (x2, y2); The front driving angle angle is the clockwise angle between the vehicle driving direction and the negative direction of the y-axis; The negative direction of the y-axis is represented by vector a (v1x, v1y), and the forward and backward movement points of the vehicle in the direction of travel of the vehicle are represented by vector b (v2x, v2y): If the absolute value of vector a |a| or the absolute value of vector b |b| is equal to 0, the front driving angle angle is 0, otherwise the front driving angle angle is expressed as: angle = 180*acos(ab / (|a|*|b|)) / 3.1415926, where acos is the arc cosine function, ab is the cross product ab of vector a and vector b, and if the horizontal coordinate x2 of the first moving point of the vehicle is less than the horizontal coordinate x1 of the second moving point of the vehicle, the front driving angle angle is expressed as: angle = 360-angle; In the S42, the two-dimensional moving coordinate points of the vehicle are converted into real-world coordinate points, and a real-world coordinate system is established. According to the time difference t between the first moving point coordinates (x1, y1) of the vehicle and the second moving point coordinates (x2, y2), the camera resolution width camera_width, the camera resolution height camera_height, the camera installation height h, the camera overhead angle (alpha), the horizontal field of view (hFoV) and the vertical field of view (vFoV), the first moving point coordinates (x1, y1) of the vehicle in the image are converted into first normalized coordinates (x1_normalized, y1_normalized) relative to the center of the image, and the second moving point coordinates (x2, y2) of the vehicle in the image are converted into second normalized coordinates (x2_normalized, y2_normalized) relative to the center of the image; According to the first normalized coordinates (x1_normalized, y1_normalized) and the second normalized coordinates (x2_normalized, y2_normalized), the first real-world coordinates (X1, Y1) corresponding to the first moving point coordinates (x1, y1) of the vehicle and the second real-world coordinates (X2, Y2) corresponding to the second moving point coordinates (x2, y2) of the vehicle are calculated; The driving speed speed is calculated according to the first real world coordinate (X1, Y1), the second real world coordinate (X2, Y2) and the time difference t.

2. A vehicle driving trajectory calculation and restoration method according to claim 1, characterized in that: The S1 comprises the following steps: S11. Use the improved YOLO V8 network framework to detect vehicles through the target detection algorithm to obtain vehicle detection images and vehicle position coordinates; S12. According to the vehicle detection image, the improved B-CNN network model is used to perform vehicle model classification and vehicle color classification to obtain vehicle model classification and vehicle color classification; S13. According to the vehicle detection image, the scaling factor is corrected, and then the license plate image is extracted by the convolutional neural network for license plate location, and then the convolutional neural network is used to recognize the license plate image, and the license plate number detection result and its confidence are output; In the S11, the YOLO V8 network framework is improved, the number of detection target categories of the YOLO V8 network framework is set to 1, the YOLO V8 network framework is trained and adjusted, the real-time video stream of the camera device is connected, the video decoding operation is performed, the video is divided into frame-by-frame pictures, that is, the vehicle detection image is input into the YOLO V8 network framework, and the vehicle detection is performed on each frame of the picture through the target detection algorithm of the YOLO V8 network framework to obtain the vehicle position coordinates, that is, the rectangular detection frame coordinates [left, top, width, height], wherein left is the upper left corner horizontal coordinate of the rectangular detection frame, top is the upper left corner vertical coordinate of the rectangular detection frame, width is the width of the rectangular detection frame, and height is the height of the rectangular detection frame; In the S12, the B-CNN network model is improved, the sub-network B of the B-CNN network model is replaced with the ResNet network, the sub-network A uses the VGGNet-16 network, an outer product operation is added, the features of the Conv4_3 network layer of the B-CNN network model are outer-producted with the features of the Conv5_3 network layer, so that the shallow features and the deep features are combined, and the vehicle model and vehicle color are input to obtain the vehicle model classification and vehicle color classification; In S13, according to the aspect ratio of the license plate bounding box, the target vehicle image is tilt-corrected using a scaling factor, and then the license plate is recognized using a convolutional neural network, and the license plate position and scale information are output to obtain a license plate image; A convolutional neural network model is used to perform license plate character recognition on license plate images. The number of prediction boxes of the convolutional neural network model is set to 2, and the aspect ratio of the license plate bounding box is 3:

1. The input size preprocessing of the convolutional neural network model is set to 240*80. The license plate image is input into the convolutional neural network model for recognition, and the license plate number detection result and its confidence are output. The confidence threshold of the license plate number detection result is set. If the confidence of the detection result is higher than the confidence threshold, the detection result is directly output. If the confidence of the detection result is lower than the confidence threshold, the detection result is discarded.

3. A vehicle driving trajectory calculation and restoration method according to claim 2, characterized in that: In S2, a vehicle tracking model is constructed by using the DeepSort algorithm, and the output result of the vehicle detection is used as the input of the vehicle tracking model. The target vehicle tracking ID and the predicted target tracking frame are output, and cascade matching is performed through the DeepSort algorithm. The intersection and union ratio of the rectangular detection frame coordinates and the predicted target tracking frame are input and combined to obtain the cascade matching result, which is input into the Hungarian algorithm for linear assignment to directly associate the rectangular detection frame coordinates with the predicted target tracking frame to perform inter-frame matching on the target vehicle tracking ID in the current frame.

4. A vehicle driving trajectory calculation and restoration method according to claim 3, characterized in that: In S3, a two-dimensional coordinate system is established to determine whether the vehicle is within the vehicle driving detection area by the coordinates (x, y) of the center point of the vehicle; Get the number of intersections between the horizontal line where the center point coordinates (x, y) of the vehicle are located and the edges of the polygon of the vehicle driving detection area. If the number of intersections is an odd number, the vehicle is within the vehicle driving detection area, otherwise the vehicle is outside the vehicle driving detection area.

5. A vehicle driving trajectory calculation and restoration method according to claim 4, characterized in that: In S6, the driving trajectory point data within a certain camera in a certain time period is selected from the database, and the driving trajectory point data with the same tracking ID of the target vehicle is obtained, and the upper left corner coordinate set {(x1, y1), (x2, y2), ..., (x n ,y n )} Perform k-order polynomial fitting to obtain the fitting curve function f(x) of the vehicle's driving trajectory.

Images