A method, device and system for estimating vehicle distance and speed

Abstract

A method, device and system for estimating vehicle distance and speed, the method comprising: collecting images of multiple vehicles in front of a road through a monocular camera, and determining a target frame corresponding to each vehicle; collecting video data of each vehicle according to the determined target frame, and determining the ID number and historical distance information corresponding to each vehicle using a preset vehicle tracking method; and calculating the speed of each vehicle and the distance between the vehicle and the monocular camera according to the determined ID number, historical distance information and a vehicle dynamics model.

Description

Technical Field

[0001] The present application relates to the technical field of road measurement, and in particular to a method, device and system for estimating vehicle distance and speed. Background Art

[0002] Forward collision warning system has become a necessary function in advanced driver assistance system (ADAS). It has been widely used in medium and large vehicles and can effectively reduce road traffic accidents. Speed ​​and distance detection of target vehicles in front of the road is the core module of forward collision warning system. In some technologies, the main ranging sensors of forward collision warning system are monocular camera and laser radar. Among them, the ranging and speed measurement method using laser radar has the highest accuracy and is not easily affected by noise, but its equipment cost and maintenance cost are too high. Compared with the laser radar method, monocular camera is low-cost and mainly estimates the speed and distance of the front vehicle by extracting image features, but it cannot directly obtain distance information and is easily affected by the lighting environment, causing huge measurement errors. Its accuracy depends entirely on the algorithm model and technical solution. Therefore, in order to balance equipment cost and accuracy, it is necessary and urgent to propose a monocular vision algorithm that can stably measure vehicle distance and speed. Summary of the invention

[0003] The present application provides a method, device and system for estimating vehicle distance and speed. The method can use a deep convolutional neural network to identify and track vehicles in video images, thereby more accurately measuring the vehicle distance and speed in forward collision warning.

[0004] The present application provides a method for estimating vehicle distance and speed, comprising: collecting images of multiple vehicles in front of a road through a monocular camera, and determining a target frame corresponding to each vehicle;

[0005] The video data of each vehicle is collected according to the determined target frame, and the ID number and historical distance information corresponding to each vehicle are determined by using a preset vehicle tracking method;

[0006] The speed of each vehicle and the distance between the vehicle and the monocular camera are calculated based on the determined ID number, historical distance information and the vehicle dynamics model.

[0007] In an exemplary embodiment, the step of collecting images of multiple vehicles in front of the road through a monocular camera and determining a target frame corresponding to each vehicle includes:

[0008] The monocular camera is used to collect images of multiple vehicles in front of the road, and the collected images are input into the target detection network;

[0009] Using the target detection network to perform feature extraction, abstraction and fitting on the collected multiple images to obtain multiple detection frames corresponding to each vehicle;

[0010] Using the non-maximum suppression method, the target box corresponding to the vehicle is filtered out from multiple detection boxes of each vehicle.

[0011] In an exemplary embodiment, the vehicle tracking method includes: a multi-target tracking algorithm based on the Hungarian algorithm and perceptual hashing.

[0012] In an exemplary embodiment, the step of collecting video data of each vehicle according to the determined target frame and determining the ID number and historical distance information corresponding to each vehicle using a preset vehicle tracking method includes:

[0013] Calculate the current frame target frame A in the collected video data i With the previous frame target frame B j The intersection-over-union ratio (IOU) i,j , where i represents the vehicle number corresponding to the target frame in the current frame, and j represents the vehicle number corresponding to the target frame in the previous frame;

[0014] Determine the weight E of the bipartite graph according to the size of the intersection ratio i,j ;

[0015] According to the current frame target frame A i , target frame B of the previous frame j And the weight E of the determined bipartite graph i,j Determine the ID number corresponding to each vehicle and the historical distance information corresponding to the ID number;

[0016] The intersection-to-union ratio is the ratio between the intersection area and the union area of ​​the two target boxes.

[0017] In an exemplary embodiment, the weight E of the bipartite graph is determined according to the size of the intersection-union ratio. i,j ,include:

[0018] If the calculated intersection-over-union ratio IOU i,j If it is greater than 0.6, then E i,j =IOU i,j ;

[0019] If the calculated intersection-over-union ratio IOU i,j If it is less than or equal to 0.6, calculate the current frame target frame A i With the previous frame target frame B j The difference between i,j , and according to S i,j The size of determines the weight E of the bipartite graph i,j .

[0020] In an exemplary embodiment, the i,j The size of determines the weight E of the bipartite graph i,j ,include:

[0021] When S i,j If it is less than or equal to 16, then E i,j =IOU i,j +E add ;

[0022] When S i,j If it is greater than 16 and less than 24, then E i,j =IOU i,j ;

[0023] When S i,j If it is greater than or equal to 24, then E i,j =0.

[0024] In an exemplary embodiment, the current frame target frame A i , target frame B of the previous frame j And the weights E of the determined multiple bipartite graphs i,j Determine the ID number corresponding to each vehicle and the historical distance information corresponding to the ID number, including:

[0025] According to the current frame target frame A i , target frame B of the previous frame j And the weight E of the determined bipartite graph i,j Form a vertex set A = {A1, A2, ..., A n}, B={B1,B2,…,B m}, edge set E = {E 1,1 ,E 1,2 ,…,E i,j ,…E m,n}, where n refers to the total number of vehicle target frames in the current frame and m refers to the total number of vehicle target frames in the previous frame;

[0026] The Hungarian matching algorithm is used to perform matching calculation on the weighted bipartite graph;

[0027] If the current frame target frame A i If the target frame of the previous frame matches successfully, then A i Inherit the ID number and historical distance information of the successfully matched target frame.

[0028] In an exemplary embodiment, the calculating the speed of each vehicle and the distance between the vehicle and the monocular camera according to the determined ID number, historical distance information and the vehicle dynamics model includes:

[0029] For each vehicle, the camera model is used to calculate the distance of each vehicle at the current moment based on the vehicle's driving trajectory and the preset vehicle dynamics model, and the distance is recorded in the ID number queue corresponding to each vehicle;

[0030] According to the historical distance information and the calculated distance under the ID number queue, the speed of each vehicle at the current moment is fitted using the least squares method and the vehicle dynamics model.

[0031] In an exemplary embodiment, the vehicle dynamics model includes:

[0032]

[0033] In the above formula, K is the vehicle dynamics matrix, v y 、v z 、a y 、a z They represent the longitudinal velocity, lateral velocity, longitudinal acceleration, and lateral acceleration of the vehicle respectively, and t is the time;

[0034] Combined with the historical distance information of the ID queue, the speed of each vehicle at the current moment is fitted using the least squares method and the vehicle dynamics model, including:

[0035] E = argmin[(N-KM) T (N-KM)+λKK T ];

[0036] Taking the partial derivative of E, we can get the actual speed of the vehicle at the current moment K = NM T (MM T +λI) -1 .

[0037] The present application also provides a device for estimating vehicle distance and speed, the device comprising: a memory and a processor; characterized in that the memory is used to store a program for estimating vehicle distance and speed, and the processor is used to read and execute the program for estimating vehicle distance and speed, and execute any one of the methods described in the above embodiments.

[0038] The present application also provides a system for estimating vehicle distance and speed, comprising: a monocular camera, a vehicle monitoring and tracking module;

[0039] The monocular camera is used to collect images of multiple vehicles ahead on the road;

[0040] The vehicle monitoring and tracking module is used to determine the target frame corresponding to each vehicle;

[0041] The video data of each vehicle is collected according to the determined target frame, and the ID number and historical distance information corresponding to each vehicle are determined by using a preset vehicle tracking method;

[0042] The speed of each vehicle and the distance between the vehicle and the monocular camera are calculated based on the determined ID number, historical distance information and the vehicle dynamics model.

[0043] Compared with the related art, the present application provides a method, device and system for estimating vehicle distance and speed, the method comprising: collecting images through a monocular camera to determine the target frame of the first vehicle in front of the road; collecting video data of the first vehicle using a preset vehicle tracking method according to the determined target frame; determining the ID number and historical distance information of the first vehicle according to the collected video data; calculating the speed of the first vehicle and the distance between the first vehicle and the monocular camera according to the determined ID number, historical distance information and vehicle dynamics model. The present application can use a deep convolutional neural network to identify and track vehicles in video images, and more accurately measure the vehicle distance and speed in forward collision warning.

[0044] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by implementing the present application. Other advantages of the present application can be realized and obtained by the schemes described in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings are used to provide an understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0046] Figure 1 A flow chart of a method for estimating vehicle distance and speed according to an embodiment of the present application;

[0047] Figure 2 The flowchart of a vehicle distance measurement and speed measurement algorithm based on monocular vision and convolutional network in an example embodiment;

[0048] Figure 3 is a flow chart of a vehicle tracking algorithm in an exemplary embodiment;

[0049] Figure 4 A flow chart of vehicle distance and speed estimation in an example embodiment;

[0050] Figure 5 A schematic diagram of a device for estimating vehicle distance and speed according to an embodiment of the present application;

[0051] Figure 6A schematic diagram of a system for estimating vehicle distance and speed according to an embodiment of the present application. DETAILED DESCRIPTION

[0052] The present application describes multiple embodiments, but the description is exemplary rather than restrictive, and it is obvious to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present application. Although many possible feature combinations are shown in the drawings and discussed in the specific embodiments, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.

[0053] The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features and elements disclosed in the present application may also be combined with any conventional features or elements to form a unique invention scheme defined by the claims. Any features or elements of any embodiment may also be combined with features or elements from other invention schemes to form another unique invention scheme defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in the present application may be implemented individually or in any appropriate combination. Therefore, except for the limitations made according to the attached claims and their equivalents, the embodiments are not subject to other restrictions. In addition, various modifications and changes may be made within the scope of protection of the attached claims.

[0054] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps of the specific order described. As will be understood by those of ordinary skill in the art, other sequences of steps are also possible. Therefore, the specific sequence of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps of performing them in the order written, and those skilled in the art can easily understand that these sequences can be changed and still remain within the spirit and scope of the embodiments of the present application.

[0055] The present disclosure provides a method for estimating vehicle distance and speed. Figure 1 As shown, the method includes steps S100-S120;

[0056] S100. Collect multiple vehicle images in front of the road through a monocular camera and determine the target frame corresponding to each vehicle;

[0057] S110. Collect video data of each vehicle according to the determined target frame, and use a preset vehicle tracking method to determine the ID number and historical distance information corresponding to each vehicle;

[0058] S120. Calculate the speed of each vehicle and the distance between the vehicle and the monocular camera according to the determined ID number, historical distance information and vehicle dynamics model.

[0059] In this embodiment, the monocular camera used may have a collection frequency higher than 30 Hz, a focal length range of 10 to 16 mm, and a field of view range greater than or equal to 35°. The calculated distance and speed of the vehicle refer to the relative distance and relative speed between the vehicle and the camera with the camera as a reference.

[0060] In an exemplary embodiment, the process of collecting images of multiple vehicles in front of the road through a monocular camera and determining a target frame corresponding to each vehicle may include:

[0061] Step 1. Capture images of multiple vehicles through the monocular camera, and input the captured multiple images of each vehicle into the target detection network respectively; in this step, the monocular camera can simultaneously capture multiple image information of multiple vehicles.

[0062] Step 2. Use the target detection network to extract, abstract and fit the features of the collected multiple images to obtain multiple detection frames for each vehicle; in this step, the target detection network should be a convolutional neural network, which can accurately predict the specific position of the vehicle in the input image. In order to meet the real-time requirements of forward collision warning, the average intersection-over-union ratio of the target frame output by the convolutional neural network and the real frame cannot be less than 0.65, and the computing power of the convolutional neural network operator cannot be less than 4.0Tflops.

[0063] Step 3. Use the non-maximum suppression method to filter out a target box corresponding to each vehicle from multiple detection boxes of each vehicle.

[0064] In an exemplary embodiment, the vehicle tracking method includes: a multi-target tracking algorithm based on the Hungarian algorithm and perceptual hashing.

[0065] In an exemplary embodiment, collecting the video data of the first vehicle according to the determined target frame, and determining the ID number and historical distance information of each vehicle using a preset vehicle tracking method may include:

[0066] Collecting video data of multiple vehicles according to the determined multiple target frames;

[0067] Calculate the current frame target frame A in the collected video data iWith the previous frame target frame B j The intersection-over-union ratio (IOU) i,j , where i represents the vehicle number corresponding to the target frame in the current frame, and j represents the vehicle number corresponding to the target frame in the previous frame;

[0068] Determine the weight E of the bipartite graph according to the size of the intersection ratio i,j ;

[0069] According to the current frame target frame A i , target frame B of the previous frame j And the weight E of the determined bipartite graph i,j Determine the ID number corresponding to each vehicle and the historical distance information corresponding to the ID number;

[0070] The intersection-to-union ratio is the ratio between the intersection area and the union area of ​​the two target boxes.

[0071] The weight E of the bipartite graph is determined according to the size of the intersection ratio i,j ,include:

[0072] If the calculated intersection-over-union ratio IOU i,j If it is greater than 0.6, then E i,j =IOU i,j ;

[0073] If the calculated intersection-over-union ratio IOU i,j If it is less than or equal to 0.6, calculate the current frame target frame A i With the previous frame target frame B j The difference between i,j , and according to S i,j The size of determines the weight E of the bipartite graph i,j ; The difference S i,j It can be a difference value obtained according to the ahash algorithm.

[0074] In an exemplary embodiment, the i,j The size of determines the weight E of the bipartite graph i,j ,include:

[0075] When S i,j If it is less than or equal to 16, then E i,j =IOU i,j +E add ;

[0076] When S i,j If it is greater than 16 and less than 24, then E i,j =IOU i,j ;

[0077] When S i,j If it is greater than or equal to 24, then Ei,j =0.

[0078] In an exemplary embodiment, the current frame target frame A i , target frame B of the previous frame j And the weights E of the determined multiple bipartite graphs i,j Determining the ID number of the first vehicle and the historical distance information corresponding to the ID number includes:

[0079] According to the current frame target frame A i , target frame B of the previous frame j And the weights E of the determined multiple bipartite graphs i,j Form a vertex set A = {A1, A2, ..., A n}, B={B1,B2,…,B m}, edge set E = {E 1,1 ,E 1,2 ,…,E i,j ,…E m,n}, where n refers to the total number of vehicle target frames in the current frame, that is, the number of vehicles that can be collected and detected in the current frame, and m refers to the total number of vehicle target frames in the previous frame, that is, the number of vehicles that can be collected and detected in the previous frame;

[0080] The Hungarian matching algorithm is used to perform matching calculation on the weighted bipartite graph;

[0081] If the current frame target frame A i If the target frame of the previous frame matches successfully, then A i Inherit the ID number and historical distance information of the successfully matched target frame;

[0082] If there is no i Matching target box, then for A i Assign a new ID number and create a queue to record the historical distance information of the ID number.

[0083] In an exemplary embodiment, the speed of each vehicle and the distance between the vehicle and the monocular camera are calculated according to the determined ID number of each vehicle, the historical distance information and the vehicle dynamics model, including:

[0084] According to the driving trajectory of each vehicle, the pre-established vehicle dynamics model uses the camera model to solve the distance of each vehicle at the current moment, and records the distance under the queue of each vehicle's corresponding ID number;

[0085] According to the historical distance information corresponding to the ID number queue and the calculated distance, the speed of each vehicle at the current moment is fitted using the least squares method and the vehicle dynamics model.

[0086] In this embodiment, the implementation process of solving the distance of each vehicle at the current moment by using the camera model may include: solving the longitudinal distance Z and the lateral distance Y of the front vehicle as shown in the following formula:

[0087]

[0088]

[0089] The above two equations are obtained from the projection relationship of the camera model.

[0090] The following formula shows the projection relationship between the vehicle in the pixel coordinate system and the camera coordinate system:

[0091]

[0092] In the above formula, u and v represent the ordinate and abscissa of the center point of the lower edge of the vehicle target box in the image, respectively, and f x 、f y 、c x 、c y is the camera internal parameter, X, Y, Z are the three-dimensional coordinates of the vehicle in the camera coordinate system, H is the actual height of the target vehicle, and h is the pixel height of the vehicle target box.

[0093] In an exemplary embodiment, the historical distance information under the ID queue is combined to fit the actual speed of the vehicle at the current moment using the least squares method and the vehicle dynamics model, and the fitting error of the least squares method is given. If the fitting error is greater than the threshold T, it means that the historical distance information does not conform to the vehicle dynamics model, the fitting fails and the result is unreliable; otherwise, the result is credible. The vehicle dynamics model includes:

[0094] N=KM

[0095] In the above formula, K is the vehicle dynamics matrix, v y 、v z 、a y 、a z denote the longitudinal velocity, lateral velocity, longitudinal acceleration and lateral acceleration of the first vehicle respectively, and t denotes time;

[0096] Combined with the historical distance information of the ID queue, the least squares method and vehicle dynamics model are used to fit the speed of each vehicle at the current moment, including:

[0097] E = argmin[(N-KM) T (N-KM)+λKK T ];

[0098] Taking the partial derivative of E, we can get the real speed of the vehicle at the current moment K = NM T(MM T +λI) -1 .

[0099] K can be solved by least squares reasoning. The specific steps are as follows:

[0100] E = argmin[(N-KM) T (N-KM)+λKK T ]

[0101]

[0102] K=NM T (MM T +λI) -1

[0103] err=(N-KM) T (N-KM)

[0104] Among them, λ is the regularization coefficient, which is usually between 0 and 1, and err is the fitting error of the least squares method.

[0105] In this embodiment, the vehicle distance measurement and speed measurement algorithm based on monocular vision and convolutional network has the following technical effects:

[0106] 1. Relying on the robustness of the convolutional network, it can identify multiple vehicles in front in real time under harsh environments, and obtain the distance and speed of a large number of vehicles in a very short time, which greatly improves the accuracy of the forward collision warning system, can effectively reduce road traffic accidents, and ensure the safety of life and property of people in driving vehicles.

[0107] 2. It only requires a monocular camera and a computing unit with certain computing power. Compared with the multi-line laser radar solution, it greatly reduces the equipment cost and improves the maintainability of the equipment.

[0108] Example 1

[0109] Vehicle ranging and speed measurement algorithms based on monocular vision and convolutional networks, such as Figure 2 As shown, the process is:

[0110] Step 200: The monocular camera collects multiple images of the two vehicles in front.

[0111] Step 210 inputs the captured image into the target detection network to obtain multiple candidate detection frames of the first vehicle and the second vehicle.

[0112] In this step, the image is input into the target detection network, and after feature extraction, abstraction, and fitting in the convolution layer and the connection layer, multiple detection frames of the first vehicle and the second vehicle are obtained.

[0113] In step 220, after performing non-maximum suppression calculation on multiple detection frames, the target frames corresponding to the first vehicle and the second vehicle are screened out.

[0114] Step 230 collects vehicle video data using a preset vehicle tracking method according to the determined target frame; and determines the ID number and historical distance information of the first vehicle and the second vehicle respectively according to the collected video data;

[0115] In this step, according to the determined target frame, the preset vehicle tracking method is used to collect the video data of the vehicle; the collected video data is input into the vehicle tracking module, the vehicle ID number is assigned and updated, and the historical distance information of the ID number is inherited.

[0116] Step 240 calculates the distance and speed of each vehicle through the distance and speed solving module.

[0117] Through the distance and speed calculation module, combined with the projection model of the monocular camera, the least squares method and vehicle dynamics model are used to calculate the real distance and speed of multiple vehicles.

[0118] Example 2

[0119] In the vehicle distance and speed measurement algorithm based on monocular vision and convolutional network, the video data of the vehicle is collected according to the determined target frame, and the ID number and historical distance information of each vehicle are determined by using the preset vehicle tracking method; wherein, the implementation process of the vehicle tracking algorithm is as follows: Figure 3 As shown:

[0120] The first step is to calculate the current frame target frame A in the collected video data i With the previous frame target frame B j The intersection-over-union ratio (IOU) i,j ; Among them, the intersection-to-union ratio is the ratio between the intersection area and the union area of ​​the two target boxes.

[0121] Step 2: Determine the weight E of the bipartite graph based on the size of the intersection ratio i,j ;

[0122] In this step, the weight E of the bipartite graph is determined according to the size of the intersection ratio i,j ,include:

[0123] 1) If the calculated intersection-over-union ratio IOU i,j If it is greater than 0.6, then E i,j =IOU i,j ;

[0124] 2) If the calculated intersection-over-union ratio IOU i,j If it is less than or equal to 0.6, calculate the current frame target frame A i With the previous frame target frame B jThe difference between ahash i,j , and according to S i,j The size of determines the weight E of the bipartite graph i,j .

[0125] In step 2), according to S i,j The size of determines the weight E of the bipartite graph i,j , which may include:

[0126] When S i,j If it is less than or equal to 16, then E i,j =IOU i,j +E add ;

[0127] When S i,j If it is greater than 16 and less than 24, then E i,j =IOU i,j ;

[0128] When S i,j If it is greater than or equal to 24, then E i,j =0.

[0129] Step 3: According to the current frame target frame A i , target frame B of the previous frame j And the weight E of the determined bipartite graph i,j Determine the ID number of each vehicle and the historical distance information corresponding to the ID number;

[0130] In this step, according to the current frame target frame A i , target frame B of the previous frame j And the weight E of the determined bipartite graph i,j Determine the vehicle ID number and the historical distance information corresponding to the ID number, including:

[0131] According to the current frame target frame A i , target frame B of the previous frame j And the weight E of the determined bipartite graph i,j Form a vertex set A = {A1, A2, ..., A n}, B={B1,B2,…,B m}, edge set E = {E 1,1 ,E 1,2 ,…,E i,j ,…E m,n}, where n refers to the total number of vehicle target frames in the current frame and m refers to the total number of vehicle target frames in the previous frame;

[0132] The Hungarian matching algorithm is used to perform matching calculation on the weighted bipartite graph;

[0133] If the current frame target frame Ai If the target frame of the previous frame matches successfully, then A i Inherit the ID number and historical distance information of the successfully matched target frame;

[0134] If there is no i Matching target box, then for A i Assign a new ID number and create a queue to record the historical distance information of the ID number.

[0135] Example 3

[0136] The distance and speed between each vehicle and the monocular camera are calculated based on the determined ID number and the vehicle dynamics model. The estimated distance and speed of the vehicle are as follows: Figure 4 As shown, the process includes:

[0137] Step 401. According to the driving trajectory of the vehicle, a vehicle dynamics model is established, and the distance of each vehicle at the current moment is solved using the camera model, and the distance is recorded in the queue under the current vehicle ID number;

[0138] Among them, the longitudinal distance Z and lateral distance Y of each vehicle are solved according to the projection relationship of the camera model. The projection relationship of the camera model is shown in the following formula, the projection relationship of the vehicle in the pixel coordinate system and the camera coordinate system:

[0139]

[0140] The camera model solves the longitudinal distance Z and lateral distance Y of each vehicle, which can be shown as follows:

[0141]

[0142]

[0143] In the above formula, u and v represent the ordinate and abscissa of the center point of the lower edge of the vehicle target box in the image, and f x 、f y 、c x 、c y is the camera internal parameter, X, Y, Z are the three-dimensional coordinates of the vehicle in the camera coordinate system, H is the actual height of the target vehicle, and h is the pixel height of the vehicle target frame.

[0144] Step 402. Combine the historical distance information under the ID number queue, use the least square method and the vehicle dynamics model to fit the actual speed of the vehicle at the current moment, and give the fitting error of the least square method.

[0145] In this step, the vehicle dynamics model includes but is not limited to the following formula:

[0146] N=KM

[0147] K can be solved by least squares reasoning. The specific steps are as follows:

[0148] E = argmin[(N-KM) T (N-KM)+λKK T ]

[0149]

[0150] K=NM T (MM T +λI) -1

[0151] err=(N-KM) T (N-KM)

[0152] Where K is the vehicle dynamics matrix, v y 、v z 、a y 、a z are the longitudinal velocity, lateral velocity, longitudinal acceleration, and lateral acceleration of the vehicle, t is the time, the λ regularization term coefficient is generally between 0 and 1, and err is the fitting error of the least squares method.

[0153] Step 403: Determine the credibility of the detection result based on the fitting error.

[0154] If the fitting error is greater than the threshold T, it means that the historical distance information does not match the vehicle dynamics model, the fitting fails and the result is unreliable; otherwise, the result is credible.

[0155] The present application also provides a device for estimating vehicle distance and speed, such as Figure 5 As shown, the device includes: a memory 510 and a processor 520; the memory is used to store a program for estimating vehicle distance and speed, and the processor is used to read and execute the program for estimating vehicle distance and speed, and execute any one of the methods in the above embodiments.

[0156] The present application also provides a system for estimating vehicle distance and speed, such as Figure 6 As shown, the system includes: a monocular camera 610, a vehicle monitoring and tracking module 620;

[0157] The monocular camera is used to collect images of multiple vehicles ahead on the road;

[0158] The vehicle monitoring and tracking module is used to determine the target frame corresponding to each vehicle;

[0159] The video data of each vehicle is collected according to the determined target frame, and the ID number and historical distance information corresponding to each vehicle are determined by using a preset vehicle tracking method;

[0160] The speed of each vehicle and the distance between the vehicle and the monocular camera are calculated based on the determined ID number, historical distance information and the vehicle dynamics model.

[0161] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

Claims

1. A method for estimating vehicle distance and speed, characterized in that: The method comprises: The monocular camera is used to collect images of multiple vehicles in front of the road and determine the target frame corresponding to each vehicle; The video data of each vehicle is collected according to the determined target frame, and the ID number and historical distance information corresponding to each vehicle are determined by using the preset vehicle tracking method, including: Calculate the current frame target frame A in the collected video data i With the previous frame target frame B j The intersection-over-union ratio (IOU) i,j , where i represents the vehicle number corresponding to the target frame in the current frame, and j represents the vehicle number corresponding to the target frame in the previous frame; Determine the weight E of the bipartite graph according to the size of the intersection ratio i,j , including: If the calculated intersection-over-union ratio IOU i,j If it is greater than 0.6, then E i,j =IOU i,j ; If the calculated intersection-over-union ratio IOU i,j If it is less than or equal to 0.6, calculate the current frame target frame A i With the previous frame target frame B j The difference between i,j , and according to S i,j The size of determines the weight E of the bipartite graph i,j ; The intersection-to-union ratio is the ratio between the intersection area and the union area of ​​the two target frames; According to the current frame target frame A i , target frame B of the previous frame j And the weight E of the determined bipartite graph i,j Determine the ID number corresponding to each vehicle and the historical distance information corresponding to the ID number, including: according to the current frame target frame A i , target frame B of the previous frame j And the weight E of the determined bipartite graph i,j Form a vertex set A={A1,A2,…,A n }, B={B1,B2,…,B m }, edge set E = {E 1,1 , E 1,2 ,…,E i,j , …E m,n }, where n refers to the total number of vehicle target frames in the current frame, and m refers to the total number of vehicle target frames in the previous frame; the weighted bipartite graph is matched using the Hungarian matching algorithm; if the current frame target frame A i If the target frame of the previous frame matches successfully, then A i Inherit the ID number and historical distance information of the successfully matched target frame; The speed of each vehicle and the distance between the vehicle and the monocular camera are calculated based on the determined ID number, historical distance information and the vehicle dynamics model.

2. The method for estimating vehicle distance and speed according to claim 1, characterized in that: The method of collecting images of multiple vehicles in front of the road through a monocular camera and determining a target frame corresponding to each vehicle includes: The monocular camera is used to collect images of multiple vehicles in front of the road, and the collected images are input into the target detection network; Using the target detection network to perform feature extraction, abstraction and fitting on the collected multiple images to obtain multiple detection frames corresponding to each vehicle; Using the non-maximum suppression method, the target box corresponding to the vehicle is filtered out from multiple detection boxes of each vehicle.

3. The method for estimating vehicle distance and speed according to claim 1, characterized in that: The vehicle tracking method includes: a multi-target tracking algorithm based on the Hungarian algorithm and perceptual hashing.

4. The method for estimating vehicle distance and speed according to claim 1, characterized in that: The calculating the speed of each vehicle and the distance between the vehicle and the monocular camera according to the determined ID number, historical distance information and the vehicle dynamics model includes: For each vehicle, the camera model is used to calculate the distance of each vehicle at the current moment based on the vehicle's driving trajectory and the preset vehicle dynamics model, and the distance is recorded in the ID number queue corresponding to each vehicle; According to the historical distance information and the calculated distance under the ID number queue, the speed of each vehicle at the current moment is fitted using the least squares method and the vehicle dynamics model.

5. The method for estimating vehicle distance and speed according to claim 4, characterized in that: The vehicle dynamics model includes: In the above formula, K is the vehicle dynamics matrix, v y 、v z 、a y 、a z They represent the longitudinal velocity, lateral velocity, longitudinal acceleration, and lateral acceleration of the vehicle respectively, Y and Z are the three-dimensional coordinates of the vehicle in the camera coordinate system, and t is the time; Combined with the historical distance information of the ID queue, the speed of each vehicle at the current moment is fitted using the least squares method and the vehicle dynamics model, including: ； Take the partial derivative of E to get the actual speed of the vehicle at the current moment ； Among them, λ is the regularization coefficient, T is the threshold, and I is the unit matrix.

6. A device for estimating vehicle distance and speed, characterized in that: The device comprises: a memory and a processor; characterized in that the memory is used to store a program for estimating vehicle distance and speed, and the processor is used to read and execute the program for estimating vehicle distance and speed, and execute the method described in any one of claims 1-5.

7. A system for estimating vehicle distance and speed, characterized in that The system includes: a monocular camera, a vehicle monitoring and tracking module; The monocular camera is used to collect images of multiple vehicles ahead on the road; The vehicle monitoring and tracking module is used to determine the target frame corresponding to each vehicle; The video data of each vehicle is collected according to the determined target frame, and the ID number and historical distance information corresponding to each vehicle are determined by using the preset vehicle tracking method, including: Calculate the current frame target frame A in the collected video data i With the previous frame target frame B j The intersection-over-union ratio (IOU) i,j , where i represents the vehicle number corresponding to the target frame in the current frame, and j represents the vehicle number corresponding to the target frame in the previous frame; Determine the weight E of the bipartite graph according to the size of the intersection ratio i,j , including: If the calculated intersection-over-union ratio IOU i,j If it is greater than 0.6, then E i,j =IOU i,j ; If the calculated intersection-over-union ratio IOU i,j If it is less than or equal to 0.6, calculate the current frame target frame A i With the previous frame target frame B j The difference between i,j , and according to S i,j The size of determines the weight E of the bipartite graph i,j ; The intersection-to-union ratio is the ratio between the intersection area and the union area of ​​the two target frames; According to the current frame target frame A i , target frame B of the previous frame j And the weight E of the determined bipartite graph i,j Determine the ID number corresponding to each vehicle and the historical distance information corresponding to the ID number, including: according to the current frame target frame A i , target frame B of the previous frame j And the weight E of the determined bipartite graph i,j Form a vertex set A={A1,A2,…,A n }, B={B1,B2,…,B m }, edge set E = {E 1,1 , E 1,2 ,…,E i,j , …E m,n }, where n refers to the total number of vehicle target frames in the current frame, and m refers to the total number of vehicle target frames in the previous frame; the weighted bipartite graph is matched using the Hungarian matching algorithm; if the current frame target frame A i If the target frame of the previous frame matches successfully, then A i Inherit the ID number and historical distance information of the successfully matched target frame; The speed of each vehicle and the distance between the vehicle and the monocular camera are calculated based on the determined ID number, historical distance information and the vehicle dynamics model.

Images