Monocular Camera-based Auxiliary Collision Warning Method

Abstract

The present invention discloses an auxiliary collision warning method based on a monocular camera. The method uses the ECO tracking algorithm in the foreground to output an estimated tracking result of the vehicle, while the backend uses the SSD target detection algorithm, which is slower but has higher target detection accuracy, to detect the vehicle, obtaining a more accurate target detection frame. The vehicle tracking result outputted from the foreground is corrected by the target detection frame, thereby improving the timeliness of the overall method. The ECO tracking algorithm is then used as the main method to quickly give a prediction frame for target detection of the object in the next frame of the image, thereby alleviating the high latency problem of the target detection neural network. The present invention conducts detection and tracking in parallel, can quickly make collision warnings, and improve the operating speed of the system; compared with other target tracking algorithms, it can obtain the position of the target more accurately, thereby greatly reducing the occurrence of errors and improving the reliability of the system.

Description

Technical Field

[0001] The present invention belongs to the field of collision warning, and in particular relates to an auxiliary collision warning method based on a monocular camera. Background Art

[0002] With the continuous development of intelligent driving technology, assisting drivers in achieving safe driving has become a must-solve issue in the current intelligent driving field, and assisted collision warning technology has become crucial. Current collision warning methods mainly use sensors such as millimeter radar waves, lasers, and infrared to measure information such as the vehicle's speed, the speed of vehicles in the target direction, and the distance between the vehicle and the target vehicle. After processing this information such as the distance and instantaneous relative speed, the system determines the safe distance between the two vehicles. If the distance between the two vehicles is less than the safe distance, a warning prompt will be issued to the driver.

[0003] The present invention, however, uses a monocular camera to capture live footage of vehicles in motion, then employs deep learning techniques such as vehicle tracking algorithms to detect and dynamically track these images in real time. This imagery is then analyzed to determine key factors such as the distance between vehicles. This enables an auxiliary collision warning function, with the tracking algorithm playing a crucial role.

[0004] Current mainstream tracking algorithms primarily track targets based on target detection results. These algorithms obtain object detection information and transmit it to tracking algorithms for tracking. However, for all continuously moving objects, detection suffers from latency issues. This is because all detection algorithms require a certain amount of time to run. During this time, the target being detected may have already left its current location, resulting in inaccurate distance information.

[0005] If the algorithm takes too long, the target is too far away, or its position in the image has significantly changed, the detection results and distance information will become meaningless for assisted driving. If the detection algorithm fails, the tracking algorithm will also fail, making it impossible to track the object effectively, ultimately making it difficult to achieve real-time dynamic tracking and positioning of the vehicle.

[0006] The distance measurement algorithm primarily uses a commonly used bird's-eye view ranging algorithm. This algorithm uses the rotation transformation matrix in camera imaging principles to transform the target from the normal image to the bird's-eye view image. The distance information is then calculated based on the target coordinates in the bird's-eye view. However, this method often has some errors. In the bird's-eye view, there is a certain distance between the camera's actual position in the world coordinate system and the actual position represented by the maximum vertical coordinate in the pixel image. There is also a deviation between the measured optical center and the actual optical center of the camera. This leads to a certain error in the horizontal coordinate of the target in the bird's-eye view, especially in some special scenarios such as driving school scenes. This ultimately leads to inaccurate distance information.

[0007] In real, complex and ever-changing scenarios, it is difficult for the mainstream tracking and ranging algorithms mentioned above to solve problems in actual scenarios. For example, in driving school scenarios, vehicles often make continuous turns and people quickly leave the side of the vehicle. The target detection algorithm that relies solely on neural networks for feature extraction and detection will have a large time overhead, resulting in a delay in the target detection results, which will cause the tracking results to lag. Ultimately, there will be an error between the obtained distance information and the actual distance information at the current moment, and the vehicle collision warning function cannot be realized.

[0008] This type of problem occurs frequently in driving school scenarios. To better address safety issues in driving school scenarios, this invention aims to design a high-precision, low-latency vehicle detection and tracking method, as well as to make some improvements and corrections to the existing ranging algorithm to improve ranging accuracy and better provide important technical support for technical fields such as assisted collision warning. Summary of the Invention

[0009] The problem to be solved by the present invention is to provide a high-precision, low-latency vehicle detection and tracking method, and to improve the existing ranging method. The main method adopted by the present invention is to use the ECO tracking algorithm in the foreground to output the estimated tracking result of the vehicle, while the background uses the SSD target detection algorithm with slower speed but higher target detection accuracy to detect the vehicle, obtain a more accurate target detection frame, and correct the vehicle tracking result output by the foreground through the target detection frame, thereby improving the timeliness of the overall method. Then, the ECO tracking algorithm is used as the main method to quickly give the target detection prediction frame of the object in the next frame of the picture, thereby alleviating the high latency problem of the target detection neural network.

[0010] The target coordinate correction method in the bird's-eye view is to calculate the fixed value error of the target in the horizontal and vertical coordinates of the bird's-eye view through a designed calculation formula, and then correct the target coordinates to the coordinates from the actual camera position. The target coordinate conversion method in the bird's-eye view combined with the SFM (Structure from Motion) perspective conversion principle regards the bird's-eye view and the normal camera image as two images from different perspectives. The original camera imaging principle in the bird's-eye view ranging algorithm is used to calculate a small number of matching points in the two-perspective imaging images to obtain the transformation matrix between the two images, which is the essential matrix set in the perspective conversion principle. The essential matrix can be used to quickly convert the required target coordinates in the two-perspective images, so there is no need to regenerate the entire bird's-eye view to obtain the coordinates of the target in the bird's-eye view.

[0011] The overall concept of this invention is to use the deepsort multi-target tracking algorithm to estimate and track objects, and then use the SSD detection algorithm to correct the estimated tracking information. The tracking algorithm runs in the foreground, while the detection algorithm runs in the background. The two are in a relatively parallel state, rather than the current mainstream serial state. This solves the problem of high latency in the detection algorithm. The parallel detection and tracking algorithm is combined with the improved ranging method of this invention to achieve real-time vehicle tracking and ranging, and achieve the purpose of assisting collision warning.

[0012] The basic idea of the present invention is to obtain video stream data information in front of the vehicle through a front camera, judge whether a collision will occur, and issue a collision warning. Frame images are intercepted based on the video stream data of the front camera, and the obtained frame images are sent to the Kalman filter network. The Kalman filter is used to predict the near-end target tracking frame in front of the vehicle, and the obtained frame images are simultaneously sent to the target detection network to give a near-end target detection frame for the near-end target in front of the vehicle (a target less than 10 meters away from the vehicle, which can be a car, a person, or other objects). The detected near-end target detection frame is sent to the Kalman filter network again to start the target tracking result correction. The target tracking result correction is specifically as follows: first, all the detected near-end target detection frames are ID matched separately according to the existing data association algorithm to determine the near-end target detection frame corresponding to each near-end target in the current frame. Then, if the cosine distance between the proximal target tracking frame under the ECO tracking algorithm's fast tracking and the proximal target detection frame given by the target detection network exceeds a predetermined threshold, the ECO tracking algorithm will suspend tracking of the target. At this time, the proximal target detection frame obtained by the target detection network will be used to immediately correct the tracking frame that exceeds the threshold, and the sample space weight will be adjusted until the cosine distance between the proximal target tracking frame and the proximal target detection frame is lower than (or equal to) the predetermined threshold. At this time, the correction and weight adjustment will be stopped, and the normal operation of the ECO tracking algorithm will be restored. Finally, the final prediction frame (proximal target tracking frame) of the Kalman filter network will be used to make a collision warning judgment (if the distance between the object and the vehicle is less than 1 meter, forced braking and other measures will be initiated to avoid possible collisions and improve safety).

[0013] The present invention proposes an auxiliary collision warning method based on a monocular camera, comprising the following steps:

[0014] Step 1: Intercept the input video stream data information obtained by the monocular camera, obtain the picture frame, and put the obtained picture frame into the picture frame queue.

[0015] Step 2: The segmentation network obtains the scene image from the image frame queue and outputs the ground information in the scene image.

[0016] Step 3: The target detection network obtains the scene image from the frame queue, detects the target in the scene image, and provides the initial vehicle and human target detection frames.

[0017] Step 4: Combine the initial vehicle and human target detection frames with the ground information, and correct the initial vehicle and human target detection frames to fall back to the ground to obtain the corrected detection frames.

[0018] Step 5: Perform data association and IOU (Intersection Over Union) calculation on the corrected detection box, and calculate and assign target IDs for multiple targets of the target detection network.

[0019] Step 6: At the same time, the corrected detection frame is used to detect the target distance using the bird's-eye view ranging algorithm that adds ground information to obtain the target distance.

[0020] Step 7: Determine whether each target among the multiple targets is within a preset distance threshold. If so, proceed to step 8; otherwise, proceed to step 13.

[0021] Step 8: For targets within 10 meters of the main vehicle, the Kalman tracking algorithm is activated to perform continuous and rapid target predictions, generating target prediction frames. The target detection frames continuously fed into the target detection network are compared with the target IDs and the target prediction frames of the Kalman tracking algorithm. The ECO filter is then used to calculate and score the frames using the optimal difference Newton iteration method. The scoring results determine whether to perform coordinate repair and scale factor update on the target currently being processed by the ECO tracking algorithm (this occurs in parallel with coordinate repair).

[0022] Step 9: At the same time, based on the scoring results of step 8, determine whether coordinate repair is needed and whether the conditions for coordinate repair are met. If necessary, proceed to step 11; otherwise, perform scale factor judgment.

[0023] Step 10: Determine whether the scale factor needs to be updated based on the scoring result of step 8. If so, proceed to step 12; otherwise, proceed to step 13.

[0024] Step 11: Correct the coordinates of the target detection frame according to its corresponding coordinates in the SSD detection algorithm (a single-stage target detection algorithm that uses a convolutional neural network to extract features and outputs detection at different feature layers, a multi-scale detection method).

[0025] Step 12: Start the Hermit interpolation algorithm to estimate the scale factor of the target detection box, replacing the original multi-scale factor of the Kalman filter (an algorithm that uses linear system state equations to optimally estimate the system state through system input and output observation data).

[0026] Step 13: Output the target tracking frame obtained by correcting the ECO tracking algorithm using the SSD detection algorithm, and perform distance determination using the bird's-eye view ranging algorithm. If the distance is less than the safe distance (1 meter for this system), a collision warning (forced braking and other measures) is initiated.

[0027] The present invention has the following advantages due to the adoption of the above technical solution:

[0028] This invention combines detection and tracking in parallel, significantly reducing the computation time required to determine the final target position compared to existing algorithms. This allows for faster distance calculations, collision warnings, and improved system speed. This invention incorporates the SSD target detection algorithm to modify the ECO target tracking algorithm, ensuring accurate target positions. Compared to other target tracking algorithms, this method achieves more accurate target positions, significantly reducing errors and improving system reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flow chart of the auxiliary collision warning method based on a monocular camera of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. That is, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments.

[0031] The present invention proposes an auxiliary collision warning method based on a monocular camera, which specifically includes:

[0032] Step 1: Obtain video stream data information through a monocular camera, intercept one frame from the video stream every 40ms, and put the frame into a frame queue of length 20 for subsequent processing.

[0033] Step 2: Get the scene image from the image frame queue and send it to the segmentation network for ground information detection to obtain ground coordinate point information.

[0034] Step 3: The target detection network obtains the scene image from the image frame queue, detects the target in the scene image, and provides the initial vehicle and human target detection frames.

[0035] The target detection network of the present invention adopts the SSD target detection algorithm. The network inputs an image of size 3×300×300. After two 3×3 convolutions and global pooling, a feature map of size 64×150×150 is obtained. After two more 3×3 convolutions and global pooling, a feature map of size 128×75×75 is obtained. After three more 3×3 convolutions and global pooling, a feature map of size 512×38×38 is obtained. Finally, after nine 3×3 convolutions and two fully connected layers, the target detection frames of the detected targets (initial vehicles and people) are obtained.

[0036] Step 4: Use the vertical coordinates of the lower bounding boxes of the initial vehicle and person detection frames as the starting point for traversal, and traverse the entire width of the detection frame. If a ground coordinate point exists within the frame, no correction is required. Otherwise, the detection frame is moved up and down until the ground coordinate point is within the range of (-h / 2, h / 2) above and below the height of the detection frame. Then, traverse again until a ground coordinate point exists within the detection frame, resulting in a corrected detection frame. Here, h is the height of the detection frame.

[0037] Step 5: Data association and IOU calculation to assign target IDs. The SSD target detection network is a multi-target detection network, so after SSD detects multiple targets in the current frame, it needs to inherit the same ID of the same target detected in the previous frame.

[0038] Step 6: Bird's-Eye View Odometry Algorithm

[0039] For all targets obtained in step 5, the distance is measured using the bird's-eye view ranging algorithm.

[0040] The following is the principle and implementation of the bird's-eye view ranging algorithm:

[0041] A bird's-eye view is typically viewed as an image of a scene scaled to a specific percentage, with each pixel uniformly representing a distance. Distance can be calculated by taking the pixel difference between the target and the camera. The camera's imaging principle states that both the intrinsic and extrinsic parameter matrices are full-rank reversible matrices. Therefore, their inverse matrices can be used to convert image pixel coordinates into world coordinates. This transformation is also known as the inverse perspective transformation.

[0042] The external parameter matrix T required to transform the world coordinate system to the camera coordinate system is only composed of the rotation matrix R, and the coordinate transformation formula can be simplified as follows:

[0043]

[0044] (Where Xc, Yc, Zc represent the coordinates of the X, Y, and Z axes in the camera coordinate system, respectively; Xw, Yw, Zw represent the coordinates of the X, Y, and Z axes in the world coordinate system, respectively; and R is the rotation matrix.)

[0045] According to the bird's-eye view, the final rotation matrix is composed of the rotation coordinate transformation matrix in the x-axis and y-axis directions.

[0046]

[0047] The rotation matrix R in formula (2) is obtained by the inner product of the transformation matrix of the pitch angle α and the transformation matrix of the yaw angle β. The rotation matrix R is multiplied by the internal parameter matrix M to form the coordinate system (X w ,Yw ,Z w ), to the entire transformation matrix process of the (u, v) coordinates on the image coordinate system, this process is also called the perspective transformation process. Combining the two transformation processes of the above camera, we can know that the perspective transformation process matrix Y p , and the calculation formula of the inverse perspective transformation matrix Y1 from image coordinates to world coordinates is as follows:

[0048] Y p =MR (3)

[0049] Y1=M -1 R -1 (4)

[0050] According to the vanishing point coordinates in the world coordinate system (the vanishing point refers to the point where two parallel lines intersect at a very far point in the image, which is called the vanishing point in the perspective image), we can get the coordinate range in the pixel image after perspective transformation, which can be used to perform inverse perspective transformation to form a bird's-eye view. At this time, we can directly ignore the vertical coordinate Z in the world coordinate system. w , select only the horizontal plane (X w Y w ) The range coordinates to be imaged (X max X min Y max Y min ) Generate a series of equally spaced point coordinates in the world coordinate system according to the imaging bird's-eye view size (nRows, nCols). The x and y coordinates of these points are:

[0051]

[0052]

[0053] Because the vehicle's camera's forward-facing perspective creates a certain downward angle when viewing objects on the ground, there are blind spots at close range, resulting in a fixed measurement error in the forward direction. Furthermore, due to a certain deviation between the camera's actual optical center and the measured optical center, measurement errors can also occur when detecting objects in the forward and side directions. Therefore, it is necessary to calibrate the horizontal and vertical coordinates of the measured target coordinates in the bird's-eye view.

[0054] OFFSET_Y is the coordinate value on the vertical axis that needs to be corrected, and image_height is the overall vertical coordinate height of the pixel image. In the two distance measurement images used for calibration and correction, the vertical coordinates of the target are y1 and y2. Based on these parameters, the ratio of the real distances of the targets in the two calibration images can be calculated, that is, the ratio between real distance1 and real distance2:

[0055]

[0056] Finally, the actual distance (LperP_Y) represented by the unit ordinate in the bird's-eye view can be calculated:

[0057]

[0058] The true distance (dist_Y) of any target on the Y axis can be calculated based on the calibrated ordinate error value and the unit distance of the ordinate pixel ratio:

[0059] dist_Y=(image_height+OFFSET_Y-y)*LperP_Y

[0060] The true distance dist_Y of the target in the longitudinal direction can be obtained, and the horizontal coordinate correction formula can be obtained according to the measured distance of the target (Detec_dis) and the horizontal coordinate x1

[0061] (OFFSET X +x1) 2 +dist_Y 2 =Detec_dis 2

[0062] Among them, OFFSET_X is the horizontal correction coordinate, x1 is the horizontal coordinate, dist_Y is the actual distance on the vertical coordinate, and Detec_dis is the measured distance of the target.

[0063] The formula for calculating the horizontal correction coordinate OFFSET_X can be solved by taking the square root of both sides at the same time.

[0064]

[0065] Thus, the value of the transverse coordinate ratio (LperP_X) can be obtained

[0066]

[0067] (where u0 is the measured optical center abscissa value)

[0068] The abscissa (dist_X) of the corrected target's true distance is obtained.

[0069] dist_X=(x+OFFSET X -u0)

[0070] After the calibration correction is completed, the true distance dist of the target in the pixel image can be calculated.

[0071]

[0072] Among them, dist_X is the horizontal coordinate of the corrected target true distance, and dist_Y is the vertical coordinate of the corrected target true distance.

[0073] Segmentation algorithms directly capture pixel-level ground information within an image. This ground information can be used to perform ground corrections on the target detection frame generated by the target detection algorithm. This allows bird's-eye view ranging algorithms, which directly use the target pixel coordinates, to achieve highly accurate ranging.

[0074] After the target detection algorithm and segmentation algorithm obtain the target detection frame and ground information, traversal begins. Generally, the vertical coordinate of the lower border of the target detection frame is used as the starting point of the traversal, and the traversal range is the width of the entire detection frame.

[0075] If the coordinates of the point exist in the ground information, no correction is required. Otherwise, the vertical coordinate is moved up or down until its range is equal to the height of the detection frame (i.e., plus or minus half). Then, the traversal is restarted until the coordinates of the point exist in the ground information, and the distance measurement line is drawn.

[0076] Step 7: Measure the distance based on the bird's-eye view distance measurement algorithm, and then determine whether the object is within the preset distance threshold. If it is within the preset distance threshold, proceed to step 8; otherwise, proceed to step 13.

[0077] Step 8: For targets within 10 meters of the main vehicle, the Kalman tracking algorithm is activated to perform continuous and rapid target predictions, obtaining target prediction frames. The corresponding target detection frames continuously fed into the target detection network are compared with the target ID and the target prediction frames of the Kalman tracking algorithm. The ECO filter is then used to calculate and score the frames using the optimal difference Newton iteration method. The scoring result determines whether to perform coordinate repair and scale factor update on the target currently being processed by the ECO tracking algorithm (this is done in parallel with coordinate repair).

[0078] Step 9: Determine whether coordinate repair is required based on the scoring result of step 8 and whether the conditions for coordinate repair are met. If necessary, proceed to step 11; otherwise, proceed to scale factor determination.

[0079] Step 10: Determine whether the scale factor needs to be updated based on the scoring result of step 8. If so, proceed to step 12; otherwise, proceed to step 13.

[0080] Step 11: Coordinate Correction Process

[0081] When the ECO tracking algorithm fails, that is, the target tracking frame deviates seriously from the rapidly moving target, the ECO tracking frame needs to be directly corrected to enable the tracking frame to keep up with the target better.

[0082] The condition for determining whether the ECO tracking frame deviates or fails is based on the cosine distance (also known as cosine similarity) between the latest detected sample frame sent by the SSD target detection algorithm and the latest tracked sample frame by ECO. It uses the cosine value of the angle between two vectors in vector space as a measure of the size of the difference between two individuals. The cosine distance focuses on the difference in direction and is independent of the value of the vector, only related to the direction of the vector. ssd If the target suddenly turns, the cosine distance will exceed the predetermined threshold, and it is preliminarily determined that the target has changed in direction, and the ECO algorithm is determined to have lost the target.

[0083] The target frame is then placed into the candidate tracking frame that requires coordinate correction, and the SSD frame is then scored and judged in the ECO tracking algorithm.

[0084] Scoring is to judge the SSD target detection frame I at this time ssd Filter ECO fed into the ECO algorithm fiter and optimal difference Newton iteration method ECO Newton If you score in the ssd Score, formula is: S ssd =ECO Newton (ECO fiter (I ssd If the absolute value of the interpolation of this score and the mean score of the neighboring historical tracking boxes is much larger than the historical standard deviation, it can be judged that the object has made a rapid turn or its motion state has changed, and ECO tracking is considered to have failed.

[0085] At this time, the result coordinates (x ssd ,y ssd ), where x ssd is the coordinate on the x-axis, y ssd The coordinates on the y-axis are selected by the data association algorithm and IOU calculation to determine the coordinates (x ssd ,y ssd ), and place it at the ECO prediction result coordinate (x pre,y pre ) is directly corrected to obtain the corrected coordinates of the target (x fix ,y fix ). The calculation formula is x fix =x ssd -x pre ,y fix =y ssd -y pre .

[0086] The corrected coordinates are used as the subsequent frames, and the results are directly added to the latest tracking frame. The coordinate correction stops when the score returns to normal when the SSD is sent to the target detection frame again.

[0087] Step 12: Scaling factor (a benchmark for data changes) update process

[0088] The original ECO algorithm predicts the size change of detected objects based on seven scale factors. In addition to the scale factor set to 1, there are three scale factors with a decreasing trend and three scale factors with an increasing trend: 0.975, 0.985, 0.9905, 1.01, 1.02, and 1.03.

[0089] ECO's scale update strategy makes predictions at these seven scales for each frame. However, in some specific situations in driving school scenarios, such as when a person leaves the side of a vehicle, the actual scale of the target person in the picture continues to increase as they continue to approach the vehicle. The scale reduction factor among the original seven scale factors is actually useless and affects the fluctuation of the tracking detection frame.

[0090] To address the above situation, a triple quadratic interpolation algorithm is designed to estimate and fill in the scale change factor between two large-interval sample points in the target frame.

[0091] The formula of the cubic double Herimit algorithm is: y(x)=y0a0(x)+y1a1(x)+y0β0(x)+y1β1(x).

[0092] (Where x represents the time when the SSD target detection is fed into the image frame, and y(x) represents the target box size of the tracked target in the fed frame. a0(x), a1(x), β0(x), and β1(x) are interpolation basis functions, all of which are polynomials of degree not exceeding three.)

[0093] When using the cubic double Hermit interpolation strategy for estimation in the driving school scenario, the target detection box given by the SSD target detection algorithm is first used to estimate the target's motion trend.

[0094] If the target detection box tends to become larger, the smaller scale factor in the ECO sample is discarded, and the scale factor estimated by Hermit interpolation is added to the scale factor with the larger trend.

[0095] If the target detection box tends to become smaller, the scale factor that becomes larger in the ECO sample is discarded, and the scale factor estimated by Hermit interpolation is added to the scale factor with a decreasing trend.

[0096] Step 13: Output the target tracking frame obtained by correcting the ECO tracking algorithm using the SSD detection algorithm, and perform distance determination using the bird's-eye view ranging algorithm. If the distance is less than the safe distance (1 meter for this system), a collision warning (forced braking and other measures) is initiated.

[0097] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An auxiliary collision warning method based on a monocular camera, characterized in that: The method comprises the following steps: Step 1: Intercept the input video stream data information obtained by the monocular camera, obtain the picture frame, and put the obtained picture frame into the picture frame queue; Step 2: The segmentation network obtains the scene image from the image frame queue and outputs the ground information in the scene image; Step 3: The target detection network obtains the scene image from the frame queue, detects the target in the scene image, and provides the initial vehicle and human target detection frames; Step 4: Combine the initial vehicle and human target detection frames with the ground information, and correct the initial vehicle and human target detection frames to fall back to the ground to obtain the corrected detection frames; Step 5: Perform data association and IOU calculation on the corrected detection box, and calculate and assign target IDs for multiple targets in the target detection network. Step 6: At the same time, the corrected detection frame is used to detect the target distance using the bird's-eye view ranging algorithm that adds ground information to obtain the target distance; Step 7: Determine whether each of the multiple targets is within a preset distance threshold. If so, proceed to step 8; otherwise, proceed to step 13. Step 8: For targets within a preset distance threshold from the host vehicle, the Kalman tracking algorithm is activated to perform continuous and rapid target predictions on the target to obtain a target prediction frame. The target detection frame of the corresponding target continuously fed into the target detection network is compared with the target prediction frame of the Kalman tracking algorithm through the target ID. The target is then passed through the ECO filter and the optimal difference Newton iteration method is used to calculate and score the target. Based on the scoring results, it is decided whether to perform coordinate repair and scale factor update on the target currently being processed by the ECO tracking algorithm. Step 9: At the same time, based on the scoring results of step 8, determine whether coordinate repair is needed and whether the conditions for coordinate repair are met. If necessary, proceed to step 11; otherwise, proceed to scale factor judgment. Step 10: Determine whether the scale factor needs to be updated based on the scoring result in step 8. If so, proceed to step 12; otherwise, proceed to step 13. Step 11: Correct the coordinates of the target detection frame according to the corresponding coordinates in the single-stage target detection algorithm SSD; Step 12: Start the Hermit interpolation algorithm to estimate the scale factor of the target detection box, replacing the original multi-scale factor of the Kalman filter; Step 13: Output the target prediction frame obtained by correcting the ECO tracking algorithm using the single-stage target detection algorithm SSD, and perform distance determination using the bird's-eye view ranging algorithm. If the distance is less than the safe distance, a collision warning is initiated.

2. The monocular camera-based auxiliary collision warning method according to claim 1, characterized in that: The step 1 specifically includes: intercepting the input video stream data information obtained by the monocular camera, intercepting one frame of picture from the video stream every 40ms, and placing the picture frame into a picture frame queue with a length of 20.

3. The monocular camera-based auxiliary collision warning method according to claim 2, characterized in that: The target detection network adopts a single-stage target detection algorithm SSD. The network input is a picture of size 3×300×300. After two 3×3 convolutions and global pooling, a feature map of size 64×150×150 is obtained. After two more 3×3 convolutions and global pooling, a feature map of size 128×75×75 is obtained. After three more 3×3 convolutions and global pooling, a feature map of size 512×38×38 is obtained. Finally, after nine 3×3 convolutions and two fully connected layers, the target detection frame of the detected target is obtained, where the detected target includes the initial vehicle and person.

4. The monocular camera-based auxiliary collision warning method according to claim 3, characterized in that: The step 4 specifically includes: using the obtained lower border vertical coordinates of the initial vehicle and person target detection frames as the starting point of the traversal, and the traversal range is the width of the entire target detection frame. If there is a ground coordinate point in it, no correction is required. Otherwise, the target detection frame is moved up and down until the ground coordinate point is within the range of positive and negative half (-h / 2, h / 2) above and below the height of the target detection frame, and then re-traversal is started until a ground coordinate point is found in the target detection frame, and the corrected detection frame is obtained, where h is the height of the target detection frame.

5. The monocular camera-based auxiliary collision warning method according to claim 4, characterized in that: In step 5, since the target detection network is a multi-target detection network, the single-stage target detection algorithm SSD needs to inherit the same ID of the same target detected in the previous frame after detecting multiple targets in the current frame.

6. The monocular camera-based auxiliary collision warning method according to claim 5, characterized in that: The scoring method in step 8 is: Scoring is to judge the target detection frame I at this time ssd Feed into the ECO tracking algorithm filter ECO fiter and optimal difference Newton iteration method ECO Newton If you score in the ssd Score, formula is: S ssd =ECO Newton (ECO fiter (I ssd If the absolute value of the interpolation of this score and the mean score of the neighboring historical target prediction boxes is greater than its historical standard deviation, it is judged that the target motion has turned rapidly or the motion state has changed, and the ECO tracking is considered to have failed.

7. The monocular camera-based auxiliary collision warning method according to claim 6, characterized in that: The preset distance threshold is 10 meters.

8. The monocular camera-based auxiliary collision warning method according to claim 7, characterized in that: The safety distance is 1 meter.

9. The monocular camera-based auxiliary collision warning method according to claim 8, characterized in that: The collision warning is a mandatory braking measure.

Images