Vehicle forward collision early warning method and device

Abstract

The invention discloses a vehicle forward collision early warning method and device, and belongs to the technical field of intelligent driving. According to the method, the monocular vision sensor is utilized, the number and types of the sensor are reduced, the system architecture is simplified, the cost and complexity are reduced, efficient and accurate front target detection and distance estimation can be realized, and the method is suitable for forward collision early warning of the unmanned mine card in a complex environment. According to the method, the time efficiency and the detection accuracy are comprehensively considered, firstly, the Haarlike features are combined with the AdaBoost classifier to quickly select the candidate region, then the classifier based on the histogram of oriented gradients and combined with the SVM is used to directly judge whether the vehicle target is a correct vehicle target in the candidate region set, and the real-time performance and the robustness of the system in the complex traffic environment are improved. And in combination with the feature matching and tracking algorithm of the multi-frame image, the phenomena of false alarm and missing alarm caused by instantaneous interference are reduced, and the accuracy and reliability of the early warning system are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to a vehicle forward collision warning method and device. BACKGROUND

[0002] Vehicle forward collision warning (FCW) is an active safety system that monitors the environment in front of the vehicle using sensors such as cameras and radars, predicts whether the vehicle will collide with the target in front (vehicles, pedestrians, etc.), and alerts the driver before the danger occurs.

[0003] Vehicle forward collision systems have been widely used in vehicle auxiliary driving systems to remind drivers of the risk of forward collision during driving. Developing an efficient and reliable forward collision warning system is crucial for vehicle automatic driving to identify collisions and reduce collision risks. Currently, vehicle forward collision warning technology mainly relies on multi-sensor fusion methods, including radar, LiDAR, ultrasonic sensors, and monocular or monocular vision sensors. However, in complex environments such as mines and construction sites, these sensors may be affected by various factors such as bad weather, dust, and light changes, resulting in decreased detection accuracy. In the application scenario of unmanned mine trucks, the accuracy and robustness of the forward collision warning system are required to be higher due to the lack of direct intervention by the driver. SUMMARY

[0004] To solve the above technical problems, the present application provides a vehicle forward collision warning method and device, which can realize efficient and accurate forward target detection and distance estimation, and is particularly suitable for forward collision warning of unmanned mine trucks in complex environments.

[0005] The technical solutions provided by the present application are as follows:

[0006] A vehicle forward collision warning method, the method comprising:

[0007] S1: establishing a mapping model of mapping the image pixel coordinate system to the vehicle body coordinate system according to the monocular camera installed on the unmanned vehicle at a designated position and the calibration object placed in front of the unmanned vehicle;

[0008] S2: acquiring a video stream in front of the unmanned vehicle captured by the monocular camera in real time, and intercepting a region of interest of the image contained in the video stream;

[0009] S3: extracting Haar_like features of the region of interest, and extracting candidate regions using a trained AdaBoost classifier based on the Haar_like features;

[0010] S4: extracting Hog features of the candidate region, and based on the Hog features, using a trained support vector machine classifier to extract a target region representing a target in front of the unmanned vehicle;

[0011] S5: tracking the target region using a tracker to obtain a series of target regions corresponding to the target in time sequence;

[0012] S6: calculating a distance from the target region corresponding target to the unmanned vehicle according to a position of the target region on the image and the mapping model established;

[0013] S7: calculating a relative collision time of the target and the unmanned vehicle according to a size change of a detection frame of the series of target regions of the target in time;

[0014] S8: performing a forward collision warning based on the distance and the relative collision time.

[0015] Further, the S1 comprises:

[0016] S11: installing a monocular camera at a rearview mirror position of the unmanned vehicle and placing a calibration object in front of the unmanned vehicle;

[0017] S12: performing distortion correction on the installed monocular camera and aligning an image center of the monocular camera to a front ground horizon vanishing point;

[0018] S13: finding pixel coordinates of four non-collinear selected points of the calibration object in an image pixel coordinate system on an image shot by the monocular camera and recording actual coordinates corresponding to the selected points in a vehicle body coordinate system;

[0019] S14: based on the pixel coordinates and the actual coordinates of the selected points, establishing a mapping model of mapping from the image pixel coordinate system to the vehicle body coordinate system according to a geometric relationship of camera imaging.

[0020] Further, the S2 comprises:

[0021] S21: obtaining a current image from a video stream in front of the unmanned vehicle shot by the monocular camera;

[0022] S22: determining a pixel coordinate at the ground horizon vanishing point on the current image and cutting a region of interest based on the pixel coordinate at the ground horizon vanishing point.

[0023] Further, the S3 comprises:

[0024] S31: calculating an integral image of the region of interest and calculating the Haar_like features based on the calculated integral image;

[0025] The Haar-like features include edge features and linear features;

[0026] S32: Based on the Haar-like features, the candidate regions are extracted using the trained Adaboost cascade classifier, which consists of multiple strong classifiers connected in series.

[0027] Furthermore, S4 includes:

[0028] S41: Set the block size, cell size, block sliding step size, and gradient interval;

[0029] S42: Slide the block on the candidate region according to the sliding step size, and divide the region within each block into several cell regions according to the size of the cells;

[0030] S43: Calculate the gradient direction and magnitude for each cell region within each block, and statistically analyze the gradient direction of each cell region within each block according to the gradient interval to obtain the Hog feature vector of each block;

[0031] S44: Concatenate the Hog feature vectors of all blocks to obtain the Hog features of the candidate region;

[0032] S45: Based on the Hog features, the target region is extracted using a trained support vector machine classifier.

[0033] Furthermore, the support vector machine classifier is pre-trained through the following process:

[0034] Construct a training set for training the support vector machine classifier;

[0035] The training set includes positive samples and negative samples, wherein the positive samples are images containing the target and the negative samples are images not containing the target;

[0036] Extract the Hog features of the positive and negative samples and add labels;

[0037] The support vector machine classifier is trained based on the Hog features of labeled positive and negative samples.

[0038] During training, a set kernel function is used to map the Hog features of positive and negative samples to a high-dimensional feature space.

[0039] Furthermore, S5 includes:

[0040] S51: Use a separate tracker to track each target area;

[0041] S52: The tracker determines the position and size of the target in the new frame image by calculating the similarity between the new frame image and the previous frame image, and obtains the target region of the new frame image;

[0042] S53: Set all pixels within the target area of ​​the newly tracked frame image to a specific value and perform masking;

[0043] S54: When the target region of the new frame image moves to the edge of the image, stop tracking and delete the target and the corresponding tracker;

[0044] S55: Determine whether the target region of the new frame image is the target based on the online supervisor equipped with each tracker; if not, mark a tracking error.

[0045] S56: When the number of tracking errors reaches a set threshold, delete the target and its corresponding tracker.

[0046] Furthermore, S6 includes:

[0047] S61: Obtain the bottom center point of the detection box of the target area, and calculate the pixel coordinates of the bottom center point in the image pixel coordinate system;

[0048] S62: Based on the mapping model, map the pixel coordinates of the bottom center point to the actual coordinates in the vehicle coordinate system;

[0049] S63: Calculate the distance from the target to the unmanned vehicle based on the coordinates of the unmanned vehicle in the body coordinate system and the actual coordinates of the bottom center point in the vehicle body coordinate system.

[0050] Furthermore, S7 includes:

[0051] S71: Calculate the pixel width of the detection box of the target region in the image at time t and time t-1. and ;

[0052] S72: When Greater than At that time, the relative collision time TTC between the target and the unmanned vehicle is calculated;

[0053]

[0054] in, Let t be the time interval between time t and time t-1.

[0055] A vehicle forward collision warning device, the device comprising:

[0056] The calibration module is used to establish a mapping model from the image pixel coordinate system to the vehicle coordinate system based on the monocular camera installed at a set position on the unmanned vehicle and the calibration object placed in front of the unmanned vehicle.

[0057] The region of interest acquisition module is used to acquire the video stream in front of the unmanned vehicle captured by the monocular camera in real time, and to extract the region of interest of the image contained in the video stream.

[0058] The first-level detection module is used to extract the Haar-like features of the region of interest, and based on the Haar-like features, use the trained AdaBoost classifier to extract candidate regions.

[0059] The second-level detection module is used to extract the Hog features of the candidate region, and based on the Hog features, to extract the target region representing the target in front of the unmanned vehicle using a trained support vector machine classifier.

[0060] The tracking module is used to track the target area using a tracker to obtain a series of target areas corresponding to the target in a time series.

[0061] The distance calculation module is used to calculate the distance from the target corresponding to the target area to the unmanned vehicle based on the position of the target area on the image and the established mapping model;

[0062] The relative collision time calculation module is used to calculate the relative collision time between the target and the unmanned vehicle based on the time-varying size changes of the detection boxes of a series of target areas corresponding to the target.

[0063] A forward collision warning module is used to provide forward collision warning based on the distance and the relative collision time.

[0064] Furthermore, the calibration module includes:

[0065] A preparation unit is used to install a monocular camera at the rearview mirror position of the unmanned vehicle and place a calibration object in front of the unmanned vehicle;

[0066] The correction unit is used to correct the distortion of the installed monocular camera and make the image center of the monocular camera aligned with the vanishing point of the horizon.

[0067] The coordinate acquisition unit is used to find the pixel coordinates of four non-collinear selected points of the calibration object in the image pixel coordinate system on the image captured by the monocular camera, and record the actual coordinates of the selected points in the vehicle coordinate system.

[0068] The mapping unit is used to establish a mapping model from the image pixel coordinate system to the vehicle coordinate system based on the pixel coordinates and actual coordinates of the selected point and according to the geometric relationship of camera imaging.

[0069] Furthermore, the region of interest acquisition module includes:

[0070] An image extraction unit is used to acquire the current image from the video stream captured by the monocular camera in front of the unmanned vehicle;

[0071] The region of interest (ROI) cropping unit is used to determine the pixel coordinates of the vanishing horizon point on the current image and crop the ROI based on the pixel coordinates of the vanishing horizon point.

[0072] Furthermore, the first-level detection module includes:

[0073] The Haar_like feature calculation unit is used to calculate the integral map of the region of interest and calculate the Haar_like features based on the calculated integral map;

[0074] The Haar-like features include edge features and linear features;

[0075] The candidate region extraction unit is used to extract the candidate region based on the Haar_like features using a trained Adaboost cascade classifier composed of multiple strong classifiers connected in series.

[0076] Furthermore, the second-level detection module includes:

[0077] The parameter setting unit is used to set the block size, cell size, block sliding step size, and gradient interval.

[0078] A partitioning unit is used to slide the block on the candidate region according to the sliding step size, and to divide the region within each block into several cell regions according to the size of the cell;

[0079] The Hog feature calculation unit is used to calculate the gradient direction and magnitude of each cell region within each block, and to statistically analyze the gradient direction of each cell region within each block according to the gradient interval to obtain the Hog feature vector of each block.

[0080] A concatenation unit is used to concatenate the Hog feature vectors of all blocks to obtain the Hog features of the candidate region;

[0081] The target region extraction unit is used to extract the target region based on the Hog features using a trained support vector machine classifier.

[0082] Furthermore, the support vector machine classifier is pre-trained through the following process:

[0083] Construct a training set for training the support vector machine classifier;

[0084] The training set includes positive samples and negative samples, wherein the positive samples are images containing the target and the negative samples are images not containing the target;

[0085] Extract the Hog features of the positive and negative samples and add labels;

[0086] The support vector machine classifier is trained based on the Hog features of labeled positive and negative samples.

[0087] During training, a set kernel function is used to map the Hog features of positive and negative samples to a high-dimensional feature space.

[0088] Furthermore, the tracking module includes:

[0089] The tracker setting unit is used to track each target area using an independent tracker;

[0090] The tracking unit is used by the tracker to determine the position and size of the target in the new frame image by calculating the similarity between the new frame image and the previous frame image, thereby obtaining the target region of the new frame image;

[0091] The masking unit is used to set all pixels within the target area of ​​the tracked new frame image to a specific value for masking processing.

[0092] The boundary processing unit is used to stop tracking and delete the target and its corresponding tracker when the target area of ​​the new frame image moves to the edge of the image.

[0093] The online supervision unit is used to determine whether the target region of the new frame image is the target based on the online supervisor equipped in each tracker; if not, a tracking error is marked.

[0094] The error handling unit is used to delete the target and its corresponding tracker when the number of tracking errors reaches a set threshold.

[0095] Furthermore, the distance calculation module includes:

[0096] The center point calculation unit is used to obtain the bottom center point of the detection box of the target area and calculate the pixel coordinates of the bottom center point in the image pixel coordinate system;

[0097] A coordinate mapping unit is used to map the pixel coordinates of the bottom center point to the actual coordinates in the vehicle coordinate system according to the mapping model.

[0098] The distance calculation unit is used to calculate the distance from the target to the unmanned vehicle based on the coordinates of the unmanned vehicle in the body coordinate system and the actual coordinates of the bottom center point in the vehicle body coordinate system.

[0099] Furthermore, the relative collision time calculation module includes:

[0100] The detection box width calculation unit is used to calculate the pixel width of the detection box of the target region in the image at time t and time t-1. and ;

[0101] The relative collision time calculation unit is used when Greater than At that time, the relative collision time TTC between the target and the unmanned vehicle is calculated;

[0102]

[0103] in, Let t be the time interval between time t and time t-1.

[0104] The present invention has the following beneficial effects:

[0105] This invention utilizes a monocular vision sensor, combined with advanced image processing and machine learning algorithms, to achieve efficient and accurate forward target detection and distance estimation, making it particularly suitable for forward collision warning of unmanned mining trucks in complex environments. Compared to multi-sensor fusion solutions, the monocular vision sensor solution reduces the number and types of sensors, simplifies the system architecture, and lowers cost and complexity. This invention comprehensively considers time efficiency and detection accuracy, employing two classifiers based on different features for rapid vehicle detection. First, Haar-like features combined with an AdaBoost classifier are used to quickly select candidate regions. Then, a classifier based on histogram of oriented gradients (HGP) features combined with an SVM is used to directly determine whether a vehicle is the correct target within the candidate region set, improving the system's real-time performance and robustness in complex traffic environments. This invention integrates an improved tracking algorithm based on kernel correlation filters after the two-stage detectors, introducing an online supervision mechanism in the tracking module for multi-target management. The online supervisor uses histogram of oriented gradients (HGP) features for online target confirmation. Combining feature matching and tracking algorithms across multiple frames reduces false alarms and missed alarms caused by transient interference, improving the accuracy and reliability of the warning system. Attached Figure Description

[0106] Figure 1This is a flowchart of the vehicle forward collision warning method of the present invention;

[0107] Figure 2 This is a schematic diagram of the calibration scheme;

[0108] Figure 3 This is a flowchart illustrating the two-stage detection module and tracking module of the present invention;

[0109] Figure 4 This is a schematic diagram of the vehicle forward collision warning device of the present invention. Detailed Implementation

[0110] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0111] Example 1:

[0112] This invention provides a method for forward collision warning of vehicles, such as... Figure 1 As shown, the method includes:

[0113] S1: Based on the monocular camera installed at the designated position on the unmanned vehicle and the calibration object placed in front of the unmanned vehicle, establish a mapping model from the image pixel coordinate system to the vehicle coordinate system.

[0114] One way to implement this step is to: install and fix the camera at a suitable position on the unmanned mining truck, place a calibration object in front of the vehicle, use an algorithm to find the four non-collinear pixel coordinate points of the calibration object, record the actual coordinate points corresponding to them in the vehicle coordinate system, and establish a model for mapping the image pixel coordinate system to the vehicle coordinate system based on the geometric relationship of the camera imaging.

[0115] Specifically, including:

[0116] S11: Install a monocular camera in the rearview mirror position of unmanned vehicles such as unmanned mining trucks, and place the calibration object in front of the unmanned vehicle.

[0117] In complex environments such as mining environments, the setup of cameras and calibration objects is particularly important to ensure that high detection accuracy can be maintained even under adverse weather conditions and changes in lighting.

[0118] In one example, the calibrator is obtained through... Figure 2 The four marker circles of a specific color (e.g., red) are used to represent the location of the unmanned vehicle. The origin is the center point of the target detection box. VET_DISTANCE is the distance between the unmanned vehicle and the target in front (the target person or vehicle, etc., generally represented by the four marker circles shown). VET_LENGTH_NEAR and VET_HEIGHT are the length and height of the target vehicle, respectively.

[0119] S12: Perform distortion correction on the installed monocular camera and align the image center of the monocular camera with the vanishing point of the horizon.

[0120] In complex environments, camera distortion may occur. Distortion correction should be performed immediately after camera installation to ensure image data accuracy. The camera should be positioned so that the image center is aligned with the vanishing horizon in the distance. The pixel coordinates of the vanishing horizon should be recorded for subsequent extraction of the region of interest for vehicle detection.

[0121] S13: Use the algorithm to find the pixel coordinates of four non-collinear selected points of the calibration object in the image pixel coordinate system on the image captured by the monocular camera, and record the actual coordinates of the selected points in the vehicle coordinate system.

[0122] In this invention, the vehicle coordinate system represents the application of the world coordinate system in vehicle-related scenarios. The vehicle coordinate system typically uses the vehicle itself (i.e., the unmanned vehicle described in this invention) as a reference, defining a specific origin and coordinate axis directions, which can intuitively reflect the position information of the target vehicle relative to the vehicle itself. In one example, the y-axis of the vehicle coordinate system points towards the vehicle's forward direction, and the x-axis is perpendicular to the y-axis and located on the same horizontal plane.

[0123] Taking the aforementioned four marker circles as examples, the following steps are performed: Marker circle region extraction and centroid calculation are conducted. Given the relatively obvious color characteristics of the marker circles in the image, this scheme uses the Lab color space for marker region extraction. The Lab color space has advantages such as a wide color gamut and device independence, enabling more accurate description of color information. By setting an appropriate threshold range, the region corresponding to the 'a' component (representing the red range) of the marker circle in the Lab color space is extracted, thereby separating the marker circle from the complex background. The centroid of the extracted marker circle region is then calculated; the centroid is the coordinate of the marker circle's center in the image pixel coordinate system. Common image processing algorithms can be used for centroid calculation, such as calculating the average value of the region's pixel coordinates, to ensure accurate determination of the marker circle's center position.

[0124] S14: Based on the pixel coordinates and actual coordinates of the selected point, and according to the geometric relationship of camera imaging, establish a mapping model from the image pixel coordinate system to the vehicle coordinate system, thereby obtaining the transformation relationship between the two.

[0125] Specifically, the derivation process of the mapping model is as follows:

[0126] Based on the assumption of flat ground, that is, assuming that the ground on which the vehicle travels is a plane, and on the basis of the pinhole imaging model, combined with the optical data of the camera (such as focal length, pixel size, etc.) and the actual installation structure (such as the installation height of the camera on the vehicle body, pitch angle, etc.), a mapping model from the image coordinate system to the vehicle coordinate system is derived, as shown in formula (1). This mapping model describes the transformation relationship between image pixel coordinates and vehicle coordinates, and is the theoretical basis for distance calculation.

[0127] (1)

[0128] in: Image pixel coordinates, Vehicle body coordinates; homography matrix Used to describe the projection transformation relationship from "world point to image point"; , , , , , These are the first two rows and first two columns of the homography matrix (describing rotation, scaling, and translation within the plane); , The main control is scaling; , Mainly controls rotation; , Mainly controls translation; , , The elements in the third column of the homography matrix (describing the perspective offset between planes); , Directly related to perspective distortion; It is usually related to the overall perspective intensity. After normalization, perspective distortion is... , leading.

[0129] In plane geometry, a single point provides two equations, while the centroid coordinates of the four marker circles used in this invention provide eight equations. By establishing and solving the system of equations, eight parameters required for model transformation can be determined. These parameters accurately reflect the correspondence between the image pixel coordinate system and the vehicle body coordinate system, providing a precise transformation basis for subsequent distance calculations.

[0130] This invention performs monocular vision calibration, which requires locating four feature points of a ground marker using the Lab color space as the basis for establishing a mapping relationship. It uses four simple red circular markers to extract feature points using the Lab color space. The calibration scheme is simple and the feature point extraction is not limited by the location or environment.

[0131] S2: Real-time acquisition of video streams captured by a monocular camera in front of the unmanned vehicle, and extraction of regions of interest from the images contained in the video stream.

[0132] In one example, the specific implementation of S2 includes:

[0133] S21: Acquire the current image from the video stream in front of the unmanned vehicle captured by the monocular camera.

[0134] S22: Determine the pixel coordinates of the vanishing horizon point on the current image, and extract the region of interest based on the pixel coordinates of the vanishing horizon point.

[0135] For example, the pixel coordinates of the vanishing point can be recorded. The value of the axis is then used to crop the image. axis The following areas are designated as regions of interest for vehicle detection.

[0136] S3: Extract Haar-like features from the region of interest, and use the trained AdaBoost classifier to extract candidate regions based on the Haar-like features.

[0137] This step involves identifying candidate regions using a primary classifier, specifically the AdaBoost classifier based on a cascaded model. Features employed include edge and linear features from Haar-like features (image feature descriptors used for object detection and recognition). The integral image of the sample features is quickly calculated, accelerating image feature extraction.

[0138] Specific methods include:

[0139] S31: Calculate the integral map of the region of interest and calculate the Haar-like features based on the calculated integral map.

[0140] In Haar-like features, this invention selects edge features and linear features to describe sample images, effectively capturing key structural information in vehicle images to distinguish between vehicle and non-vehicle regions. To quickly calculate the Haar-like features of samples, an integral image is used. An integral image is a data structure that can calculate the pixel sum of any rectangular region in an image in constant time. By pre-calculating the integral images of the sample images, the calculation process of Haar-like features is greatly accelerated, reducing the time consumed by feature extraction. Using integral images, the Haar-like features of samples are calculated efficiently. For each image, based on its integral image, the values ​​of each Haar-like feature are quickly obtained through computation, thus enabling the determination of possible vehicle candidate regions in the first-level classifier stage with minimal time.

[0141] S32: Based on Haar-like features, candidate regions are extracted using an Adaboost cascade classifier, which is composed of multiple strong classifiers connected in series after training.

[0142] This invention employs the AdaBoost classifier, based on a cascaded model, as the primary classifier. The cascaded classifier consists of multiple strong classifiers connected in series. Each strong classifier judges a sample, and only if the current strong classifier classifies it as positive will the sample be sent to the next strong classifier for further judgment. AdaBoost is an iterative algorithm; during training, each strong classifier is carefully trained to achieve high accuracy in judging negative samples. When a strong classifier at one level classifies a sample as negative, the next level classifier will not continue processing that sample.

[0143] The Adaboost classifier needs to be trained before use. During training, samples need to be collected and preprocessed first.

[0144] Vehicle positive and negative samples were collected for training the AdaBoost classifier. Positive samples are images containing the target (vehicle or person, etc.), while negative samples are images without the target. The ratio of positive to negative samples was set to approximately 1:3, which helps train a classifier with stronger ability to distinguish between positive and negative samples. The collected positive samples were uniformly scaled to 24x24 pixels. This uniform size facilitates subsequent feature extraction and classifier training, and also meets the requirements for Haar-like feature calculation. Then, Haar-like features were extracted as described above, and the AdaBoost classifier was trained.

[0145] This invention uses the aforementioned cascaded Adaboost classifier as the primary classifier, which can accelerate detection speed and control the false detection rate. In traffic scenarios, most areas in an image are negative samples (i.e., do not contain vehicles or other targets). This characteristic of cascaded classifiers allows for the discarding of a large number of negative samples in the initial stages of classification, avoiding unnecessary subsequent computation and accelerating the detection speed. Only samples classified as positive by the previous classifier are allowed to proceed to the next classifier for further evaluation, ensuring that the false detection rate of potential vehicle areas is effectively reduced during the initial vehicle detection stage, thus improving the accuracy of the detection results.

[0146] S4: Extract the Hog features of the candidate regions, and based on the Hog features, use the trained support vector machine classifier to extract the target regions representing the targets in front of the unmanned vehicles.

[0147] This invention uses a Support Vector Machine (SVM) classifier as a secondary detector to further detect the candidate regions output by the primary detector, thereby obtaining the target region. The feature used is the Hog feature (Histogram of Oriented Gradients).

[0148] This invention calculates Hog features using positive and negative sample sets to obtain Hog ​​feature descriptors. The calculated Hog features are then trained using a support vector machine classifier. The trained Hog feature support vector machine classifier is applied to candidate regions determined by the primary classifier to filter false targets and accelerate the detection speed, thus obtaining the final result of the detection stage.

[0149] Specific implementation methods include:

[0150] S41: Set the block size, cell size, block sliding step size, and gradient interval.

[0151] This invention sets the detection window size to 56x56 pixels. This size is determined based on statistical analysis of vehicle target size and actual detection requirements, ensuring good coverage of the vehicle target while avoiding the impact of excessively large or small windows on feature extraction. The block size is set to 16x16 pixels, and the cell size to 8x8 pixels. A block is one of the basic units for Hog feature calculation, while a cell is a smaller division within a block. Appropriate block and cell sizes effectively capture local gradient information in the vehicle image. The block sliding step size is set to 8x8 pixels, and the number of gradient intervals is set to 9. These intervals are used to discretize and statistically analyze the gradient direction; nine intervals can describe the changes in gradient direction in detail, providing sufficient directional information for Hog features.

[0152] S42: Slide the block across the candidate region according to the sliding step size, and divide the region within each block into several cell regions according to the size of the cell.

[0153] S43: Calculate the gradient direction and magnitude for each cell region within each block, and statistically analyze the gradient directions of each cell region within each block according to the gradient interval to obtain the Hog feature vector of each block.

[0154] S44: Concatenate the Hog feature vectors of all blocks to obtain the Hog features of the candidate region.

[0155] The aforementioned steps S42-S44 are used to generate Hog feature descriptors: based on the parameters set above, the gradient direction and magnitude within the detection window of the image are calculated, and the distribution of the gradient direction within each cell in each interval is statistically analyzed to obtain the Hog feature vector of each block. The Hog feature vectors of all blocks are concatenated to form the complete Hog feature descriptor of the sample image.

[0156] S45: Based on Hog ​​features, the target region is extracted using a trained support vector machine classifier.

[0157] Support Vector Machine (SVM) classifiers need to be trained before use. The training process is as follows:

[0158] 1. Construct a training set for training a support vector machine classifier.

[0159] The training set includes positive samples and negative samples. Positive samples are images containing the target, and negative samples are images that do not contain the target.

[0160] 2. Extract the Hog features of positive and negative samples and add labels.

[0161] Based on the aforementioned method, Hog features of positive and negative samples are extracted. All Hog feature vectors calculated from the positive and negative sample sets are organized, and corresponding sample labels are added to each feature vector (positive sample label is 1, negative sample label is -1) to form a training dataset.

[0162] 3. The support vector machine classifier is trained offline based on the Hog features of the labeled positive and negative samples.

[0163] During training, a suitable kernel function (such as a linear kernel function or radial basis function, which can be selected based on the characteristics of the actual data and experimental results) is used to map the samples to a high-dimensional feature space to solve the problem of linear inseparability. After training, an SVM classifier model based on Hog ​​features is obtained that can accurately distinguish between positive and negative vehicle samples.

[0164] This invention uses a Support Vector Machine (SVM) classifier as a secondary detector, which accelerates detection speed and filters out false targets: the trained SVM classifier based on HOG features operates only on candidate regions identified by the primary classifier. The primary classifier has already performed preliminary screening of the image, identifying candidate regions that may contain vehicles. These regions are significantly fewer than the total number of regions in the entire image, thus improving detection speed. Each candidate region is divided according to a set detection window size, its HOG feature vector is extracted, and input into the trained SVM classifier for judgment. The SVM classifier classifies each candidate region according to the learned classification rules, filtering out false targets (i.e., candidate regions judged as negative samples), and retaining only the candidate regions judged as positive samples as the final result of the detection stage.

[0165] S5: Use a tracker to track the target area and obtain a series of target areas corresponding to the target in the time series.

[0166] After the detection phase of this invention completes the target detection operation in the image, all detected target information is sent to the multi-target tracking management module. This target information typically includes key parameters such as the target's position coordinates in the image and the target's size, providing an initial basis for subsequent target tracking. When the tracker updates the target region in the next frame, it synchronously masks the target region already in the tracking management queue in the original image.

[0167] The specific implementation method is as follows Figure 3 As shown, it includes the following steps:

[0168] S51: Use a separate tracker to track each target area.

[0169] In the multi-target tracking management module, an independent tracker is initialized for each input target. This tracker employs an improved version of the kernel correlation filter. The improved kernel correlation filter, based on the traditional kernel correlation filter, is optimized for target characteristics in real-world application scenarios (such as changes in the appearance and movement patterns of vehicle targets). For example, the kernel function of the filter is improved to better adapt to changes in the target's appearance, thereby enhancing tracking accuracy and robustness.

[0170] Specifically, for multi-target tracking scenarios, a tracking queue is constructed. This queue exists in a first-in, first-out (FIFO) or other reasonable data structure format, with each element in the queue corresponding to an independent tracker, and each tracker specifically responsible for tracking a particular target. This one-to-one tracking method ensures that each target can be accurately tracked and managed.

[0171] S52: The tracker determines the position and size of the target in the new frame image by calculating the similarity between the new frame image and the previous frame image, and obtains the target region in the new frame image.

[0172] Specifically, when processing the next frame of the image, each tracker updates the target region according to its own algorithm model. The tracker calculates the similarity between the target region in the current frame and the target region in the previous frame, and uses an improved kernel correlation filtering algorithm to determine the new position and size of the target in the current frame, thereby achieving dynamic updating of the target region.

[0173] S53: Set all pixels within the target area of ​​the tracked new frame image to a specific value and perform masking.

[0174] Specifically, while the tracker updates the target region, it masks the target region already in the tracking management queue in the original image. For example, it sets all pixel values ​​of the corresponding target region in the original image to a specific value (255 in this embodiment). This operation can be quickly implemented using image processing algorithms, such as iterating through all pixels within the target region and modifying their pixel values ​​to 255.

[0175] S54: When the target region of a new frame image moves to the edge of the image, tracking is stopped, and the target and its corresponding tracker are deleted.

[0176] This step handles boundary targets: Considering the specific characteristics of traffic scenarios, when a target gradually moves to the image boundary region, based on the target's motion patterns and actual needs within the traffic scene, it can be reasonably determined that the target has exceeded the effective tracking range and no longer needs to be tracked. At this point, the tracker corresponding to the target is promptly removed from the tracking queue to free up system resources, improve tracking efficiency, and avoid the impact of invalid tracking on system performance.

[0177] S55: Determine whether the target area of ​​the new frame image is the target based on the online supervisor equipped with each tracker. If not, mark a tracking error.

[0178] To achieve long-term tracking of multiple targets and at multiple scales, this invention equips each tracker in the tracking queue with an online supervisor (an SVM classifier based on Hog ​​features). This online supervisor runs synchronously with the corresponding tracker, monitoring and evaluating the tracker's performance in real time. The introduction of the online supervisor provides a dynamic and real-time mechanism for tracker performance evaluation, helping to promptly identify problems that arise during the tracking process.

[0179] The online supervisor uses histogram of oriented gradients (HOR) features to describe the image features of the tracking region (similar to the aforementioned Hog features). The specific operation is as follows: For each small tracking region, the gradient information of each pixel within the region is calculated, including the magnitude and direction of the gradient. The gradient direction is evenly divided into 9 intervals, and the number of pixels in each interval is counted. Based on the statistical results, a statistical histogram of the tracking region is generated. This histogram serves as the feature vector of the tracking region for subsequent classification.

[0180] Then, the SVM algorithm is used to classify the image within the tracking area based on the extracted feature vector. SVM, as a powerful classification algorithm, can build an effective classification model by learning the feature distribution of positive and negative samples. In this embodiment, the SVM classifier is used to determine whether the target currently being tracked by the tracker is a vehicle or other similar object.

[0181] When the online monitor determines that a tracker is tracking a target that is not a vehicle or similar object, it considers that the tracker has made a tracking error. At this time, a penalty point is deducted from the tracker for each error detected.

[0182] S56: When the number of tracking errors reaches a set threshold, delete the target and its corresponding tracker.

[0183] When a tracker's accumulated score reaches 5 points, the online monitor will immediately send an application to the multi-target tracking management module to cancel the tracker's tracking qualification and remove it from the tracking queue. This penalty and removal mechanism can effectively avoid tracking drift caused by trackers tracking the wrong target for a long time, ensuring that trackers in the tracking queue always accurately track valid targets.

[0184] This invention utilizes a two-stage detector and tracking module to perform real-time detection and tracking of targets such as vehicles ahead. Combined with the improved kernel correlation filter multi-scale, multi-target long-term tracking method mentioned earlier, it ensures accurate and stable acquisition of the precise location of vehicle target detection boxes on the image. The detection boxes should be able to completely define the targets such as vehicles ahead, providing a reliable basis for subsequent analysis of changes in the size of the detection boxes.

[0185] S6: Based on the location of the target region in the image and the established mapping model, calculate the distance from the target region to the unmanned vehicle.

[0186] In this step, the coordinates of the bottom center point pixel of the target detection box in the actual vehicle coordinate system can be calculated based on the position of the target detection box on the image, according to the mapping relationship obtained by the calibration model. The distance from the target vehicle to the vehicle can then be directly calculated using the two-point distance formula.

[0187] Specific implementation methods include:

[0188] S61: Obtain the bottom center point of the detection box of the target region and calculate the pixel coordinates of the bottom center point in the image pixel coordinate system.

[0189] Based on the aforementioned two-stage detector and tracking module, target detection bounding boxes of the vehicle ahead are acquired in the image, clarifying the specific location of the bounding boxes within the image. From the acquired target detection bounding boxes, the bottom center point is selected as a key point, and its pixel coordinates in the image pixel coordinate system are determined. The bottom center point is chosen as the key point because it has relatively stable features in the vehicle image and is strongly correlated with the vehicle's position in the real world, which helps improve the accuracy of distance calculation.

[0190] S62: Based on the mapping model, map the pixel coordinates of the bottom center point to the actual coordinates in the vehicle coordinate system.

[0191] S63: Calculate the distance from the target to the unmanned vehicle based on the coordinates of the unmanned vehicle in the body coordinate system and the actual coordinates of the bottom center point in the vehicle body coordinate system.

[0192] This distance can be directly calculated using the distance formula between two points (the coordinates of the unmanned vehicle in the body coordinate system and the actual coordinates of the bottom center point in the vehicle body coordinate system).

[0193] S7: Calculate the relative collision time between the target and the unmanned vehicle based on the changes in the size of the detection boxes of a series of target areas corresponding to the target over time.

[0194] This invention obtains the position of the target detection box on the image based on the aforementioned steps, and statistically analyzes the change in the size of the detection box for each target over a certain time interval. If there is a risk of forward collision, the detection box will gradually increase in size. The relative collision time is calculated from the change in the detection box.

[0195] Specific implementation methods include:

[0196] S71: Calculate the pixel width of the detection box of the target region in the image at time t and time t-1. and .

[0197] A suitable time interval can be set to continuously count the size of the detection box for each target within that interval. During vehicle movement, if there is a risk of forward collision, according to the principle of perspective projection, as the distance between the vehicle and targets such as vehicles ahead gradually decreases, the area occupied by the target detection box in the image will gradually increase. That is, the width and height of the detection box will usually show a trend of gradually increasing from small to large. Focusing on the change in the width of the detection box is crucial to more intuitively reflect the change in vehicle distance.

[0198] Specifically, recorded in The pixel width occupied by the target detection bounding box in the image at time is Simultaneously record At time t, the pixel width occupied by the target detection box in the image is 1. By comparing the width of the detection box at these two moments, we can clearly understand how the size of the detection box has changed.

[0199] S72: When Greater than The relative collision time (TTC) between the target and the unmanned vehicle is calculated when the detection box width is detected to be changing from small to large, indicating that the distance between the vehicle and the vehicle in front is gradually decreasing and there is a potential collision risk.

[0200]

[0201] in, Let t be the time interval between time t and time t-1.

[0202] S8: Forward collision warning based on distance and relative collision time.

[0203] This step combines the calculated distance between the vehicle and the target ahead, as well as the relative collision time between the vehicle and the target ahead, to make a judgment. For example, if either result reaches the set warning threshold, a forward collision warning signal will be issued.

[0204] This invention utilizes a monocular vision sensor, combined with advanced image processing and machine learning algorithms, to achieve efficient and accurate forward target detection and distance estimation, making it particularly suitable for forward collision warning of unmanned mining trucks in complex environments. Compared to multi-sensor fusion solutions, the monocular vision sensor solution reduces the number and types of sensors, simplifies the system architecture, and lowers cost and complexity. This invention comprehensively considers time efficiency and detection accuracy, employing two classifiers based on different features for rapid vehicle detection. First, Haar-like features combined with an AdaBoost classifier are used to quickly select candidate regions. Then, a classifier based on histogram of oriented gradients (HOR) features combined with an SVM is used to directly determine whether a vehicle target is correct within the candidate region set, improving the system's real-time performance and robustness in complex traffic environments. Furthermore, unlike common two-stage detection methods, this invention uses multi-threading to accelerate the confirmation of targets within the candidate region set, as the regions are independent of each other. This reduces the false detection rate while increasing detection speed, thus meeting the real-time operation requirements of the system.

[0205] This invention integrates an improved tracking algorithm based on kernel correlation filters after a two-stage detector. An online supervision mechanism is introduced into the tracking module for multi-target management, with the online supervisor using gradient histogram features for online target confirmation. Combining feature matching from multiple frames with the tracking algorithm reduces false alarms and missed alarms caused by transient interference, improving the accuracy and reliability of the early warning system. Furthermore, instead of using a fixed frame interval for confirmation, a supervisor is immediately activated as soon as the tracker begins tracking the target to perform real-time monitoring and confirmation without any intervening frames. Once the error counter in the supervisor reaches a set threshold, the tracker is removed from the tracking queue, thus avoiding tracking drift and improving tracking accuracy.

[0206] Example 2:

[0207] This invention provides a vehicle forward collision warning device, such as... Figure 4 As shown, the device includes:

[0208] The calibration module 1 is used to establish a mapping model from the image pixel coordinate system to the vehicle coordinate system based on the monocular camera installed at a set position on the unmanned vehicle and the calibration object placed in front of the unmanned vehicle.

[0209] The Region of Interest (ROI) acquisition module 2 is used to acquire the video stream in front of the unmanned vehicle captured by the monocular camera in real time, and to extract the region of interest from the image contained in the video stream.

[0210] The first-level detection module 3 is used to extract Haar-like features of the region of interest, and based on the Haar-like features, to extract candidate regions using the trained AdaBoost classifier.

[0211] The second-level detection module 4 is used to extract the Hog features of the candidate region, and based on the Hog features, the trained support vector machine classifier is used to extract the target region representing the target in front of the unmanned vehicle.

[0212] Tracking module 5 is used to track the target area using a tracker to obtain a series of target areas corresponding to the target in the time series.

[0213] The distance calculation module 6 is used to calculate the distance from the target to the unmanned vehicle based on the position of the target area on the image and the established mapping model.

[0214] The relative collision time calculation module 7 is used to calculate the relative collision time between the target and the unmanned vehicle based on the time-varying size changes of the detection boxes of a series of target areas corresponding to the target.

[0215] Forward collision warning module 8 is used to provide forward collision warning based on distance and relative collision time.

[0216] As an example, the calibration module includes:

[0217] The preparation unit is used to install a monocular camera in the rearview mirror position of the unmanned vehicle and place a calibration object in front of the unmanned vehicle.

[0218] The correction unit is used to correct distortion of the installed monocular camera and make the image center of the monocular camera aligned with the vanishing point of the horizon.

[0219] The coordinate acquisition unit is used to find the pixel coordinates of four non-collinear selected points of the calibration object in the image pixel coordinate system on the image captured by the monocular camera, and record the actual coordinates of the selected points in the vehicle coordinate system.

[0220] The mapping unit is used to establish a mapping model from the image pixel coordinate system to the vehicle coordinate system based on the pixel coordinates of selected points and the actual coordinates, according to the geometric relationship of camera imaging.

[0221] Furthermore, the region of interest acquisition module includes:

[0222] The image extraction unit is used to acquire the current image from the video stream in front of the unmanned vehicle captured by a monocular camera.

[0223] The region of interest (ROI) cropping unit is used to determine the pixel coordinates of the vanishing horizon point on the current image and crop the ROI based on the pixel coordinates of the vanishing horizon point.

[0224] Specifically, the aforementioned first-level detection module includes:

[0225] The Haar_like feature calculation unit is used to calculate the integral map of the region of interest and to calculate Haar_like features based on the calculated integral map.

[0226] Haar-like features include edge features and linear features.

[0227] The candidate region extraction unit is used to extract candidate regions based on Haar-like features using a trained Adaboost cascade classifier, which consists of multiple strong classifiers connected in series.

[0228] The second-level detection module includes:

[0229] The parameter setting unit is used to set the block size, cell size, block sliding step size, and gradient interval.

[0230] The partitioning unit is used to slide the block across the candidate region according to the sliding step size, and divide the region within each block into several cell regions according to the cell size.

[0231] The Hog feature calculation unit is used to calculate the gradient direction and magnitude of each cell region within each block, and to statistically analyze the gradient direction of each cell region within each block according to the gradient interval to obtain the Hog feature vector of each block.

[0232] The concatenation unit is used to concatenate the Hog feature vectors of all blocks to obtain the Hog features of the candidate region.

[0233] The target region extraction unit is used to extract the target region based on Hog ​​features using a trained support vector machine classifier.

[0234] The support vector machine classifier is pre-trained through the following process:

[0235] Construct a training set for training a support vector machine classifier.

[0236] The training set includes positive samples and negative samples. Positive samples are images containing the target, and negative samples are images that do not contain the target.

[0237] Extract Hog features from positive and negative samples and add labels.

[0238] The support vector machine classifier is trained based on the Hog features of labeled positive and negative samples.

[0239] During training, a set kernel function is used to map the Hog features of positive and negative samples to a high-dimensional feature space.

[0240] As an improvement, the tracking module includes:

[0241] The tracker setting unit is used to track each target area using an independent tracker.

[0242] The tracking unit is used by the tracker to determine the position and size of the target in the new frame image by calculating the similarity between the new frame image and the previous frame image, thereby obtaining the target region of the new frame image.

[0243] The masking unit is used to set all pixels within the target area of ​​the tracked new frame image to a specific value for masking processing.

[0244] The boundary processing unit is used to stop tracking and delete the target and its corresponding tracker when the target region of the new frame image moves to the edge of the image.

[0245] The online supervision unit is used to determine whether the target area of ​​the new frame image is the target based on the online supervisor equipped on each tracker. If not, a tracking error is marked.

[0246] The error handling unit is used to delete the target and its corresponding tracker when the number of tracking errors reaches a set threshold.

[0247] In one implementation, the distance calculation module includes:

[0248] The center point calculation unit is used to obtain the bottom center point of the detection box of the target area and calculate the pixel coordinates of the bottom center point in the image pixel coordinate system.

[0249] The coordinate mapping unit is used to map the pixel coordinates of the bottom center point to the actual coordinates in the vehicle coordinate system according to the mapping model.

[0250] The distance calculation unit is used to calculate the distance from the target to the unmanned vehicle based on the coordinates of the unmanned vehicle in the body coordinate system and the actual coordinates of the bottom center point in the vehicle body coordinate system.

[0251] One implementation of the relative collision time calculation module includes:

[0252] The detection box width calculation unit is used to calculate the pixel width of the detection box of the target region in the image at time t and time t-1. and .

[0253] The relative collision time calculation unit is used when Greater than At that time, the relative collision time (TTC) between the target and the unmanned vehicle is calculated.

[0254]

[0255] in, Let t be the time interval between time t and time t-1.

[0256] The apparatus provided in this embodiment of the invention has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the apparatus embodiment can be referred to the corresponding content in the aforementioned method embodiment 1. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the apparatus and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0257] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the scope of the technology disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention.

Claims

1. A method for forward collision warning of a vehicle, characterized in that, The method includes: S1: Based on the monocular camera installed at the designated position of the unmanned vehicle and the calibration object placed in front of the unmanned vehicle, establish a mapping model from the image pixel coordinate system to the vehicle coordinate system. S2: Real-time acquisition of the video stream in front of the unmanned vehicle captured by the monocular camera, and extraction of the region of interest of the image contained in the video stream; S3: Extract the Haar-like features of the region of interest, and based on the Haar-like features, use the trained AdaBoost classifier to extract candidate regions; S4: Extract the Hog features of the candidate region, and based on the Hog features, use the trained support vector machine classifier to extract the target region representing the target in front of the unmanned vehicle. S5: Use a tracker to track the target area to obtain a series of target areas corresponding to the target in the time series; S6: Based on the position of the target area on the image and the established mapping model, calculate the distance from the target corresponding to the target area to the unmanned vehicle; S7: Calculate the relative collision time between the target and the unmanned vehicle based on the time-varying size changes of the detection boxes of a series of target areas corresponding to the target; S8: Provide a forward collision warning based on the distance and the relative collision time.

2. The vehicle forward collision warning method according to claim 1, characterized in that, S1 includes: S11: Install a monocular camera in the rearview mirror position of the unmanned vehicle and place a calibration object in front of the unmanned vehicle; S12: Perform distortion correction on the installed monocular camera and make the image center of the monocular camera aligned with the vanishing point of the horizon in front. S13: Find the pixel coordinates of the four non-collinear selected points of the calibration object in the image pixel coordinate system on the image captured by the monocular camera, and record the actual coordinates of the selected points in the vehicle coordinate system. S14: Based on the pixel coordinates and actual coordinates of the selected point, and according to the geometric relationship of camera imaging, establish a mapping model from the image pixel coordinate system to the vehicle coordinate system.

3. The vehicle forward collision warning method according to claim 1, characterized in that, S2 includes: S21: Obtain the current image from the video stream in front of the unmanned vehicle captured by the monocular camera; S22: Determine the pixel coordinates of the vanishing horizon point on the current image, and extract the region of interest based on the pixel coordinates of the vanishing horizon point.

4. The vehicle forward collision warning method according to claim 1, characterized in that, S3 includes: S31: Calculate the integral map of the region of interest, and calculate the Haar_like feature based on the calculated integral map; The Haar-like features include edge features and linear features; S32: Based on the Haar-like features, the candidate regions are extracted using the trained Adaboost cascade classifier, which consists of multiple strong classifiers connected in series.

5. The vehicle forward collision warning method according to claim 1, characterized in that, S4 includes: S41: Set the block size, cell size, block sliding step size, and gradient interval; S42: Slide the block on the candidate region according to the sliding step size, and divide the region within each block into several cell regions according to the size of the cells; S43: Calculate the gradient direction and magnitude for each cell region within each block, and statistically analyze the gradient direction of each cell region within each block according to the gradient interval to obtain the Hog feature vector of each block; S44: Concatenate the Hog feature vectors of all blocks to obtain the Hog features of the candidate region; S45: Based on the Hog features, the target region is extracted using a trained support vector machine classifier.

6. The vehicle forward collision warning method according to claim 5, characterized in that, The support vector machine classifier is pre-trained through the following process: Construct a training set for training the support vector machine classifier; The training set includes positive samples and negative samples, wherein the positive samples are images containing the target and the negative samples are images not containing the target; Extract the Hog features of the positive and negative samples and add labels; The support vector machine classifier is trained based on the Hog features of labeled positive and negative samples. During training, a set kernel function is used to map the Hog features of positive and negative samples to a high-dimensional feature space.

7. The vehicle forward collision warning method according to claim 1, characterized in that, S5 includes: S51: Use a separate tracker to track each target area; S52: The tracker determines the position and size of the target in the new frame image by calculating the similarity between the new frame image and the previous frame image, and obtains the target region of the new frame image; S53: Set all pixels within the target area of ​​the newly tracked frame image to a specific value and perform masking; S54: When the target region of the new frame image moves to the edge of the image, stop tracking and delete the target and the corresponding tracker; S55: Determine whether the target region of the new frame image is the target based on the online supervisor equipped with each tracker; if not, mark a tracking error. S56: When the number of tracking errors reaches a set threshold, delete the target and its corresponding tracker.

8. The vehicle forward collision warning method according to claim 1, characterized in that, S6 includes: S61: Obtain the bottom center point of the detection box of the target area, and calculate the pixel coordinates of the bottom center point in the image pixel coordinate system; S62: Based on the mapping model, map the pixel coordinates of the bottom center point to the actual coordinates in the vehicle coordinate system; S63: Calculate the distance from the target to the unmanned vehicle based on the coordinates of the unmanned vehicle in the body coordinate system and the actual coordinates of the bottom center point in the vehicle body coordinate system.

9. The vehicle forward collision warning method according to claim 1, characterized in that, S7 includes: S71: Calculate the pixel width of the detection box of the target region in the image at time t and time t-1. and ; S72: When Greater than At that time, the relative collision time TTC between the target and the unmanned vehicle is calculated; in, Let t be the time interval between time t and time t-1.

10. A vehicle forward collision warning device, characterized in that, The device includes: The calibration module is used to establish a mapping model from the image pixel coordinate system to the vehicle coordinate system based on the monocular camera installed at a set position on the unmanned vehicle and the calibration object placed in front of the unmanned vehicle. The region of interest acquisition module is used to acquire the video stream in front of the unmanned vehicle captured by the monocular camera in real time, and to extract the region of interest of the image contained in the video stream. The first-level detection module is used to extract the Haar-like features of the region of interest, and based on the Haar-like features, use the trained AdaBoost classifier to extract candidate regions. The second-level detection module is used to extract the Hog features of the candidate region, and based on the Hog features, to extract the target region representing the target in front of the unmanned vehicle using a trained support vector machine classifier. The tracking module is used to track the target area using a tracker to obtain a series of target areas corresponding to the target in a time series. The distance calculation module is used to calculate the distance from the target corresponding to the target area to the unmanned vehicle based on the position of the target area on the image and the established mapping model; The relative collision time calculation module is used to calculate the relative collision time between the target and the unmanned vehicle based on the time-varying size changes of the detection boxes of a series of target areas corresponding to the target. A forward collision warning module is used to provide forward collision warning based on the distance and the relative collision time.

Images