A method for recognizing car collision warning icons
By using real-time shooting and feature matching algorithms, the system can quickly identify car collision warning icons, solving the problem of low recognition efficiency in existing technologies, improving testing efficiency and reducing human resource consumption.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HENAN UNIV OF SCI & TECH
- Filing Date
- 2023-07-04
- Publication Date
- 2026-05-05
AI Technical Summary
In existing technologies, the recognition efficiency of car collision warning icons is low, and manual recognition methods are inefficient and cannot accurately obtain the icon position, resulting in low testing efficiency and poor practicality.
By capturing real-time images of obstacle targets using vehicle-mounted cameras, combined with GPS positioning, calculating color histogram similarity, performing inter-frame similarity convolution smoothing, extracting SIFT feature points, matching feature points using FLANN and RANSAC algorithms, performing perspective transformation, and slicing to obtain warning icon areas, rapid recognition is achieved.
It enables fast and effective recognition of car collision warning icons, improving testing efficiency and reducing human resource costs.
Smart Images
Figure REF-OBJ-1774320081855-000001 
Figure REF-OBJ-1774320081855-000002 
Figure REF-OBJ-1774320081855-000003
Abstract
Description
Technical Field
[0001] This invention relates to the field of vehicle collision warning, and more particularly to a method for recognizing vehicle collision warning icons. Background Technology
[0002] Currently, automakers and parts suppliers worldwide are vigorously developing and commercializing forward collision warning systems, which are gradually entering the market. Correspondingly, the International Organization for Standardization (ISO) has published ISO 15623:2013, "Performance requirements and test procedures for forward collision warning systems in intelligent transportation systems." Based on this standard and the national standard GB / T 20608-2006, "Performance requirements and test methods for adaptive cruise control systems in intelligent transportation systems," my country has published the national standard GB / T 33577-2017, "Performance requirements and test procedures for forward collision warning systems in intelligent transportation systems." GB / T 33577-2017 specifies the system functions, requirements, and performance test procedures for collision warning systems. Performance testing includes testing the detection area, alarm distance and accuracy, and target recognition capability. Each test is further subdivided into multiple tests based on its performance requirements, and repeated experiments are required.
[0003] Current technology typically involves manually searching dashboard videos recorded by cameras during testing to pinpoint the time when the warning icon appears. This data is then combined with GPS and speed information of both vehicles recorded by other instruments at that moment for analysis. However, given that such tests require repeated experiments, manual identification is highly inefficient. Furthermore, it is generally impossible to accurately obtain the specific image and location of the car collision warning icon before testing, necessitating the identification of an unknown icon. If the warning icon were obtained in advance and trained for real-time detection, it would require significant time for image training and machine learning. Moreover, the appearance and location of car collision warning icons vary between different car manufacturers, resulting in limited practicality. Summary of the Invention
[0004] The purpose of this invention is to provide a method for recognizing car collision warning icons, which can effectively improve the efficiency of car collision system testing and reduce the consumption of human resources.
[0005] The technical solution adopted by the present invention to solve the above-mentioned technical problems is: a method for recognizing car collision warning icons, comprising the following steps:
[0006] Step 1: The vehicle-mounted camera takes real-time pictures of obstacles on the road ahead of the vehicle and uses GPS to determine the real-time distance between the vehicle and the obstacles. When the real-time distance is within a preset range, the video captured by the vehicle's dashboard is read frame by frame. A timestamp is added to each frame of the read image to obtain a set of images to be identified, sorted by timestamp. Color histograms are plotted for all images in the set of images to be identified to obtain a set of color histograms sorted by timestamp.
[0007] Step 2: Read the color histogram set, and calculate the color histogram similarity between the color histogram of each timestamp and the color histogram of the next adjacent timestamp in turn, to obtain the color histogram similarity sequence sorted by timestamp.
[0008] The formula for calculating the similarity of color histograms is:
[0009] ;
[0010] In the above formula, The i-th point of the color histogram curve for the first timestamp. The i-th point of the color histogram curve for the next timestamp;
[0011] Step 3: Perform convolution smoothing on all color histogram similarities in the color histogram similarity sequence using the Hanning window to obtain an inter-frame similarity sequence sorted by timestamp. Then, compare each inter-frame similarity in the sequence with its adjacent previous and next inter-frame similarities. When an inter-frame similarity is less than the similarities of two adjacent inter-frames in the sequence, select the image in the image set to be identified corresponding to that inter-frame similarity as a keyframe image to obtain a keyframe image set.
[0012] Step 4: Select a reference image containing the warning icon from the last ten frames in the keyframe image set according to the timestamp order, cut out the warning icon from the reference image, and perform SIFT feature extraction on the reference image and the warning icon respectively to obtain the feature point group of the reference image and the feature point group of the warning icon respectively.
[0013] Step 5: Match the feature point group of the reference image and the feature point group of the warning icon using the FLANN matching algorithm to obtain the nearest neighbor matching distance and the second nearest neighbor matching distance for each feature point. When the ratio of the nearest neighbor matching distance to the second nearest neighbor matching distance is less than the threshold, select the feature point and the feature point corresponding to its nearest neighbor matching distance as a matching point pair. Then, use the RANSAC random sampling algorithm to remove erroneous matching point pairs and retain no less than four valid matching point pairs. Then, perform homography transformation on the valid matching point pairs to obtain the homography matrix.
[0014] Step 6: Perform a perspective transformation on the warning icon and the homography matrix to obtain the coordinates of the rectangular border of the warning icon area;
[0015] Step 7: Based on the coordinates of the rectangular border of the warning icon area, slice the keyframe images other than the base image to obtain the rectangular area of the warning icon in the keyframe image, and perform SIFT feature extraction on the rectangular area of the warning icon to obtain the feature point groups of all keyframe images other than the base image.
[0016] Step 8: Use the FLANN matching algorithm to match the feature point group of the warning icon with the feature point group of multiple key frame images respectively, and obtain the nearest neighbor matching distance and the second nearest neighbor matching distance for each feature point. When the ratio of the nearest neighbor matching distance to the second nearest neighbor matching distance is less than the threshold, select the feature point and the feature point corresponding to its nearest neighbor matching distance as a matching point pair, and obtain the number of effective matching point pairs corresponding to multiple key frame images respectively.
[0017] Step 9: Compare the number of valid matching point pairs obtained in Step 8 with the preset baseline number. When the number of valid matching point pairs is greater than the preset baseline number, select the keyframe image corresponding to the number of valid matching point pairs as the recognized image containing the warning icon.
[0018] Preferably, the homography matrix is a 3x3 matrix, represented as follows:
[0019] .
[0020] According to the above technical solution, the beneficial effects of the present invention are:
[0021] This invention first calculates the inter-frame similarity sequence using a color histogram to obtain keyframe images. Then, it extracts and matches features from a baseline image, and after perspective transformation, obtains the rectangular border coordinates of the warning icon area. Based on these coordinates, the remaining keyframe images are sliced. By extracting and matching features from the sliced areas of each keyframe image, the number of valid matching point pairs for all keyframe images is obtained. Comparing this number with a preset baseline number allows for the selection of recognized images containing the warning icon, thus enabling the recognition of car collision warning icons. The invention also outputs keyframes containing time information, achieving rapid and effective recognition of car collision warning icons, significantly improving the efficiency of car collision system testing and reducing manpower costs. Detailed Implementation
[0022] A method for recognizing car collision warning icons includes the following steps:
[0023] Step 1: The vehicle-mounted camera takes real-time pictures of obstacles on the road ahead of the vehicle, and uses GPS to determine the real-time distance between the vehicle and the obstacles. When the real-time distance is within a preset range, the video captured by the vehicle's dashboard is read frame by frame. A timestamp is added to each frame of the read image to obtain a set of images to be identified, sorted by timestamp. A color histogram is plotted for all images in the set of images to be identified to obtain a set of color histograms sorted by timestamp.
[0024] Step 2: Read the color histogram set, and calculate the color histogram similarity between the color histogram of each timestamp and the color histogram of the next adjacent timestamp in turn, to obtain the color histogram similarity sequence sorted by timestamp.
[0025] The formula for calculating the similarity of color histograms is:
[0026] ;
[0027] In the above formula, The i-th point of the color histogram curve for the first timestamp. The i-th point of the color histogram curve for the next timestamp.
[0028] Step 3: Perform convolution smoothing on all color histogram similarities in the color histogram similarity sequence using the Hanning window to obtain an inter-frame similarity sequence sorted by timestamp. Then, compare each inter-frame similarity in the sequence with its adjacent previous and next inter-frame similarities. When an inter-frame similarity is less than the similarities of two adjacent inter-frames in the sequence, select the image in the image set to be identified corresponding to that inter-frame similarity as a keyframe image to obtain a keyframe image set.
[0029] Step 4: Select a reference image containing the warning icon from the last ten frames in the keyframe image set according to the timestamp order. Cut out the warning icon from the reference image. Perform SIFT feature extraction on the reference image and the warning icon respectively to obtain the feature point group of the reference image and the feature point group of the warning icon.
[0030] Step 5: Match the feature point group of the reference image and the feature point group of the warning icon using the FLANN matching algorithm to obtain the nearest neighbor matching distance and the second nearest neighbor matching distance for each feature point. When the ratio of the nearest neighbor matching distance to the second nearest neighbor matching distance is less than a threshold, select the feature point and the feature point corresponding to its nearest neighbor matching distance as a matching point pair. Then, use the RANSAC random sampling algorithm to remove erroneous matching point pairs, retaining no less than four valid matching point pairs. Then, perform homography transformation on the valid matching point pairs to obtain the homography matrix. The homography matrix is a 3x3 matrix, represented as:
[0031] .
[0032] Step 6: Perform a perspective transformation on the warning icon and the homography matrix to obtain the coordinates of the rectangular border of the warning icon area.
[0033] Step 7: Based on the coordinates of the rectangular border of the warning icon area, slice the keyframe images other than the base image to obtain the rectangular area of the warning icon in the keyframe image, and perform SIFT feature extraction on the rectangular area of the warning icon to obtain the feature point groups of all keyframe images other than the base image.
[0034] Step 8: Use the FLANN matching algorithm to match the feature point group of the warning icon with the feature point group of multiple keyframe images respectively, and obtain the nearest neighbor matching distance and the second nearest neighbor matching distance for each feature point. When the ratio of the nearest neighbor matching distance to the second nearest neighbor matching distance is less than the threshold, select the feature point and the feature point corresponding to its nearest neighbor matching distance as a matching point pair, and obtain the number of effective matching point pairs corresponding to multiple keyframe images respectively.
[0035] Step 9: Compare the number of valid matching point pairs obtained in Step 8 with the preset baseline number. When the number of valid matching point pairs is greater than the preset baseline number, select the keyframe image corresponding to the number of valid matching point pairs as the recognized image containing the warning icon, thus realizing the recognition of the car collision warning icon.
Claims
1. A method for recognizing car collision warning icons, characterized in that, Includes the following steps: Step 1: The vehicle-mounted camera takes real-time pictures of obstacles on the road ahead of the vehicle and uses GPS to determine the real-time distance between the vehicle and the obstacles. When the real-time distance is within a preset range, the video captured by the vehicle's dashboard is read frame by frame. A timestamp is added to each frame of the read image to obtain a set of images to be identified, sorted by timestamp. Color histograms are plotted for all images in the set of images to be identified to obtain a set of color histograms sorted by timestamp. Step 2: Read the color histogram set, and calculate the color histogram similarity between the color histogram of each timestamp and the color histogram of the next adjacent timestamp in turn, to obtain the color histogram similarity sequence sorted by timestamp. The formula for calculating the similarity of color histograms is: ； In the above formula, The i-th point of the color histogram curve for the first timestamp. The i-th point of the color histogram curve for the next timestamp; Step 3: Perform convolution smoothing on all color histogram similarities in the color histogram similarity sequence using the Hanning window to obtain an inter-frame similarity sequence sorted by timestamp. Then, compare each inter-frame similarity in the sequence with its adjacent previous and next inter-frame similarities. When an inter-frame similarity is less than the similarities of two adjacent inter-frames in the sequence, select the image in the image set to be identified corresponding to that inter-frame similarity as a keyframe image to obtain a keyframe image set. Step 4: Select a reference image containing the warning icon from the last ten frames in the keyframe image set according to the timestamp order, cut out the warning icon from the reference image, and perform SIFT feature extraction on the reference image and the warning icon respectively to obtain the feature point group of the reference image and the feature point group of the warning icon respectively. Step 5: Match the feature point group of the reference image and the feature point group of the warning icon using the FLANN matching algorithm to obtain the nearest neighbor matching distance and the second nearest neighbor matching distance for each feature point. When the ratio of the nearest neighbor matching distance to the second nearest neighbor matching distance is less than the threshold, select the feature point and the feature point corresponding to its nearest neighbor matching distance as a matching point pair. Then, use the RANSAC random sampling algorithm to remove erroneous matching point pairs and retain no less than four valid matching point pairs. Then, perform homography transformation on the valid matching point pairs to obtain the homography matrix. Step 6: Perform a perspective transformation on the warning icon and the homography matrix to obtain the coordinates of the rectangular border of the warning icon area; Step 7: Based on the coordinates of the rectangular border of the warning icon area, slice the keyframe images other than the base image to obtain the rectangular area of the warning icon in the keyframe image, and perform SIFT feature extraction on the rectangular area of the warning icon to obtain the feature point groups of all keyframe images other than the base image. Step 8: Use the FLANN matching algorithm to match the feature point group of the warning icon with the feature point group of multiple key frame images respectively, and obtain the nearest neighbor matching distance and the second nearest neighbor matching distance for each feature point. When the ratio of the nearest neighbor matching distance to the second nearest neighbor matching distance is less than the threshold, select the feature point and the feature point corresponding to its nearest neighbor matching distance as a matching point pair, and obtain the number of effective matching point pairs corresponding to multiple key frame images respectively. Step 9: Compare the number of valid matching point pairs obtained in Step 8 with the preset baseline number. When the number of valid matching point pairs is greater than the preset baseline number, select the keyframe image corresponding to the number of valid matching point pairs as the recognized image containing the warning icon.
2. The method for recognizing car collision warning icons according to claim 1, characterized in that: In step five, the homography matrix is a 3x3 matrix, represented as follows: 。