An infrared pedestrian detection system based on image ROI
By performing image preprocessing and ROI generation in the infrared pedestrian detection system, combined with area feature generation and the use of classifiers, the problem of low pedestrian detection accuracy in complex scenarios is solved, and more efficient pedestrian recognition in infrared images is achieved.
Patent Information
- Application Number
- CN202111515182.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-13
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-12-13
AI Technical Summary
In complex scenes and poor lighting conditions, it is difficult for the prior art to effectively distinguish pedestrians from backgrounds from visible light images, resulting in low pedestrian detection accuracy.
An infrared pedestrian detection system based on image ROI is adopted to reduce noise and background information through the image preprocessing module. The ROI generation module extracts possible pedestrian areas and uses the area feature generation module to generate feature vectors for classification and judgment, so as to improve the accuracy of pedestrian detection.
The detection rate of pedestrians is significantly improved in low-contrast infrared images, and the accuracy and accuracy of pedestrian detection are enhanced.
Smart Images

Figure CN114419727B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to pedestrian detection technology in the field of computer vision, and mainly relates to an infrared pedestrian detection method. Background Art
[0002] Pedestrian detection is a popular and challenging research direction in computer vision. Its purpose is to distinguish pedestrians from the background in images or video frames and locate pedestrians. This technology has great practical value in the fields of assisted driving and intelligent transportation, and can effectively improve the safety of vehicle driving. Pedestrian detection usually uses visible light images as detection objects, but it is difficult to distinguish pedestrians from the background from visible light images under the interference of unfavorable conditions such as complex scenes and poor lighting conditions. Compared with visible light imaging, the brightness of an object in an infrared image depends on the temperature and thermal radiation of the object. Therefore, infrared imaging has a stronger anti-interference ability and can better adapt to various environments. In non-high or low temperature weather, pedestrians usually release more heat than background objects. Therefore, the pixels in the area where pedestrians are located in the infrared image are brighter than the surrounding background objects, making it easier to locate the area where pedestrians exist, thereby improving the detection rate of pedestrians. Summary of the invention
[0003] The object of the present invention is to provide an infrared pedestrian detection system based on image ROI, which is beneficial to improving the detection rate of pedestrians in infrared images with low contrast between pedestrians and backgrounds.
[0004] The system of the present invention includes four modules: preprocessing, ROI generation, regional feature generation and classifier. The original infrared image is input from the image preprocessing module, and the ROI generation module outputs the area that may contain pedestrians. The classifier makes the final judgment on whether the area contains pedestrians.
[0005] The image preprocessing module reduces the noise and background information in the original infrared image through filtering transformation. Then, a suitable threshold is selected to generate the corresponding binary image. In order to improve the efficiency of preprocessing, the present invention uses a simpler mean filter transformation to divide the infrared image into n blocks and calculate the mean of the pixels in each block. By comparing each pixel in the block with the mean of the block, the pixels smaller than the mean are replaced by the mean. This method can effectively reduce the background information and suppress the noise in the infrared image to a certain extent, integrating it into the surrounding pixels.
[0006] The ROI generation module will further process the binary image to eliminate scattered noise points and small connected domains, and preliminarily determine the ROI where pedestrians may exist in the infrared image based on the circumscribed rectangle of the remaining connected domain. Since there may be no pedestrians or multiple pedestrians in the initially extracted ROI, the ROI module will filter the ROI belonging to the background based on the motion target analysis and the width and height of the ROI. For the retained ROI, the grayscale histogram of each column of pixels is calculated based on the brightness distribution within the ROI, and the column with a darker brightness relative to the mean value of the ROI is found and used as the boundary to split the ROI into several sub-ROIs, so as to detect as many single pedestrians as possible from the dense pedestrians.
[0007] For the final ROI set obtained by the above ROI generation module, the regional feature generation module is used to generate feature vectors for each ROI respectively and input them into the subsequent classifier for accurate classification to determine whether the object surrounded by the ROI is a pedestrian.
[0008] As an optional implementation, the filter transformation in the preprocessing module can be replaced by a conventional mean filter or a Gaussian filter;
[0009] As an optional implementation method, the ROI generation module can use common low-pass filtering or binary image opening operation to reduce the noise of the binary image;
[0010] As an optional implementation, the moving target analysis in the ROI module can use the adjacent frame difference method or the three-frame difference method plus the dilation operation;
[0011] As an optional implementation, the region generation module is used to generate LBP (local binary pattern) features or Hog (histogram of oriented gradients) features of the ROI;
[0012] As an optional implementation method, SVM (support vector machine) is used as the final classifier for judging pedestrians and non-pedestrians;
[0013] As an optional implementation method, deep learning technology is used to build a lightweight deep neural network as a classifier.
[0014] Compared with the prior art, the present invention has the following beneficial effects: the preprocessing module is used to reduce the noise and background information in the infrared image and generate the corresponding binary image, and then the ROI generation module extracts the ROI from the preprocessed image based on the binary image and adjusts it to obtain more accurate pedestrian positioning. In addition, the regional feature generation module further generates the corresponding feature vector for the adjusted ROI and transmits it to the final classifier to accurately judge whether the object surrounded by the region is a pedestrian, further improving the accuracy of pedestrian detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 It is a framework diagram of the infrared pedestrian detection system of the present invention;
[0016] Figure 2 It is the flowchart of the image preprocessing module;
[0017] Figure 3 It is a flowchart of the work of ROI generation module;
[0018] Figure 4 It is the principle diagram of the LBP feature calculation method;
[0019] Figure 5 It is the principle diagram of SVM classifier. DETAILED DESCRIPTION
[0020] The following is a further description of an implementation of the present invention in conjunction with the accompanying drawings.
[0021] This invention proposes an infrared pedestrian detection system based on image ROI. The system framework is as follows: Figure 1 As shown in the figure, it consists of four modules connected in series: image preprocessing, ROI generation, regional feature generation and classification.
[0022] The workflow of the image preprocessing module is as follows: Figure 3 As shown, the specific steps are as follows:
[0023] 1) For the original infrared image input to the system, the image preprocessing module divides it into multiple blocks of size S*S, and calculates the mean of the pixels in each block. There is no overlap between blocks, and no padding is performed for blocks with image edge size less than SxS. Then the pixels in each block are compared with the corresponding mean one by one, and the pixels smaller than the mean are replaced with the mean, and the pixels larger than the mean are retained. The specific mathematical expression is as follows:
[0024]
[0025] I in the above formula k (x,y) and I' k (x, y) represents the pixel value of the point (x, y) in the kth block of the original image and the pixel value after transformation. Represents the mean value of pixels in the kth block in the original image.
[0026] 2) Calculate the mean and standard deviation of the infrared image after filtering and transform, and determine the threshold value based on the two to generate a binary image corresponding to the infrared image. The specific operation is shown in the following expression:
[0027]
[0028] In the above formula, I(x,y) and b(x,y) represent the pixels of the infrared image and the corresponding binary image at the (x,y) position, respectively. and σ represent the pixel mean and standard deviation of the infrared image respectively.
[0029] The workflow of the ROI generation module is as follows: Figure 4 As shown, the specific steps are as follows:
[0030] 1) Based on the infrared image and binary image obtained by the preprocessing module, the ROI generation module performs morphological processing on the binary image and eliminates scattered bright spots in the binary image through binary image opening operations. Then the areas of all connected domains in the image are calculated, and the connected domains with smaller areas are filtered out by selecting a suitable area threshold. For the retained connected domains, the corresponding external rectangular frame is generated to preliminarily generate the ROI where pedestrians may exist.
[0031] 2) Calculate the difference between the current infrared image and the previous infrared image. The time interval between two adjacent frames is generally set between 0.3s and 0.5s to ensure that the pedestrian has a significant position change in the two adjacent frames. For positions where the difference between adjacent frames is greater than 20% of the maximum pixel value of the current image, it can be considered that the image information at that position has changed significantly. Count the proportion of pixels with significant changes in each connected domain in the corresponding connected domain, and retain the connected domains with the above proportion greater than 15%.
[0032] 3) Calculate the histogram of each retained ROI column by column and compare it with the brightness mean of the current ROI, and count the percentage of pixels in each column whose brightness is less than the mean. Since there may be dense pedestrians in a large connected domain, the brightness of the columns where the gaps between pedestrians are located is relatively low. Excluding the columns close to the boundaries on both sides of the ROI, the columns in the middle where 60% of the pixels have brightness less than the mean are considered to be gaps between pedestrians. The current ROI is split into multiple sub-ROIs that surround a single pedestrian through these low-brightness columns.
[0033] 4) Calculate the aspect ratio of all ROIs, filter out ROIs with similar width and height, and only retain ROIs with a width-to-height ratio no greater than 0.7.
[0034] For all the ROIs obtained above that may contain a single pedestrian, these ROIs need to be scaled to a uniform size and then input into the regional feature generation module to extract features from these ROI regions for subsequent classification. Here, the LBP feature of the ROI is selected for extraction. The LBP feature is a commonly used feature in infrared image target detection, with significant advantages such as grayscale invariance and rotation invariance, and has a strong expressiveness for the local texture features of the image. Figure 4This is the principle diagram of the LBP feature calculation method. The definition of the LBP operator is that in any 3*3 window in the grayscale image, the pixel value at the center of the window is used as the threshold and compared with the grayscale values of the remaining 8 pixels in the window. If the comparison result is greater than or equal to the threshold, the position of the pixel is set to 1, otherwise it is set to 0. As shown in the figure, the 8 points in the 3*3 neighborhood can generate an 8-bit binary number after comparison. The 8-bit binary number is sorted clockwise from the upper left corner to form the LBP binary sequence of the center pixel of the window, and the corresponding decimal number is the LBP value of the center pixel of the window. The corresponding LBP feature can be obtained by calculating the LBP values of all pixels in the image.
[0035] Finally, the LBP features corresponding to each ROI are input into the classifier to determine whether the object in the area is a pedestrian. Here, SVM is used as a classifier for judging pedestrians. SVM is a linear classifier for binary classification. Its working principle is shown in Figure 5 In essence, SVM is a hyperplane in a feature space that divides feature vectors in the space into two categories based on spatial distance, and the feature vectors at the boundary position of the two categories are farthest away. The mathematical expression of this hyperplane is as follows:
[0036]
[0037] w=(w 1 ,w 2 ,w 3 ,...,w p )
[0038] x i =(x i1 ,x i2 ,x i3 ,...,x ip ),i=1,2,3,...,n
[0039] y i = ±1, i = 1, 2, 3, ..., n
[0040] In the above formula, x i and i represents the features and actual category of the i-th object, and w and b are unknown parameters of the hyperplane. From the above formula, it can be obtained that for the extracted LBP feature x, if w·x+b≥1, it can be considered that the ROI corresponding to the feature contains pedestrians; if w·x+b≤-1, the corresponding ROI does not contain pedestrians. The above shows and describes the basic principles, main features and advantages of the present invention.
[0041] Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements fall within the scope of the present invention to be protected. The scope of the present invention to be protected is defined by the attached claims and their equivalents.
Claims
1. An infrared pedestrian detection system based on image ROI, characterized in that: It includes four modules: image preprocessing module, ROI generation module, regional feature generation module and classifier; The image preprocessing module reduces the noise and background information in the input original infrared image through filtering transformation, and then selects a suitable threshold to generate a corresponding binary image; The ROI generation module further processes the binary image to eliminate scattered noise points and connected domains with smaller areas, and preliminarily determines the ROIs where pedestrians may exist in the infrared image based on the circumscribed rectangles of the remaining connected domains; since there may be no pedestrians or multiple pedestrians in the initially extracted ROI, the ROI module subsequently filters the ROIs belonging to the background based on the motion target analysis and the width and height of the ROI, calculates the histogram column by column for each retained ROI and compares it with the brightness mean of the current ROI, and counts the percentage of pixels with brightness less than the mean in each column. Since there may be dense pedestrians in a connected domain with a larger area, the brightness of the columns where the gaps between pedestrians are located is relatively low, and the columns close to the boundaries on both sides of the ROI are removed. The columns with 60% pixel brightness less than the mean in the middle part are considered to be gaps between pedestrians. The current ROI is split into multiple sub-ROIs surrounding a single pedestrian through these low-brightness columns, and the aspect ratios of all ROIs are calculated, and the ROIs with similar widths and heights are filtered out, and only the ROIs with a width-to-height ratio not greater than 0.7 are retained; The regional feature generation module generates a feature vector for each ROI and inputs it into a subsequent classifier; The classifier performs accurate classification to determine whether the object surrounded by the ROI is a pedestrian.
2. The infrared pedestrian detection system based on image ROI according to claim 1, characterized in that: The filter transformation in the preprocessing module can be replaced by conventional mean filtering or Gaussian filtering.
3. The infrared pedestrian detection system based on image ROI according to claim 1, characterized in that: The ROI generation module may perform noise reduction processing on the binary image by using a common low-pass filter or a binary image opening operation.
4. The infrared pedestrian detection system based on image ROI according to claim 1, characterized in that: The moving target analysis in the ROI generation module may use an adjacent frame difference method or a three-frame difference method plus a dilation operation.
5. The infrared pedestrian detection system based on image ROI according to claim 1, characterized in that: The regional feature generation module is used to generate local binary pattern features or directional gradient histogram features of the ROI.
6. The infrared pedestrian detection system based on image ROI according to claim 1, characterized in that: The classifier support vector machine is used as the final classifier for determining pedestrians and non-pedestrians.
Citation Information
Patent Citations
Method for detecting head and shoulders of pedestrian in video based on overhead pixel gradient direction
CN103310194A
Low-altitude aerial infrared image based pedestrian detection method
CN105574488A
Infrared pedestrian target detection method based on image fusion
CN113688722A
Method for detecting pedestrians based on far infrared ray camera at night
US20140334672A1
