A pedestrian detection method for AGV
Through the interference area determination of binocular parallax map and the dimensionality reduction method of feature fusion, the error detection problem caused by background interference and small pedestrian ratio in pedestrian detection is solved, and the detection efficiency and accuracy are improved.
Patent Information
- Application Number
- CN202111637231.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-29
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-12-29
AI Technical Summary
Existing pedestrian detection methods are susceptible to background interference in complex working environments, resulting in mis-checking. Since pedestrians account for a small proportion, a lot of time is spent on non-peedy areas during the inspection process, affecting the detection speed and accuracy.
The correspondence between objects and the world coordinate system in the binocular parallax map is adopted, and the area to be detected is reduced by the interference area determination method, the area of interest of pedestrians is extracted, the characteristics of HOG and LBP are combined, and the classification is performed using SVM after PCA dimensionality reduction.
It significantly improves the efficiency and accuracy of pedestrian detection, overcomes the misdetection problem caused by long-distance complex background interference, and improves the problem of weak detection effect caused by changing pedestrian postures.
Smart Images

Figure CN114332935B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of pedestrian detection methods, and in particular to a pedestrian detection method applied to AGV. Background Art
[0002] Automated Guided Vehicle (AGV) is a transport vehicle equipped with an automatic guidance device such as electromagnetic or optical, which can travel along a specified guidance path and has safety protection and various transfer functions.
[0003] In a human-machine collaborative environment, in order to avoid collisions with pedestrians, the obstacle avoidance system of the automated guided vehicle (AGV) needs to accurately obtain the position and motion trajectory of pedestrians.
[0004] However, the current pedestrian detection methods still have the following shortcomings:
[0005] 1. In complex working environments, vision-based pedestrian detection methods are susceptible to background interference and may cause false detections;
[0006] 2. Current pedestrian detection methods usually combine the sliding window method and the image size scaling method to detect pedestrians. In the process, a large number of windows that need to be identified by the classifier are generated. However, the proportion of pedestrians in actual images is often small, and most of the detection time is spent on non-pedestrian areas, which seriously affects the speed of pedestrian detection.
[0007] Therefore, the present invention proposes a pedestrian detection method applied to AGV, the purpose of which is to improve the detection efficiency and detection accuracy of the method, and effectively overcome the false detection problem caused by long-distance complex background interference. Summary of the invention
[0008] The purpose of the present invention is to solve the problems raised in the above background technology. The present invention provides a pedestrian detection method applied to AGV.
[0009] The main technical scheme of the present invention is as follows: in the region of interest extraction stage, the coordinate correspondence between the object in the binocular disparity map and the actual object in the world coordinate system is utilized, and an interference region determination method based on the disparity map is used to segment and extract the pedestrian region of interest, thereby reducing the area to be detected, reducing the computational complexity of pedestrian detection, and avoiding erroneous information outside the detection distance range.
[0010] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:
[0011] A pedestrian detection method applied to AGV, comprising the following steps:
[0012] (1) Using a binocular camera to obtain an image of the detection area;
[0013] (2) performing image grayscale conversion, median filtering, and histogram equalization processing on the acquired image in sequence to obtain a processed disparity map;
[0014] (3) Obtain the three-dimensional coordinates of the effective disparity points in the disparity map;
[0015] (4) Determine the ground interference area based on the obtained disparity map and the three-dimensional coordinates of the effective disparity points;
[0016] (5) Set the pedestrian detection range value and calculate the area of interest and background interference area;
[0017] (6) According to the determined interference area data, a binary segmentation method is used to extract the region of interest on the disparity map;
[0018] (7) Extract HOG features in the region of interest in the disparity map;
[0019] (8) Extracting LBP features in the region of interest in the disparity map;
[0020] (9) The extracted HOG features and LBP features are fused in series to obtain the fused features HOG-LBP features of the entire image;
[0021] (10) PCA method is used to perform fusion and dimensionality reduction processing on HOG-LBP features;
[0022] (11) The HOG-LBP features after dimension reduction are identified and classified using the SVM classification method.
[0023] Furthermore, in step (3), the steps of obtaining the three-dimensional coordinates of the effective disparity points are as follows:
[0024] ① Assume that in the physical coordinate system of the image, the principal point o l , o r The coordinates of x ,c y )、(c x ′,c y ′), the point P(X P ,Y P ,Z P ) of the imaging point P l , P r The coordinates of are (x l ,y l )、(x r ,y r ), x l -x r is the parallax d of point P, the baseline distance between the left and right cameras is B, and the focal length is f;
[0025] ②In OpenCV, the default world coordinate system of the binocular vision system is OX w Y w Z w and the left camera coordinate system O l -X c Y c Z c They are coincident. According to the principle of geometry, we can get the formula:
[0026]
[0027] The above formula can be used to calculate the coordinate value X of point P in the world coordinate system. P , Y P , Z P :
[0028]
[0029] Furthermore, in step (4), the determination step is as follows:
[0030] ① Establish the camera coordinate system OX based on the left camera C Y C Z C , point O is the optical center of the left camera, X C The axis points to the right camera direction, Z C The axis is parallel to the ground and points in the forward direction. C The axis is perpendicular to the ground;
[0031] ② Let the distance from the camera optical center O to the ground be H, and the three-dimensional coordinates of a point P in three-dimensional space are (X P ,Y P ,Z P ), so the distance from point P to the ground is:
[0032] h P =HY P ;
[0033] ③Set a height threshold Δh, if h P <Δh, the current point is regarded as ground interference information.
[0034] Furthermore, in step (5), the steps of setting the pedestrian detection range value are as follows:
[0035] ① Assume that the distance between the binocular camera and the object in the environment is D, and set a fixed distance threshold D m ;
[0036] ②When D <D m , objects in the environment are likely to be pedestrian targets, and objects within this range are extracted as regions of interest;
[0037] ③When D>D m , this area is considered as background interference area.
[0038] Furthermore, in step (6), the step of extracting the region of interest is as follows:
[0039] ①Let h P <Δh or D>D m The grayscale value of the pixel is set to 0, and the grayscale values of the remaining pixels are set to 255;
[0040] ②Call OpenCV's findContours function to extract the contour of the foreground suspicious target;
[0041] ③Remove contours with a pixel area less than 3000;
[0042] ④Use the boundingRect function to obtain the minimum rectangular boundary of the contour;
[0043] ⑤Extract the region of interest from the original left image.
[0044] Furthermore, in step (6), the HOG feature extraction step is as follows:
[0045] ① Grayscale and Gamma normalization;
[0046] ② Calculate image gradient;
[0047] ③ Gradient direction histogram weighted statistics;
[0048] ④Block construction and normalization;
[0049] ⑤Calculate the overall eigenvector.
[0050] Furthermore, in step (8), the LBP feature extraction method is as follows:
[0051] Assume that each pixel has a gray value f c As the threshold, if the gray value of the surrounding pixels meets the threshold, the point is set to 1, otherwise it is set to 0. The 8 neighborhoods are coded with 0 / 1 to obtain a binary string of length 8, which is used as the encoding of the pixel, as follows:
[0052]
[0053]
[0054] f c is the gray value of the center pixel;
[0055] f pis the gray value of the domain pixel point, and each sampling point in the domain has different weight coefficients;
[0056] 2 p is the assigned sampling point, and finally a summation operation is performed.
[0057] Further, in the step (10), the fusion dimensionality reduction process is as follows:
[0058] Assume that there are m samples {x (1) , x (2) , …, x (m)}, where each sample has n-dimensional HOG features, expressed as where n = 3780, and the range of i is from 1 to m. Combine these m samples and the n-dimensional features contained in each sample to form an m×n-dimensional sample matrix X, expressed as:
[0059]
[0060] First, preprocess the samples and calculate the mean of each column of X:
[0061]
[0062] Then, subtract the mean u j of the current column from each value in the column to obtain a sample matrix X with zero mean, and then calculate the covariance matrix:
[0063]
[0064] Next, find the eigenvalues and eigenvectors of the covariance matrix C:
[0065] Cθ t = λ t θ t ;
[0066] where λ t is the eigenvalue, and θ t is the eigenvector corresponding to λ t ;
[0067] Finally, arrange λ t in descending order. If it is necessary to reduce the n dimensions to k dimensions (k < n), then take the eigenvectors of the first k eigenvalues and form a projection matrix P n×k by columns, expressed as:
[0068] P n×k = [θ1, θ2, …, θ k
[0069] Therefore, the projection formula of the sample matrix X based on PCA is:
[0070] X′=XP n×k ;
[0071] Among them, X′ is the sample matrix after dimensionality reduction, and each row is the feature after dimensionality reduction.
[0072] The beneficial effects of the present invention are as follows:
[0073] 1. The detection efficiency and accuracy of the method of the present invention are significantly improved, which effectively overcomes the problem of false detection caused by long-distance complex background interference, and also improves the problem of weak detection effect caused by the changeable posture of pedestrians to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS
[0074] Figure 1 is a two-dimensional plane diagram of the binocular vision camera model of the present invention;
[0075] Figure 2 It is a geometric model diagram for determining the ground interference area of the present invention;
[0076] Figure 3 This is a physical picture of the AGV in the embodiment of the present invention;
[0077] Figure 4 It is a flow chart of extracting HOG features of the present invention;
[0078] Figure 5 It is a basic structural diagram of each layer in the HOG feature of the present invention;
[0079] Figure 6 It is a linear separable classification model diagram of the present invention. DETAILED DESCRIPTION
[0080] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0081] Compared with the prior art, the present invention makes full use of the coordinate correspondence between objects in the disparity map and actual objects in the world coordinate system, and studies the pedestrian detection method based on binocular vision. First, the region of interest is found in the binocular disparity map, and the foreground target is distinguished from irrelevant background, ground and other interference areas to reduce the area to be detected. Then, the image block of the region of interest in the original left eye image is scaled, and then the feature optimization method of feature fusion and feature dimensionality reduction is used to obtain features in the detection window. Finally, SVM is combined to obtain the accurate position of the pedestrian.
[0082] The method of the present invention firstly obtains the region of interest by adopting a segmentation method based on interference region removal on the disparity map, thereby reducing the area to be detected and avoiding erroneous information outside the detection distance range;
[0083] Then, the Histogram of Oriented Gradients (HOG) features and Local Binary Pattern (LBP) features are extracted in the region of interest and fused in series;
[0084] Finally, the principal component analysis (PCA) method is used to reduce the dimension of the fused feature vector obtained in the region of interest, avoiding the interference of redundant information in the fused features on the performance of the support vector machine (SVM) and accelerating the classification speed of the SVM.
[0085] The detection efficiency and accuracy of the method of the present invention are significantly improved, which effectively overcomes the problem of false detection caused by long-distance complex background interference, and also improves the problem of weak detection effect caused by the changeable posture of pedestrians to a certain extent.
[0086] In order to better understand the content of the present invention, the present invention is described in detail below.
[0087] like Figure 1-6 As shown, a pedestrian detection method applied to an AGV proposed in one embodiment of the present invention includes the following steps:
[0088] (1) Using a binocular camera to obtain an image of the detection area;
[0089] (2) performing image grayscale conversion, median filtering, and histogram equalization processing on the acquired image in sequence to obtain a processed disparity map;
[0090] In the prior art, the images collected by the binocular camera will inevitably be interfered by various types of noise in the environment. In order not to reduce the accuracy of subsequent stereo matching, it is necessary to perform preprocessing operations on the images, wherein the preprocessing operations include image grayscale, median filtering, and histogram equalization;
[0091] Image grayscale is to convert a three-channel color image into a single-channel grayscale image, which can improve computing efficiency;
[0092] Median filtering is used to eliminate isolated point noise;
[0093] Histogram equalization is to extend the histogram based on the grayscale image, which can enhance the image contrast.
[0094] Hereinafter, the present invention further describes the specific schemes of image grayscale, median filtering and histogram.
[0095] 1) Image grayscale
[0096] The image captured by the binocular camera is an RGB three-channel color image. The combination of red, green, and blue components in different proportions can express all known colors. If the three components are the same, it is a grayscale image.
[0097] On the surface, graying a color image will reduce some color information, but from the distribution characteristics of image color and brightness, the information described by the grayscale image and the color image is consistent. There are generally three methods for graying an image:
[0098] (1) Average value method: The average value of the three components is the gray value, as shown in the formula:
[0099] Gray=(R+B+G) / 3;
[0100] (2) Maximum value method: The maximum value of the three components is the gray value, as shown in the formula:
[0101] Gray = max(R+B+G);
[0102] (3) Weighted average method: According to the importance level, different weights are assigned to the three components of R, G, and B and then added together. The gray value formula is as follows:
[0103] Gray=0.299×R+0.578×G+0.114×B;
[0104] By comparing the above three methods, it can be seen that it is more reasonable to use the weighted average method for grayscale conversion. The grayscale image in the present invention is obtained by using the weighted average method.
[0105] 2) Median filtering
[0106] There is a certain amount of noise information in the images collected by the binocular camera. In order to obtain a high-quality disparity map, the binocular image needs to be filtered to remove the noise;
[0107] Median filtering is very effective in suppressing isolated point noise. It replaces the value of the center point of the window with the median of the grayscale value in the window area, filtering out isolated point noise from the perspective of statistical sorting.
[0108] The median filtering method is as follows: first create a filtering template, which is usually a 3*3 or 5*5 square area, then slide the template on the image, sort the odd grayscale values in the template, and finally assign the median value to the center point.
[0109] 3) Histogram equalization
[0110] Usually, after the image is grayed, the contrast is low and the detail features are not prominent. The histogram equalization technology can transform the original histogram into a nearly uniform distribution, thereby enhancing the image brightness contrast.
[0111] After histogram equalization, the grayscale contrast between the foreground target and the background area can be enhanced, making the foreground target more prominent;
[0112] For a given image, let r be the grayscale value of the original image, s be the grayscale value after histogram equalization, normalize r so that 0≤r≤1, then the conversion relationship between r and s is:
[0113] s = T(r);
[0114] This conversion function satisfies the following two conditions:
[0115] ① Monotonically increasing within the domain, that is, after histogram equalization, the brightness and darkness cannot be reversed;
[0116] ②The value range is [0,1] and cannot be exceeded.
[0117] (1) Considering that the presence of ground shadows in the disparity map is not conducive to the segmentation of the region of interest, the foundation is laid for the subsequent segmentation of the region of interest. In this step, the ground interference area is determined by obtaining the three-dimensional coordinates of the effective disparity points in the disparity map. The steps for obtaining the three-dimensional coordinates of the effective disparity points are as follows:
[0118] ① Such as Figure 1 As shown, suppose that in the physical coordinate system of the image, the principal point o l , o r The coordinates of x ,c y )、(c x ′,c y ′), the point P(X P ,Y P ,Z P ) of the imaging point P l , P r The coordinates of are (x l ,y l )、(x r ,y r ), x l -x r is the parallax d of point P, the baseline distance between the left and right cameras is B, and the focal length is f;
[0119] ②In OpenCV, the default world coordinate system of the binocular vision system is OX w Y w Z w and the left camera coordinate system O l -X c Y c Z c They are coincident. According to the principle of geometry, we can get the formula:
[0120]
[0121] The above formula can be used to calculate the coordinate value X of point P in the world coordinate system. P , YP , Z P :
[0122]
[0123] Therefore, the position of the corresponding point in the three-dimensional scene in the world coordinate system can be calculated based on the two-dimensional coordinates of the pixel points in the disparity map. In OpenCV, the reprojectImageTo3D function can extract the coordinates of each point in the disparity map in the world coordinate system through the binocular camera parameters.
[0124] (2) Determine the ground interference area based on the obtained disparity map and the three-dimensional coordinates of the effective disparity points, wherein the specific determination steps are as follows:
[0125] ① Such as Figure 2 As shown, the camera coordinate system OX is established based on the left camera C Y C Z C , point O is the optical center of the left camera, X C The axis points to the right camera direction, Z C The axis is parallel to the ground and points in the forward direction. C The axis is perpendicular to the ground;
[0126] ② Let the distance from the camera optical center O to the ground be H, and the three-dimensional coordinates of a point P in three-dimensional space are (X P ,Y P ,Z P ), so the distance from point P to the ground is:
[0127] h P =HY P ;
[0128] ③Set a height threshold Δh, if h P <Δh, the current point is regarded as ground interference information.
[0129] (3) Considering that pedestrian detection and tracking technology is to provide an accurate data analysis basis for the AGV's obstacle avoidance system, not all pedestrians within the AGV's field of view require the AGV to take emergency measures to avoid obstacles. Usually, the AGV has a slow travel speed and a relatively fixed working path. If all pedestrians within the AGV's field of view are detected and tracked just to prevent collisions, it will cause great efficiency reduction and resource waste. Therefore, it is necessary to set a pedestrian detection range value. The present invention takes the AGV in the prior art as an example. Figure 3 As shown, its characteristics are as follows: height 2.2m, travel speed 0.6m / s, braking distance less than 1m;
[0130] Assume that the distance between the binocular camera and the object in the environment is D, and set a fixed distance threshold Dm , when D <D m When D>D m When , this area is regarded as the background interference area; in order to select a suitable threshold D m At the same time, in order to verify the accuracy of using a binocular camera to obtain the depth information of objects in the scene, a binocular ranging experiment is performed to obtain the effective working range of the camera. In the experiment, a smooth cardboard is used as a reference, and a binocular camera is used to shoot it every 1000mm in the range of 2000mm to 6000mm. The collected image pairs are preprocessed and stereo corrected, and then the disparity map is obtained using the semi-global stereo matching method. Finally, the reproject ImageTo3D() function is used to obtain the image depth information, and the mouse click event is used to obtain the depth value of the sampling point on the plane of the cardboard facing the camera. This depth value is compared with the actual distance, and the error obtained is shown in Table 1:
[0131] Table 1 Depth error at different distances
[0132]
[0133] As shown in Table 1, the actual distance and the measurement error are roughly positively correlated. As the distance increases, the error becomes larger and larger. When the actual distance is less than 5000mm, the relative error does not exceed 5%, which can basically ensure the reliability of the binocular camera detection distance. When the actual distance reaches 6000mm, the measured distance has reference value, but the error is too large. Therefore, combined with the AGV's own conditions and the accuracy of binocular ranging, the maximum detection distance, that is, the distance threshold D m It is 5500mm.
[0134] (6) Based on the determined interference area data, the region of interest is extracted on the disparity map using a binary segmentation method. Since the grayscale image has 256 grayscale levels, where 0 is black and 255 is white, h P <Δh or D>D m The grayscale value of the pixel is set to 0, and the grayscale values of the remaining pixels are set to 255.
[0135] Next, call OpenCV's findContours function to extract the contours of the foreground suspicious target. Considering that matching errors may occur during the stereo matching process, resulting in some noise areas after binary segmentation, contours with a pixel area less than 3000 are removed.
[0136] In order to make the subsequent pedestrian detection process proceed smoothly, the foreground area contour is further processed, and the minimum rectangular boundary of the contour is obtained using the boundingRect function. Finally, the region of interest is extracted from the original left eye image.
[0137] (7) Extract the HOG features in the region of interest in the disparity map. The HOG feature extraction process is as follows: Figure 4 As shown, the extraction steps are as follows:
[0138] ① Grayscale and Gamma Normalization
[0139] The HOG feature converts a three-channel color image into a feature vector of a certain length. Considering that the HOG feature is independent of color information, the original image is converted into a grayscale image, and then the entire image is normalized using Gamma correction to reduce the impact of illumination changes, thereby enhancing the robustness and stability of the method to different illumination problems.
[0140] The formula for gamma correction is:
[0141] I(x,y)=I(x,y) gamma
[0142] Here gamma is taken
[0143] ②Calculate image gradient
[0144] For a digital image, the first-order differential is the difference between adjacent pixels. The gradient of each pixel (x, y) in the image is defined as:
[0145] T x (x,y)=I(x+1,y)-I(x-1,y);
[0146] T y (x,y)=I(x,y+1)-I(x,y-1);
[0147] Among them, I(x,y) is the gray value of point (x,y), T x (x,y) and T y (x, y) represents the horizontal gradient and vertical gradient of point (x, y). Therefore, the gradient magnitude T(x, y) and gradient direction α(x, y) of point (x, y) are expressed as:
[0148]
[0149] ③ Gradient direction histogram weighted statistics
[0150] Based on the second step, the gradient magnitude and direction of each pixel in the image are known, and the next step is to perform gradient statistics on each area in the image. This step can maintain weak sensitivity to slight deformation of pedestrians to a certain extent.
[0151] The specific method is: divide the entire image into several 8×8 pixel units. In each unit, all gradient directions are divided into 9 levels. Here, unsigned gradient is used, that is, the angle is assumed to be the same as , then the 9 levels are the 9 bins of the gradient histogram, corresponding to the angles of 0°, 20°, 40°, ..., 160°, respectively. Use bilinear interpolation to determine the bin where each pixel is located, then each unit can get a histogram of length 9.
[0152] ④Block construction and normalization
[0153] After obtaining the 9-bin histogram of each unit, that is, the 9-dimensional feature vector, the image is further divided, and each 2×2 unit is combined into a block. Therefore, the feature vector in each block is 2×2×9=36 dimensions.
[0154] Since the image gradient is sensitive to changes in illumination, and the feature contrast between the foreground (i.e. pedestrians) and the background is large, the combined effects of the two will make the gradient change range in the local area particularly large. Therefore, the gradient needs to be compressed, that is, the gradient value is normalized. The L2 norm is used to normalize the 36-dimensional feature v in the block. The specific formula is:
[0155]
[0156] Among them, ε is a constant and takes a minimum value to prevent the denominator from being 0.
[0157] ⑤Calculate the overall eigenvector
[0158] When calculating the gradient histogram, the steps are carried out from unit to block, and then from block to the whole. In the detection window, the block is slid from left to right and from top to bottom with a step length of 8 pixels until the entire image is traversed. The structural diagram of this process is shown in the figure. Figure 5 shown.
[0159] (8) Extract the LBP features in the region of interest in the disparity map. In the region of interest, each pixel is divided into a 3*3 area. The grayscale values of the eight adjacent pixels are compared with the center pixel of the image as the threshold. The central grayscale value of the pixel is compared with the central pixel. If the surrounding pixel value is greater than the central pixel value, the position of the pixel is marked as 1, otherwise it is marked as 0.
[0160] In this way, in a 3×3 area, 8 binary numbers can be generated after comparison. In general, they are converted into decimal numbers, called LBP codes, which represent the LBP value of the central pixel of the area, thus reflecting the texture information of the area.
[0161] Therefore, the LBP feature extraction method is as follows:
[0162] Assume that each pixel has a gray value f c As the threshold, if the gray value of the surrounding pixels meets the threshold, the point is set to 1, otherwise it is set to 0. The 8 neighborhoods are coded with 0 / 1 to obtain a binary string of length 8, which is used as the encoding of the pixel, as follows:
[0163]
[0164] f c is the gray value of the center pixel;
[0165] f p is the gray value of the pixel in the domain, and each sampling point in the domain has different weight coefficients;
[0166] 2 p To assign sampling points, the final summation operation is performed.
[0167] (9) The extracted HOG features and LBP features are fused in series to obtain the fused features HOG-LBP features of the entire image. The fusion operation is as follows: Assume that the image is divided into a rows and b columns, G(x, y) represents the yth region in the xth row, and the range of x and y are both between [0, r). For each G(x, y) region, the HOG and LBP operators are used to obtain the corresponding values of the position of each region, and then the two are fused to obtain the fused features of each small area. There are a×b small areas in the whole image. After connecting them, the fused features of the entire image are obtained;
[0168] (10) Considering that the HOG-LBP feature dimension is too high and there is a lot of redundant information, which will lead to a long classification and recognition time for pedestrian detection and will also reduce the recognition rate, which cannot meet the actual application requirements. Therefore, the PCA method is used to fuse and reduce the dimensionality of the HOG-LBP features;
[0169] PCA is a classic unsupervised dimensionality reduction method that projects high-dimensional data into a low-dimensional space with less correlation.
[0170] The principle of this method is: by using spatial transformation, the feature data in the original coordinate space is projected into a low-dimensional coordinate space where the basis vectors are mutually orthogonal, thereby generating a new set of data. Its essence is vector basis change.
[0171] The original feature data itself is closely related to the new coordinate space. Considering that when reducing a set of n-dimensional feature vectors to k dimensions (k < n), it is necessary to retain the original feature information as much as possible. Therefore, it is required that the projected values after projection are dispersed to the greatest extent. The degree of dispersion of data is generally represented by variance. For the problem of reducing two dimensions to one dimension, the direction with the largest variance of the feature elements after projection is used as the first coordinate axis of the new coordinate space. For the dimensionality reduction problem of three dimensions and above, in the first step, the direction with the largest variance of the feature elements after projection is still used as the first coordinate axis. In order to enable each feature element after projection to contain more original information, it should be ensured that each feature element has no correlation, because correlation means information duplication. From a mathematical perspective, the correlation between data is represented by covariance. Therefore, the second coordinate axis needs to be selected in a direction orthogonal to the first coordinate axis. Since there is a preprocessing operation of zero-mean normalization in the initial step of PCA, the orthogonality of the coordinate axes means that the covariance is zero, that is, there is no correlation between the two feature elements. Similarly, the third coordinate axis is the direction orthogonal to the first and second coordinate axes and with the largest variance. Therefore, reducing a set of n-dimensional vectors to k dimensions means selecting k unit orthogonal bases. After projecting the original data onto this set of bases, the covariance between each feature element is 0, and the variance takes the relative maximum value.
[0172] Therefore, in the present invention, the steps of the fusion dimensionality reduction process are as follows:
[0173] Suppose there are m samples {x (1) , x (2) , …, x (m)}, where each sample has n-dimensional HOG features, expressed as where n = 3780, and the range of i is from 1 to m. These m samples and the n-dimensional features contained in each sample are combined to form an m×n-dimensional sample matrix X, expressed as:
[0174]
[0175] First, preprocess the samples and calculate the mean of each column of X:
[0176]
[0177] Then, subtract the mean u j of the current column from the value of each column to obtain a sample matrix X with zero mean. Then calculate the covariance matrix:
[0178]
[0179] Next, find the eigenvalues and eigenvectors of the covariance matrix C:
[0180] Cθ t = λ t θt ;
[0181] Among them, λ t is the eigenvalue, and θ t is the eigenvector corresponding to λ t ;
[0182] Finally, arrange λ t in descending order. If it is necessary to reduce the dimension from n to k (k < n), then take the eigenvectors of the first k eigenvalues to form the projection matrix P n×k , which is expressed as:
[0183] P n×k = [θ1, θ2, …, θ k ;
[0184] Therefore, the projection formula of the sample matrix X based on PCA is:
[0185] X′ = XP n×k ;
[0186] Among them, X′ is the sample matrix after dimensionality reduction, and each row of it is the feature after dimensionality reduction.
[0187] (11) Identify and classify the HOG-LBP features after dimensionality reduction using the SVM classification method. The SVM method is a supervised learning method that finds an optimal hyperplane in the feature space to distinguish different types of sample data.
[0188] The present invention analyzes SVM through two cases: linearly separable and linearly inseparable.
[0189] 1) Linearly separable case
[0190] Let the sample set to be classified be (x i , y i ), i = 1, …, n, y ∈ {+1, -1}. If there exists an optimal hyperplane H that can separate the two types of samples to the greatest extent, then let this hyperplane be:
[0191] ωx + b = 0;
[0192] In the formula, ω is an n-dimensional weight vector, and b is the bias. The linearly separable classification model is as Figure 6 shown in the figure. The black circles and white diamonds in the figure are two types of samples. H1 and H2 are hyperplanes parallel to H. The distance between H1 and H2 is the classification margin, and the sample points on H1 and H2 are the support vectors.
[0193] In order for the hyperplane H to correctly classify all samples, all sample points satisfy the formula:
[0194] y i (ω · xi +b)-1≥0;
[0195] To find the hyperplane H, we need to find the maximum classification interval first. According to the vector geometry theory, the classification interval is:
[0196]
[0197] To maximize D, even if ||ω|| is minimized, the problem of finding the optimal hyperplane can be converted into the formula:
[0198]
[0199] The Lagrangian optimization method is used to optimize the above formula:
[0200]
[0201] Among them, α i is the Lagrange coefficient. Taking partial derivatives of ω and b respectively and setting them equal to zero, we get:
[0202]
[0203] Substituting in:
[0204]
[0205] If the optimal solution exists, then we get the optimal ω * 、b * :
[0206]
[0207] Finally, the optimal classification function is:
[0208]
[0209] 2) Linearly inseparable case
[0210] For linearly inseparable sample data, SVM uses the kernel function method to map low-dimensional input to high-dimensional space, and by establishing the optimal hyperplane, it can achieve linear separability in high-dimensional space.
[0211] If there exists a function that maps low-dimensional input to high-dimensional Then the optimal classification function is transformed into:
[0212]
[0213] The above formula converts the calculation of the optimal classification surface into The inner product operation of , the kernel function is defined as:
[0214]
[0215] At present, the commonly used kernel functions of the SVM method are as follows:
[0216] ① Linear kernel function: K(x,x i )=(x·x i );
[0217] ② Polynomial kernel function: K(x,x i )=[(x·x i )+1] λ ,λ>0;
[0218] ③Gaussian kernel function:
[0219] ④S-type kernel function: K(x,x i ) = tanh(λ(x·x i )+c);
[0220] According to the number of sample features, sample quantity, recognition time requirements, etc., it is necessary to select a suitable kernel function to solve the corresponding practical problem. Considering that the binary classification problem to be solved in this paper has high speed requirements, the SVM method with a linear kernel is selected to identify and detect pedestrians.
[0221] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A pedestrian detection method applied to AGV, characterized in that: The following steps are involved: (1) Using a binocular camera to obtain an image of the detection area; (2) performing image grayscale conversion, median filtering, and histogram equalization processing on the acquired image in sequence to obtain a processed disparity map; (3) Obtain the three-dimensional coordinates of the effective disparity points in the disparity map; (4) Determine the ground interference area based on the obtained disparity map and the three-dimensional coordinates of the effective disparity points; (5) Set the pedestrian detection range value and calculate the area of interest and background interference area; (6) According to the determined interference area data, a binary segmentation method is used to extract the region of interest on the disparity map; (7) Extracting HOG features in the region of interest in the disparity map; in step (7), the steps of extracting HOG features are as follows: ① Grayscale and Gamma normalization; ② Calculate image gradient; ③ Gradient direction histogram weighted statistics; ④Block construction and normalization; ⑤Calculate the overall eigenvector; (8) Extracting LBP features in the region of interest in the disparity map; In the step (8), the method for extracting LBP features is as follows: Assume that each pixel has a gray value f c As the threshold, if the gray value of the surrounding pixels meets the threshold, the point is set to 1, otherwise it is set to 0; its 8 neighborhoods are coded by 0 / 1 to obtain a binary string of length 8, which is used as the encoding of the pixel, as follows: f c is the gray value of the center pixel; f p is the gray value of the pixel in the domain, and each sampling point in the domain has different weight coefficients; 2 p To assign sampling points, finally perform a summation operation; (9) The extracted HOG features and LBP features are fused in series to obtain the fused features HOG-LBP features of the entire image; (10) PCA method is used to perform fusion and dimensionality reduction processing on HOG-LBP features; (11) The HOG-LBP features after dimension reduction are identified and classified using the SVM classification method.
2. The pedestrian detection method applied to AGV according to claim 1, characterized in that: In step (3), the steps of obtaining the three-dimensional coordinates of the effective disparity point are as follows: ① Assume that in the physical coordinate system of the image, the principal point o l , o r The coordinates of x ,c y )、(c x ′,c y ′), the point P(X P ,Y P ,Z P ) of the imaging point P l , P r The coordinates of are (x l ,y l )、(x r ,y r ), x l -x r is the parallax d of point P, the baseline distance between the left and right cameras is B, and the focal length is f; ②In OpenCV, the default world coordinate system of the binocular vision system is OX w Y w Z w and the left camera coordinate system O l -X c Y c Z c They are coincident. According to the principle of geometry, we can get the formula: The above formula can be used to calculate the coordinate value X of point P in the world coordinate system. P , Y P , Z P :
3. The pedestrian detection method applied to AGV according to claim 1, characterized in that: In step (4), the determination steps are as follows: ① Establish the camera coordinate system OX based on the left camera C Y C Z C , point O is the optical center of the left camera, X C The axis points to the right camera direction, Z C The axis is parallel to the ground and points in the forward direction. C The axis is perpendicular to the ground; ② Let the distance from the camera optical center O to the ground be H, and the three-dimensional coordinates of a point P in three-dimensional space are (X P ,Y P ,Z P ), so the distance from point P to the ground is: h P =H-Y P ; ③Set a height threshold Δh, if h P <Δh, the current point is regarded as ground interference information.
4. The pedestrian detection method applied to AGV according to claim 1, characterized in that: In step (5), the steps for setting the pedestrian detection range value are as follows: ① Assume that the distance between the binocular camera and the object in the environment is D, and set a fixed distance threshold D m ; ②When D <D m , objects in the environment are likely to be pedestrian targets, and objects within this range are extracted as regions of interest; ③When D>D m , this area is considered as background interference area.
5. The pedestrian detection method applied to AGV according to claim 1, characterized in that: In the step (6), the steps of extracting the region of interest are as follows: ①Let h P <Δh or D>D m The grayscale value of the pixel is set to 0, and the grayscale values of the remaining pixels are set to 255; ②Call OpenCV's findContours function to extract the contour of the foreground suspicious target; ③Remove contours with a pixel area less than 3000; ④Use the boundingRect function to obtain the minimum rectangular boundary of the contour; ⑤Extract the region of interest from the original left image.
6. The pedestrian detection method applied to AGV according to claim 1, characterized in that: In the step (10), the fusion dimension reduction processing steps are as follows: Assume there are m samples {x (1) ,x (2) ,…,x (m) }, where each sample has n-dimensional HOG features, expressed as Where n=3780, i ranges from 1 to m; these m samples and the n-dimensional features contained in each sample form an m×n-dimensional sample matrix X, expressed as: First, preprocess the samples and calculate the mean of each column of X: Then, subtract the mean uj of the current column from the value of each column to obtain a sample matrix X with zero mean, and then calculate the covariance matrix: Next, find the eigenvalues and eigenvectors of the covariance matrix C: Cth t =λ t i t ; Among them, λ t is the eigenvalue, θ t is t The corresponding eigenvector; Finally, arrange λ t in descending order. If it is necessary to reduce the dimension from n to k (k < n), then take the eigenvectors corresponding to the first k eigenvalues to form the projection matrix P column by column n×k , which is expressed as: P n×k =[θ1,θ2,…,θ k ]; Therefore, the projection formula of the sample matrix X based on PCA is: X′=XP n×k ; Among them, X′ is the sample matrix after dimensionality reduction, and each row is the feature after dimensionality reduction.
