A gap foreign matter detection method based on visual image feature matching

Abstract

The application discloses a kind of gap foreign matter detection methods based on visual image feature matching, comprising the following steps: obtaining the video of gap detection area, feature extraction and initial matching are carried out to reference image and image to be detected;Using the nearest neighbor distance ratio NNDR algorithm carries out preliminary elimination of mis-matching point;Using local deletion method is further eliminated to the mis-matching point of one-to-many for the matching of a feature point in reference image and multiple feature points of image to be detected, to obtain effective matching point;Foreign matter detection is carried out by the number and distribution area ratio of effective matching point.The method can effectively eliminate mis-matching point when feature matching, improve matching accuracy;Foreign matter detection is carried out to gap using the relative number of effective matching feature point and the distribution area of effective matching point and other information, with the advantages of fast and accurate.

Description

Technical Field

[0001] This invention relates to the field of gap foreign object detection, and more particularly to a gap foreign object gun detection method based on visual image feature matching. Background Technology

[0002] There is a certain gap between the platform screen doors and the subway doors, called the clearance. This gap is necessary to prevent interference between the train and equipment inside the tunnel, as trains tend to sway during operation. However, due to this gap, when passenger flow is high or passengers are improperly pushing and shoving into a closing subway train, there is a possibility that people or objects could be trapped or left between the platform screen doors and the subway doors, causing a safety accident. To avoid such accidents, in addition to adding physical anti-pinch devices such as anti-pinch baffles and anti-climb frames between the platform screen doors and the train doors, automatic detection technologies are also used to detect obstacles, i.e., foreign objects.

[0003] Currently, the main methods for detecting foreign object intrusion between subway trains and platform screen doors include: infrared light curtain-based methods, laser detection-based methods, and laser scanning-based methods. With the rapid development of computer vision technology, methods have also emerged that use visual sensors to identify and detect obstacles between train doors and platform screen doors. For example, image processing methods can be used to identify the integrity of background light strips to detect whether people or objects are trapped between trains and platform screen doors on straight subway platforms; by installing light strips under each platform screen door as background references and using differential images for light strip color recognition, foreign object detection can be performed between trains and platform screen doors on curved subway platforms. However, these detection methods mostly rely on external references (such as external light strips) for foreign object intrusion detection. Such methods are ineffective at detecting short or small foreign objects, and their final detection performance is highly sensitive to changes in lighting, easily leading to false positives.

[0004] Feature extraction and matching are fundamental to many applications in computer vision. While not yet applied to foreign object detection in subway platform screen doors, they have been extensively studied in other areas. The feature extraction and matching process generally involves three steps: feature point detection, feature point descriptor construction, and feature point descriptor matching. In feature matching, Euclidean distance is most commonly used as a similarity metric. While this can quickly find matching points, when the image is complex and contains multiple similar elements, the differences between feature descriptors are small, leading to "one-to-many" or "many-to-one" mismatches. These mismatches affect the accuracy of matching and detection, necessitating mismatch removal. Current methods for mismatch removal include Pierre correlation coefficient-based methods, epipolar geometry constraints, homography constraints, RANSAC algorithms, and angle reduction methods. However, these methods increase computational complexity and can only eliminate local mismatches. Summary of the Invention

[0005] Purpose of the invention: The present invention aims to provide a gap foreign object detection method based on visual image feature matching that can quickly and accurately identify foreign objects.

[0006] Technical solution: The present invention provides a gap foreign object detection method based on visual image feature matching, comprising the following steps:

[0007] (1) Obtain the video of the gap detection area, and perform feature extraction and initial matching on the reference image and the image to be detected;

[0008] (2) The nearest neighbor ratio (NNDR) algorithm is used to initially remove mismatched points;

[0009] (3) The local deletion method is used to further eliminate mismatched points that match one feature point in the reference image with multiple feature points in the image to be detected, so as to obtain effective matching points;

[0010] (4) Foreign object detection is performed by comparing the number and distribution area of ​​effective matching points between the reference image and the image to be inspected.

[0011] Preferably, the method for feature extraction and initial matching in step (1) is the SURF algorithm.

[0012] Preferably, the nearest neighbor ratio (NNDR) algorithm in step (2) includes: for the initial matching point (X) between the reference image and the image to be detected... i Y j’ ), calculate the minimum Euclidean distance D i，j The second smallest Euclidean distance D i，j′ The ratio, i.e.

[0013]

[0014] The ratio η is compared with the first threshold η ε Comparison, if η > η ε If the feature point is a mismatch, it should be removed.

[0015] Preferably, the local deletion method in step (3) includes: calculating the matching points Y in the image to be inspected that correspond to the feature points in the reference image. i The pixel coordinates and the minimum Euclidean distance matching point Y s The distance between the pixel coordinates and the minimum Euclidean distance matching point Y. s The pixel coordinates and the second smallest Euclidean distance of the matching point Y s′ The distance between the pixel coordinates is calculated, and the ratio of the two distances is also calculated.

[0016]

[0017] Among them, (x s y s Y is the minimum Euclidean distance matching point. s Pixel coordinates; (x s′ y s′ Y is the second smallest Euclidean distance matching point. s′ pixel coordinates, (x i y i Y is the matching point Y in the image to be inspected that corresponds to the feature point in the reference image. i The pixel coordinates, i≠s≠s′:

[0018] The obtained ratio θ s，i With the second threshold θ ε Comparison, if θ s，j >θ ε If θ s，j ≤θ ε If the match is valid, the matching point is retained.

[0019] Preferably, in step (4), foreign object detection is performed by comparing the number of valid matching points between the reference image and the image to be inspected, including removing the number p of valid matching points after removing erroneous matching points. o Compared with the first standard reference value p κ Compare, if p o <αp κ If the value is positive, it is considered that a foreign object exists; otherwise, the effective matching point distribution area is detected, where the first standard reference value p is used. κ The number of valid matching points between the reference image and the detection image when there are no foreign objects is represented, and α is an adjustable coefficient.

[0020] Preferably, in step (4), foreign object detection is performed by comparing the distribution area of ​​effective matching points in the reference image and the image to be inspected, including calculating the area of ​​the polygon formed by edge feature points, q k (k = 1, 2, ..., M) are the vertices of the feature point boundary arranged in a positive direction, with corresponding coordinates (x1, y1), (x2, y2), ..., (x...). M y M The calculation formula is as follows:

[0021]

[0022] The obtained feature point distribution area P Ω Compared with the second standard reference value P Ω 'Compare, if P Ω <βP Ω If the value is ', then the presence of a foreign object is considered; otherwise, the absence of a foreign object is considered. The second standard reference value P... κ β represents the effective matching point distribution area between the reference image and the detection image when there are no foreign objects, and β is an adjustable coefficient.

[0023] Preferably, before feature extraction and initial matching, the gap detection areas of the reference image and the image to be inspected are divided into N detection blocks respectively to facilitate the location of foreign objects.

[0024] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: it effectively eliminates mismatched points during feature matching, thereby improving the matching accuracy; based on points and / or surfaces, that is, using information such as the relative number of effective matching feature points and the distribution area of ​​effective matching points to detect foreign objects in gaps, it has the advantages of being fast and accurate. Attached Figure Description

[0025] Figure 1 This is a flowchart of the method of the present invention;

[0026] Figure 2 These are the reference image and the target image for Experiment Group 1 of the present invention, wherein (a) is the reference image and (b) is the target image;

[0027] Figure 3 These are the reference image and the target image for Experiment Group 2 of the present invention, where (a) is the reference image and (b) is the target image. Detailed Implementation

[0028] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0029] like Figure 1As shown, the gap foreign object detection method based on visual image feature matching of the present invention mainly includes four parts: detection area segmentation, feature extraction and initial matching, removal of mismatched points and foreign object detection.

[0030] (1) Detection area segmentation

[0031] The video of the gap detection area is acquired, and the reference image and the image to be inspected are processed into corresponding blocks. That is, the reference image is divided into N detection blocks, and the image to be inspected is divided into N detection blocks accordingly.

[0032] (2) Feature extraction and initial matching

[0033] The SURF algorithm is used to extract features and perform initial matching on each detection block of the reference image and the image to be detected after the detection region is divided into blocks, including feature point localization, feature descriptor selection and feature point matching.

[0034] Feature point localization: A box filter is used for filtering. The box filter simplifies the Gaussian filter's weight coefficients, reducing the image convolution operation to a simple addition / subtraction of pixel sums between different image regions, thus decreasing computational complexity and increasing processing speed. For the image filtered by the Box filter, the Hessian determinant value at the corresponding scale coordinates is calculated. All determinant values ​​constitute a Hessian determinant image. A 3×3×3 neighborhood space is selected within this scale space for feature point selection. Each Hessian determinant value in the middle layer is used as a candidate feature point. Each candidate feature point is compared with the Hessian determinant values ​​of its own layer and the two adjacent layers. If the value of a candidate point is greater than the other 26 points, it is considered a feature point. Furthermore, to ensure the quantity and accuracy of feature points, a given threshold is selected and compared with the Hessian determinant value of each point. Points with determinant values ​​higher than this threshold are identified as feature points.

[0035] Feature descriptor selection: Centered on the feature point, within a neighborhood of radius 6ε (ε being the scale of the feature point), calculate the Haar wavelet responses in the horizontal and vertical directions, and then sum the weighted Haar wavelet response values ​​in the horizontal and vertical directions. Based on this, a fan-shaped sliding window with an angle of π / 6 is set centered on the feature point. As the sliding window rotates, the accumulated wavelet response values ​​within the window are statistically analyzed, forming a vector. The direction of the vector with the largest sum is determined as the principal direction of the feature point, which is also the direction of the most significant gray-level change around that point. After confirming the main direction of the feature point, a square region with a side length of 20ε is selected with the feature point as the center. The square region is divided into 4×4 sub-regions along the main direction of the feature point. In each sub-region, 5×5 uniformly distributed sampling points are selected, and the Haar wavelet response values ​​(filter size 2ε) along and perpendicular to the main direction are calculated. Gaussian weighted calculation is performed on the wavelet response in the corresponding direction. Each sub-region can obtain a 4-dimensional feature vector. Finally, a 4×4×4-dimensional feature vector is formed. This feature vector has scale and rotation invariance and is robust to changes in illumination.

[0036] Feature point matching: Based on the sign (positive or negative) of the calculated Hessian matrix trace, feature points are divided into two groups. The Euclidean distance between the descriptors of two feature points to be matched in the same group is then calculated.

[0037]

[0038] Among them, X ik For the k-th element of the descriptor of the i-th feature point in the reference image, Y ik Let n be the k-th element of the j-th feature descriptor of the target image, and n be the dimension of the descriptor. The Euclidean distance is compared with a selected threshold. If the distance is less than the selected threshold, the feature points are considered to be a set of matching points; otherwise, they are discarded.

[0039] (3) Removal of mismatched points

[0040] For the initial matching point (X) between the reference image and the image to be detected i Y j’ ), calculate the minimum Euclidean distance D i，j The second smallest Euclidean distance D i，j′ The ratio, i.e.

[0041]

[0042] The ratio η is compared with the first threshold η ε Comparison, if η > η ε If the feature point is a mismatch, it should be removed.

[0043] By comparing the ratio of the minimum Euclidean distance to the second minimum Euclidean distance with a threshold, the initial elimination of mismatched points is completed. However, there may still be cases where a feature point in the reference image matches multiple feature points in the target image, requiring further elimination of mismatched pairs in the one-to-many relationship.

[0044] Assume the t-th feature point X in the reference image t The corresponding matching points in the target image are Y. m Y m+1 Y m+2 ...Y m+n In the initial elimination algorithm, the minimum Euclidean distance matching point has been found, meaning there exists a point Y. s :

[0045] Y s =arg min{dist(X t Y m ), dist(X t Y m+1 ), ..., dist(X) t Y m+n )}

[0046] Where s∈(m, m+n); similarly, there exists a matching point Y with the second smallest Euclidean distance. s s′, s′≠s∈(m, m+n). Perform the following calculations:

[0047]

[0048] Among them, (x s y s Y is the minimum Euclidean distance matching point. s Pixel coordinates; (x s′ y s′ Y is the second smallest Euclidean distance matching point. s′ pixel coordinates, (x i y i Y is the matching point Y in the image to be inspected that corresponds to the feature point in the reference image. i The pixel coordinates, i≠s≠s′.

[0049] The obtained ratio θ s，i With the set threshold θ ε Comparison, if θ s，j >θ ε If θ s，j ≤θ ε If the match point is valid, it will be retained.

[0050] (4) Foreign object detection

[0051] Foreign object detection and location are performed by comparing the number and distribution area of ​​effective matching points in each detection block.

[0052] For detection block A in the reference image o And detection block B in the target image o For each feature point o∈(1,N), after feature extraction, matching, and removal of mismatched points, the number of feature points p can be obtained. o Compare it with the first standard reference value p κ For comparison, the first standard reference value p κ This represents the number of valid matching points between the reference image and the detection image when there are no foreign objects.

[0053] If p o <αp κ If α is an adjustable coefficient, then the detection block is considered to contain foreign objects; otherwise, feature point area distribution detection is performed.

[0054] The area of ​​valid matching points in the detection block is obtained by calculating the area of ​​the polygon formed by the edge feature points, using the following formula:

[0055]

[0056] Where, q k (k = 1, 2, ..., M) are the vertices of the feature point boundary arranged in a positive direction, with corresponding coordinates (x1, y1), (x2, y2), ..., (x...). M y M );

[0057] The obtained feature point distribution area P Ω Compared with the second standard reference value P Ω 'Compare, if P Ω <βP Ω If the value is ', then the presence of a foreign object is considered; otherwise, the absence of a foreign object is considered. The second standard reference value P... κ β represents the effective matching point distribution area between the reference image and the detection image when there are no foreign objects, and β is an adjustable coefficient.

[0058] Two sets of test experiments were conducted on the gap foreign object detection method based on visual image feature matching described in this invention. The experimental hardware environment was a PC with Windows 10 system, Intel(R) Core(TM) i7-8550U 1.8GHz CPU, and 8GB of memory. The software development platform was Matlab. Video images were captured at 30fps (30 frames per second) with a resolution of 1920*1080. One target frame was uniformly sampled from the video every ten frames for detection. That is, the foreign object presence state of each target frame represents the detection results of the adjacent ten frames.

[0059] For experimental group one, the reference image and the target image are as follows: Figure 2 As shown in Table 1, taking N=4 as an example, the reference image and the target image are each divided into 4 detection blocks. The ratio of the number of effective feature points and the ratio of their area distribution in each detection block are shown in Table 1. The ratio of the number of feature points and the ratio of their area distribution in detection block 1 and detection block 2 are both greater than the threshold of 0.7, therefore, these parts are determined to be free of foreign object intrusion. The ratio of the number of feature points in detection block 3 and detection block 4 are both less than 0.7, at 0.422 and 0.102 respectively. This indicates that the number of matching pairs in these two parts has been significantly reduced compared to the initial state when there were no foreign objects, suggesting a high probability of foreign object intrusion. Therefore, according to the algorithm, they are determined to contain foreign objects.

[0060] Table 1 Experimental Group 1 (α=0.7, β=0.7)

[0061]

[0062] For experimental group two, the reference image and the target image are as follows: Figure 3 As shown in Table 2, taking N=4 as an example, the reference image and the target image are each divided into 4 detection blocks. The ratio of the number of effective feature points and the ratio of their area distribution in each detection block are shown in Table 2. The ratios of the number of feature points and the area distribution in detection blocks 1 and 4 are both greater than the threshold, therefore, it is determined that there are no foreign objects in these blocks. For detection block 2, the ratio of the number of feature points is less than the threshold, indicating a significant reduction in the number of feature points and the possibility of foreign object intrusion. For detection block 3, the ratio of the number of feature points is greater than the threshold, but the area distribution of the feature points is less than the threshold, indicating a significant difference in the distribution of feature points compared to when there are no foreign objects, and the detection result is that there are foreign objects.

[0063] Table 2 Experimental Group 2 (α=0.7, β=0.7)

[0064]

Claims

1. A method for detecting foreign objects in gaps based on visual image feature matching, characterized in that, Includes the following steps: (1) Obtain the video of the gap detection area, and perform feature extraction and initial matching on the reference image and the image to be detected; (2) The nearest neighbor ratio (NNDR) algorithm is used to initially remove mismatched points; (3) After completing the initial elimination of mismatches, the local deletion method is used to further eliminate one-to-many mismatches to obtain valid matching points; The local deletion method in step (3) includes: calculating the matching points Y in the image to be inspected that correspond to the feature points in the reference image. i The pixel coordinates and the minimum Euclidean distance matching point Y s The distance between the pixel coordinates and the minimum Euclidean distance matching point Y. s The pixel coordinates and the second smallest Euclidean distance of the matching point Y s' The distance between the pixel coordinates is calculated, and the ratio of the two distances is calculated. The resulting ratio θ is then used to calculate the distance between the pixel coordinates. s,i With the second threshold θ ε Comparison, if θ s,j >θ ε If θ s,j ≤θ ε If the match is valid, the match point is retained. (4) Foreign object detection is performed by comparing the number and distribution area of ​​effective matching points between the reference image and the image to be inspected; In step (4), foreign object detection is performed by comparing the number of valid matching points between the reference image and the image to be inspected, including removing the number of valid matching points p after removing the mismatched points. o Compared with the first standard reference value p κ Compare, if p o <αp κ If the value is positive, it is considered that a foreign object exists; otherwise, the effective matching point distribution area is detected, where the first standard reference value p is used. κ α represents the number of effective matching points between the reference image and the detection image when there are no foreign objects, and α is an adjustable coefficient. In step (4), foreign object detection is performed by comparing the distribution area of ​​effective matching points in the reference image and the image to be inspected. This includes calculating the area of ​​the polygon formed by the edge feature points and then calculating the distribution area P of the obtained feature points. Ω Compared with the second standard reference value P Ω 'Compare, if P Ω <βP Ω If the value is ', then the presence of a foreign object is considered; otherwise, the absence of a foreign object is considered. The second standard reference value P... Ω ' represents the effective matching point distribution area between the reference image and the detection image when there are no foreign objects, and β is an adjustable coefficient.

2. The gap foreign object detection method according to claim 1, characterized in that, The method for feature extraction and initial matching in step (1) is the SURF algorithm.

3. The gap foreign object detection method according to claim 1, characterized in that, The nearest neighbor ratio (NNDR) algorithm in step (2) includes: for the initial matching point (X) between the reference image and the image to be detected... i Y j’ ), calculate the minimum Euclidean distance D i,j The second smallest Euclidean distance D i,j' The ratio, i.e. The ratio η is compared with the first threshold η ε Comparison, if η > η ε If the feature point is a mismatch, it should be removed.

4. The gap foreign object detection method according to claim 1, characterized in that, Before feature extraction and initial matching, the gap detection regions of the reference image and the image to be inspected are divided into N detection blocks respectively.

Images