Image foreign matter detection method based on feature library search

Abstract

The invention discloses an image foreign matter detection method based on feature library search. Comprising the following steps: marking a suspected area in an image; extracting feature information of the suspected area, wherein the feature information comprises one or more of a gray-level co-occurrence vector, a gradient direction feature vector, a gray-level feature vector, a key point coordinate set and a morphological feature vector; according to the feature information / the area of the suspected region, marking the suspected region as a foreign matter region / false detection region, and correspondingly storing the feature information in a foreign matter feature library / false detection feature library to complete foreign matter detection of the current image; continuously marking the newly collected images until the foreign matter detection of all the images is completed; the method aims at improving the accuracy of foreign matter detection, foreign matter samples do not need to be collected in advance, the feature library is continuously expanded along with the actual detection process, the detection efficiency and accuracy are improved, and missed recognition and wrong recognition are avoided.

Description

Technical Field

[0001] This invention relates to the field of foreign object detection, and more specifically to an image foreign object detection method based on feature library search. Background Technology

[0002] Foreign object and defect detection in images has always been a key topic in the field of visual inspection. When image quality is poor and the contrast between foreign objects and real features is not obvious, existing image foreign object detection methods are prone to false positives and false negatives. For example, in the process of foreign object detection in pantograph images, because the pantograph is in an outdoor environment, the image is easily affected by external ambient light. Traditional methods have poor robustness and cannot resist external interference such as ambient light, resulting in a high false positive rate. They also rely too much on prior knowledge, and the algorithms cannot detect foreign objects of different sizes and types. When a foreign object (plastic bag, balloon, dead animal carcass, construction machinery parts, etc.) intrudes into the pantograph, it can cause malfunctions such as pantograph getting stuck in the wire mesh, wire mesh being pulled, and line fires, and may even cause personal injury. Therefore, once a foreign object intrusion occurs into the pantograph, it is necessary to immediately stop the train for inspection and repair to ensure the safe operation of the train.

[0003] The following foreign object intrusion detection methods exist in the prior art:

[0004] CN111738342A proposes a foreign object detection method based on contour extraction and SVM classification. This method has two problems: first, the contour extraction has poor stability and cannot adapt to the ever-changing situation on site; second, there is a lack of sufficient negative sample defect images in reality to train the SVM model, making the method less feasible.

[0005] CN113487561A proposes a foreign object detection method based on gray-level gradient anomaly voting. This method compares the gray-level changes of consecutive frames of images based on the edge of the pantograph to locate the foreign object region. This method relies entirely on gray-level information for detection, which is inevitably affected by season, weather, and lighting, resulting in an artificially high false detection rate.

[0006] CN113298059A proposes a foreign object detection method for pantographs based on ROI localization and monitoring. This method first locates the ROI area of ​​the pantograph through target detection, and then filters the foreign object area through segmentation, grayscale difference and other processing. However, this method cannot avoid ambient light interference, and the target detection cannot achieve pixel-level localization, which inevitably leads to a high false detection rate.

[0007] CN114494161A proposes a foreign object detection method for pantographs based on image comparison. This method uses a small sliding window to slide across the image and compares the similarity threshold within the window to locate the foreign object region. However, this method has certain limitations. First, the similarity of the small sliding window cannot evaluate the entire foreign object region. Second, the setting of the small sliding window size relies too much on prior knowledge and is not universally applicable to foreign objects of different sizes. Finally, the algorithm calculates a large number of small sliding window similarities across the entire image, resulting in low algorithm efficiency.

[0008] CN115170954A proposes a method based on ROI region and edge extraction for detecting foreign object regions. This method first uses YOLO to locate the pantograph ROI region, segments the region, and searches for the contour information of foreign objects to locate the foreign object region. However, this method also has some problems. First, the use of object detection to locate the ROI region has a deviation. Second, the strategy of searching for foreign objects by edge contour information has a certain effect on the detection of foreign objects on the edge of the skateboard, but the effect is poor for the detection of objects in the support and suspended areas, and the feasibility is low.

[0009] In summary, current foreign object intrusion detection solutions have the following shortcomings:

[0010] ① Existing methods have low accuracy in locating key ROI regions in images and are affected by background information, resulting in a high false detection rate.

[0011] ② Detection based on deep learning models requires the prior collection of a large number of images containing foreign objects for training. Since foreign object intrusion events are low-probability anomalies with randomness and diversity, it is difficult to collect pantograph samples with foreign object intrusion, making such solutions difficult to implement. Summary of the Invention

[0012] To address the aforementioned technical problems, this invention provides an image foreign object detection method based on feature library search; this method aims to improve the accuracy of image foreign object detection and avoid missed or false identification.

[0013] The technical solution is as follows:

[0014] A method for detecting foreign objects in images based on feature library search includes the following steps:

[0015] 1) Extract the region of interest from the acquired image; compare the region of interest with the standard image, identify the regions that differ from the standard image, and mark them as suspected regions;

[0016] The standard image is: an image of the region of interest from a pre-stored normal image;

[0017] 2) Extract feature information of the suspected region, wherein the feature information includes one or more of the following: gray-level co-occurrence vector, gradient direction feature vector, gray-level feature vector, key point coordinate set, and morphological feature vector;

[0018] 3) Based on the feature information / area of ​​the suspected area, mark the suspected area as a foreign object area / false detection area, and store the feature information accordingly in the foreign object feature library / false detection feature library to complete the foreign object detection of the current image;

[0019] 4) Continue with step 1) on the newly acquired images until all images have been detected for foreign objects.

[0020] Furthermore, in step 3), the method of marking the suspected area as a foreign object area / false detection area based on the feature information / area of ​​the suspected area is as follows: based on the similarity between the feature information and the feature information already stored in the foreign object feature library / false detection feature library, or based on the size of the suspected area, the suspected area is identified as a foreign object area / false detection area.

[0021] Furthermore, in step 3), the steps of marking the suspected area as a foreign object area / false detection area based on the feature information and storing the feature information accordingly in the foreign object feature library / false detection feature library are as follows:

[0022] ① Determine whether the feature information has been stored in the foreign object feature database:

[0023] If not, proceed directly to step ②;

[0024] If so, retrieve the feature information of each element from the foreign object feature database, and calculate the similarity between each element and the feature information of the suspected region. Take the maximum similarity value, denoted as L. 异物 When L 异物 If the value exceeds a preset threshold I, the suspected area is marked as a foreign object area and the corresponding feature information is stored in the foreign object feature library, proceeding to step ③; otherwise, it continues to determine whether feature information has been stored in the false detection feature library. If not, proceed to step ②; if so, retrieve the feature information of each item in the false detection feature library and calculate the similarity between each item and the feature information of the suspected area, taking the maximum similarity value as L. 误检 ; Determine L 误检 If the value is greater than the threshold I, then the suspected area is recorded as a false detection area and the corresponding feature information is stored in the false detection feature library, and step ③ is performed; if the value is not greater than the threshold I, then step ② is performed.

[0025] ② Determine whether the area of ​​the suspected region is greater than the threshold. If it is, mark the suspected region as a foreign object region and store its corresponding feature information in the foreign object feature library. If it is not greater, mark the suspected region as a false detection region and store its corresponding feature information in the false detection feature library.

[0026] ③ Complete foreign object detection in the current image.

[0027] Furthermore, the feature information of each element in the foreign object feature database / false detection feature database is retrieved, and the similarity between each element and the feature information corresponding to the suspected region is calculated, as follows:

[0028] When the feature information contains one or more of gray-level co-occurrence vectors, gradient direction feature vectors, gray-level feature vectors, and morphological feature vectors, the similarity calculation is a cosine similarity calculation.

[0029] When the feature information contains a set of key point coordinates, the similarity calculation method is as follows: calculate the Euclidean distance between each point coordinate in the set of key point coordinates corresponding to the suspected area and each point coordinate in the set of key point coordinates in the foreign object feature library / false detection feature library. If the distance value is less than the preset distance value, the point coordinates are marked as a positive key point; otherwise, they are marked as a negative key point. Count the total number of positive and negative key points, and use the ratio of the number of positive key points to the total number as the similarity.

[0030] If the feature information contains multiple features such as gray-level co-occurrence vector, gradient direction feature vector, gray-level feature vector, morphological feature vector, and key point coordinate set, the resulting multiple similarities will be weighted and fused to obtain the final similarity.

[0031] Preferably, step 4) further includes the following step:

[0032] I. Review the foreign object area / false detection area. When a suspected area is found to be incorrectly marked, correct the marking type of the suspected area and store its corresponding feature information in the correct feature library.

[0033] II. When the number of foreign object areas / false detection areas reaches the preset value N, the recognition accuracy is calculated. If the accuracy meets the requirements, step I ends; otherwise, step I continues.

[0034] Where N≥100.

[0035] Preferably, the set of key point coordinates includes the coordinates of the centroid of the suspected region and / or the coordinates of key points with scale invariance, wherein the coordinates are relative coordinates with the corner of the smallest bounding rectangle of the suspected region as the starting point.

[0036] Preferably, the morphological feature vector includes multiple features such as: the ratio of the suspected region area to the area of ​​the smallest circumscribed rectangle, the ratio of the suspected region area to the area of ​​the smallest circumscribed circle, the ratio of the suspected region area to the perimeter, and the directional features of the suspected region.

[0037] Among them, the directional feature of the suspected region is the angle between the line segment from the centroid of the suspected region to the midpoint and the horizontal direction. The midpoint is: traversing any two edge points in the suspected region, taking the midpoint of the line connecting the two farthest edge points, and recording it as the midpoint.

[0038] Furthermore, the gray-level co-occurrence vector includes energy features, contrast features, inverse difference moment features, entropy features, and autocorrelation features, which are derived from the gray-level co-occurrence matrix;

[0039] The gradient direction feature vector is obtained by the following method: obtaining the gradient direction of all pixels in the suspected area, dividing 0° to 180° into multiple angle intervals according to a preset angle interval, and summing the frequency of the gradient direction of the pixel in each angle interval to form the gradient direction feature vector.

[0040] The gray-level feature vector is a 256-dimensional vector, which is derived from the gray-level histogram.

[0041] Preferably, in step 1),

[0042] In step 1), the method for extracting the region of interest from the acquired image is as follows:

[0043] Collect multiple images of the object under normal conditions, select the region of interest in each image and label it to form a sample dataset; use the sample dataset to train a deep learning model;

[0044] In the actual detection process, the collected image of the object to be tested is input into the pre-trained deep learning model, and the model outputs the region of interest in the image and coarsely locates the region of interest.

[0045] Then, template matching and image alignment are used to refine the coarsely located region of interest to obtain the region of interest.

[0046] Preferably, in step 1), the method for marking suspected areas is optical flow, background subtraction, or frame difference.

[0047] This method has the following characteristics:

[0048] 1) This method designs a foreign object feature library and a false detection feature library, which stores the grayscale, texture, key points and other feature information corresponding to real foreign objects and false detection foreign objects detected in the past. The feature information is used to compare the features with the current suspected area to identify the type of suspected area.

[0049] As the detection process progresses, the foreign object feature library and the false detection feature library will continuously store the feature information corresponding to newly identified foreign object areas and false detection areas, preparing for the next identification. As the feature information in the foreign object feature library and the false detection feature library continues to expand, the identification accuracy will continue to rise and reach a stable level.

[0050] This solution does not require pre-collection of foreign object samples. Instead, it continuously expands the feature library as the actual detection process progresses, thereby improving detection efficiency and accuracy, making it highly feasible.

[0051] 2) By utilizing multiple features (gray-level co-occurrence vector, gradient direction feature vector, gray-level feature vector, key point coordinate set and morphological feature vector) to construct feature information of suspected regions, it can further resist the interference of ambient light and weather changes on the image of the object under test (e.g., pantograph image). At the same time, it also has a good detection effect on foreign objects of the same type but different sizes, further improving the algorithm performance.

[0052] 3) When extracting regions of interest from images, the method of coarse localization by target detection and fine localization by template matching is used to achieve pixel-level accurate localization, reduce the interference of background information, greatly improve the accuracy of detection, and reduce the false detection rate of the algorithm. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the suspected area identification process in a specific implementation method;

[0054] Figure 2 This is a schematic diagram of the feature library update process;

[0055] Figure 3 A schematic diagram illustrating the process of obtaining directional features for suspected regions. Detailed Implementation

[0056] The technical solution of the present invention will be described in detail below with reference to specific embodiments.

[0057] A method for detecting foreign objects in images based on feature library search includes the following steps:

[0058] 1) Extract the region of interest from the collected data;

[0059] Taking pantograph foreign object detection as an example, the processing method is as follows:

[0060] Collect multiple pantograph images under normal conditions, select the carbon skateboard area in each image and label it to form a sample dataset; use the sample dataset to train a deep learning model;

[0061] In the actual detection process, the collected pantograph images are input into a pre-trained deep learning model, and the model outputs the coarse localization area of ​​the carbon skateboard in the image.

[0062] Then, template matching and image alignment are used to refine the coarse localization area of ​​the carbon skateboard, resulting in the carbon skateboard region (region of interest).

[0063] The region of interest is then compared with the standard image to identify areas that differ from the standard image and mark them as suspected areas. Specific methods include optical flow, background subtraction, or frame difference for foreground detection.

[0064] The standard image is: an image of the region of interest in a pre-stored normal image; for example: a pre-stored image of the carbon skateboard region in its normal state;

[0065] If there are no suspected areas, meaning the current carbon skateboard area is highly consistent with the standard image, then the current pantograph image is a normal image and no further identification is needed. Proceed directly to step 4) and input the next pantograph image.

[0066] 2) Extract feature information of suspected regions. Feature information includes one or more of the following: gray-level co-occurrence vector, gradient direction feature vector, gray-level feature vector, key point coordinate set, and morphological feature vector.

[0067] In specific implementation, the grayscale feature vector is a 256-dimensional vector, which is obtained through the grayscale histogram.

[0068] The gradient direction eigenvector is obtained in the following way:

[0069] Obtain the gradient direction of all pixels in the suspected region (Sobel operator / Scharr operator), divide 0° to 180° into multiple angle intervals according to preset angle intervals, and summarize the frequency of the gradient direction of the pixel in each angle interval to form a gradient direction feature vector.

[0070] The gray-level co-occurrence vector includes energy features, contrast features, inverse difference moment features, entropy features, and autocorrelation features, which are derived from the gray-level co-occurrence matrix.

[0071] The set of key point coordinates includes the coordinates of the centroid of the suspected region and / or the coordinates of key points with scale invariance, which facilitates subsequent comparison of the similarity between coordinates. More preferably, the coordinates are relative coordinates with the corner of the smallest bounding rectangle of the suspected region as the starting point.

[0072] Specifically, taking the centroid (x, y) as an example, its corresponding relative coordinate P is in the form of:

[0073] (x p y p ) represents the coordinates of the top-left corner of the smallest bounding rectangle of the suspected region, and W and H are the column width and row height of the smallest bounding rectangle of the suspected region, respectively.

[0074] Storing the relative position information of key points and the minimum bounding rectangle in this form of relative coordinates can avoid the key point coordinate deviation caused by the different sizes of suspected foreign objects.

[0075] The morphological feature vector includes: the ratio of the suspected region area to the area of ​​the smallest circumscribed rectangle, the ratio of the suspected region area to the area of ​​the smallest circumscribed circle, the ratio of the suspected region area to the perimeter, and various directional features of the suspected region;

[0076] Among them, the directional feature of the suspected region is the angle between the line segment from the centroid of the suspected region to the midpoint and the horizontal direction. The midpoint is: traversing any two edge points in the suspected region, taking the midpoint of the line connecting the two farthest edge points, and recording it as the midpoint.

[0077] To facilitate the calculation, in practice, the 0 to 360 degree range can be divided into 20-degree intervals. Each angle interval is marked with a number in a counterclockwise direction to obtain interval numbers from 1 to 18. The included angle is then classified into the angle interval, and the corresponding interval number is stored as the suspected region direction feature.

[0078] 3) Based on the feature information / area of ​​the suspected area, mark the suspected area as a foreign object area / false detection area, and store the feature information accordingly in the foreign object feature library / false detection feature library to complete the foreign object detection of the current image;

[0079] Specifically, such as Figure 1 , 2 As shown, based on the similarity between the feature information and the feature information already stored in the foreign object feature library / false detection feature library, the suspected area is identified as the foreign object area / false detection area.

[0080] That is: if the similarity between the feature information and the feature information already stored in the foreign object feature library is higher than the preset threshold I, then the suspected area is identified as a foreign object area; if the similarity between the feature information and the feature information already stored in the false detection feature library is higher than the preset threshold I, then the suspected area is identified as a false detection area.

[0081] Alternatively, based on the size of the suspected area, the suspected area can be identified as a foreign object area or a false detection area.

[0082] In practice, the exemplary execution steps are as follows:

[0083] ① Determine whether the feature information has been stored in the foreign object feature database:

[0084] If not, proceed directly to step ②;

[0085] If so, retrieve the feature information of each element from the foreign object feature database, and calculate the similarity between each element and the feature information of the suspected region. Take the maximum similarity value, denoted as L. 异物 When L 异物If the value exceeds a preset threshold I, the suspected area is marked as a foreign object area and the corresponding feature information is stored in the foreign object feature library, proceeding to step ③; otherwise, it continues to determine whether feature information has been stored in the false detection feature library. If not, proceed to step ②; if so, retrieve the feature information of each item in the false detection feature library and calculate the similarity between each item and the feature information of the suspected area, taking the maximum similarity value as L. 误检 ; Determine L 误检 If the value is greater than the threshold I, then the suspected area is recorded as a false detection area and the corresponding feature information is stored in the false detection feature library, and step ③ is performed; if the value is not greater than the threshold I, then step ② is performed.

[0086] ② Determine whether the area of ​​the suspected region is greater than the threshold. If it is, mark the suspected region as a foreign object region and store its corresponding feature information in the foreign object feature library. If it is not greater, mark the suspected region as a false detection region and store its corresponding feature information in the false detection feature library.

[0087] ③ Complete foreign object detection in the current image.

[0088] In step ①, the feature information of each item in the foreign object feature library / false detection feature library is retrieved, and the similarity between each item and the feature information corresponding to the suspected area is calculated, as follows:

[0089] When the feature information contains one or more of the following: gray-level co-occurrence vector, gradient direction feature vector, gray-level feature vector, and morphological feature vector, the similarity calculation is a cosine similarity calculation.

[0090] When the feature information contains a set of key point coordinates, the similarity is calculated as follows: calculate the Euclidean distance between each point coordinate in the set of key point coordinates corresponding to the suspected area and each point coordinate in the set of key point coordinates in the foreign object feature library / false detection feature library. If the distance value is less than the preset distance value, the point coordinates are marked as a positive key point; otherwise, they are marked as a negative key point. Count the total number of positive and negative key points, and use the ratio of the number of positive key points to the total number as the similarity.

[0091] If the feature information includes multiple features such as gray-level co-occurrence vector, gradient direction feature vector, gray-level feature vector, morphological feature vector, and keypoint coordinate set, the resulting similarity scores are weighted and fused to obtain the final similarity score. In practice, since the gray-level co-occurrence vector and gray-level feature vector are related to the image gray level, and the image gray level is easily affected by external factors such as ambient light, they are given relatively small weights; the gradient direction feature vector, morphological feature vector, and keypoint coordinate set mainly reflect the image texture features, and therefore are given larger weights.

[0092] For example: Let E be a single feature in the foreign object feature library / false detection feature library, which includes the gray-level co-occurrence vector G.k Gradient direction eigenvector T j and the set of key point coordinates S;

[0093] Let E' be the feature information corresponding to the suspected region, which includes the gray-level co-occurrence vector G'. k and gradient direction eigenvector T' j and the set of key point coordinates S';

[0094] In this embodiment, the grayscale feature vector is a 256-dimensional vector.

[0095] At this point, the gray-level feature vector G is calculated. k G' k Cosine similarity F1 between them:

[0096]

[0097] In this embodiment, the gradient direction feature vector is obtained as follows: the gradient directions of all pixels within the suspected region are acquired, and the 0° to 180° angles are divided into 18 angular intervals according to a preset angular interval. The frequency of the gradient direction of each pixel in each angular interval is then summarized to form the gradient direction feature vector. Therefore, the gradient direction feature vector is an 18-dimensional vector.

[0098] Calculate the gradient direction eigenvector T j 、T' j Cosine similarity F2 between them:

[0099]

[0100] In this embodiment, the key point coordinate set S' corresponding to the suspected area contains 3 key point coordinates; the similarity calculation method is as follows: let the 3 point coordinates in the key point coordinate set S' corresponding to the suspected area calculate the Euclidean distance with the coordinates of each point in the key point coordinate set S in the foreign object feature library / false detection feature library. If the distance value is less than the preset distance value, the point coordinate is marked as a positive key point, otherwise it is marked as a negative key point. The total number of positive key points and negative key points is counted, and the ratio of the number of positive key points to the total number is used as the similarity.

[0101] Calculate the cosine similarity F3 between the gradient direction eigenvectors S and S':

[0102]

[0103] P represents the number of positive keypoints, and N represents the number of negative keypoints.

[0104] The final similarity F = 0.2 × F3 + 0.4 × F2 + 0.4 × F3;

[0105] When multiple feature information has been stored in the foreign object feature library / false detection feature library, the similarity calculation is performed on each feature information and the feature information corresponding to the suspected area to obtain multiple similarity F values.

[0106] 4) Continue with step 1) on the newly acquired images until all images have been detected for foreign objects.

[0107] To ensure the accuracy of the method, the identification results are reviewed during implementation. As the feature information in the foreign object feature library / false detection feature library expands and the accuracy improves, the review is stopped when the requirements are met. Specifically, step 4) may also include the following steps:

[0108] I. Review the foreign object area / false detection area. When a suspected area is found to be incorrectly marked, correct the marking type of the suspected area and store its corresponding feature information in the correct feature library.

[0109] II. When the number of foreign object areas / false detection areas reaches the preset value N, the recognition accuracy is calculated. If the accuracy meets the requirements, step I ends (indicating that the feature library can effectively identify foreign objects, and there is no need to review the marking results of suspected areas). Otherwise, continue with step I.

[0110] Where N≥100.

[0111] For example: When N=100, the results of marking the 1st to 100th suspected areas as foreign object areas / false detection areas are reviewed. The number of corrections, S, during the review period (1st to 100th areas) is counted. If the ratio of S to N exceeds the set value, the recognition accuracy meets the requirements, and step I ends (no further review is performed). Otherwise, the results of marking the 101st suspected area as a foreign object area / false detection area are reviewed, and the number of corrections, S, during the review period (2nd to 101st areas) is counted. If the ratio of S to N exceeds the set value, the recognition accuracy meets the requirements. Otherwise, the results of marking the 102nd suspected area as a foreign object area / false detection area are reviewed, and the number of corrections, S, during the review period (3rd to 102nd areas) is counted. If the ratio of S to N exceeds the set value, the recognition accuracy meets the requirements, and step I ends (no further review is performed). Otherwise, the results of marking the 103rd...

[0112] As the detection process progresses, the foreign object feature database and the false detection feature database will continuously store the feature information corresponding to the newly identified foreign object area and the false detection area, in preparation for the next identification.

[0113] As the feature information in the foreign object feature database and the false detection feature database continues to expand, the recognition accuracy will continue to rise and eventually stabilize.

[0114] The foregoing description of specific exemplary embodiments of the present invention is for illustrative and descriptive purposes. It is not intended to be exhaustive, nor to limit the invention to the precise forms disclosed; obviously, many changes and variations are possible in accordance with the foregoing teachings. The exemplary embodiments were chosen and described to explain the specific principles of the invention and its practical application, thereby enabling others skilled in the art to implement and utilize various exemplary embodiments of the invention, as well as their different alternatives and modifications. The scope of the invention is intended to be defined by the appended claims and their equivalents.

Claims

1. A method for detecting foreign objects in images based on feature library search, characterized in that, Includes the following steps: 1) Extract the region of interest from the acquired image; compare the region of interest with the standard image, identify the regions that differ from the standard image, and mark them as suspected regions; The standard image is: an image of the region of interest from a pre-stored normal image; 2) Extract feature information of the suspected region, wherein the feature information includes one or more of the following: gray-level co-occurrence vector, gradient direction feature vector, gray-level feature vector, key point coordinate set, and morphological feature vector; 3) Based on the feature information / area of ​​the suspected area, mark the suspected area as a foreign object area / false detection area, and store the feature information accordingly in the foreign object feature library / false detection feature library to complete the foreign object detection of the current image; 4) Continue with step 1) on the newly acquired images until all images have been detected for foreign objects.

2. The image foreign object detection method based on feature library search as described in claim 1, characterized in that: In step 3), the method of marking the suspected area as a foreign object area / false detection area based on the feature information / area of ​​the suspected area is as follows: based on the similarity between the feature information and the feature information already stored in the foreign object feature library / false detection feature library, or based on the size of the suspected area, the suspected area is identified as a foreign object area / false detection area.

3. The image foreign object detection method based on feature library search as described in claim 1 or 2, characterized in that: In step 3), the steps of marking the suspected area as a foreign object area / false detection area based on the feature information and storing the feature information accordingly in the foreign object feature library / false detection feature library are as follows: ① Determine whether the feature information has been stored in the foreign object feature database: If not, proceed directly to step ②; If so, retrieve the feature information of each element from the foreign object feature database, and calculate the similarity between each element and the feature information of the suspected region. Take the maximum similarity value, denoted as L. 异物 When L 异物 If the value exceeds a preset threshold I, the suspected area is marked as a foreign object area and the corresponding feature information is stored in the foreign object feature library, proceeding to step ③; otherwise, it continues to determine whether feature information has been stored in the false detection feature library. If not, proceed to step ②; if so, retrieve the feature information of each item in the false detection feature library and calculate the similarity between each item and the feature information of the suspected area, taking the maximum similarity value as L. 误检 ; Determine L 误检 If the value is greater than the threshold I, then the suspected area is recorded as a false detection area and the corresponding feature information is stored in the false detection feature library, and then step ③ is performed. If it is not greater than, proceed to step ②; ② Determine whether the area of ​​the suspected region is greater than the threshold. If it is, mark the suspected region as a foreign object region and store its corresponding feature information in the foreign object feature library. If it is not greater, mark the suspected region as a false detection region and store its corresponding feature information in the false detection feature library. ③ Complete foreign object detection in the current image.

4. The image foreign object detection method based on feature library search as described in claim 3, characterized in that: The feature information of each item in the foreign object feature database / false detection feature database is retrieved, and the similarity between each item and the feature information corresponding to the suspected area is calculated as follows: When the feature information contains one or more of gray-level co-occurrence vectors, gradient direction feature vectors, gray-level feature vectors, and morphological feature vectors, the similarity calculation is a cosine similarity calculation. When the feature information contains a set of key point coordinates, the similarity calculation method is as follows: calculate the Euclidean distance between each point coordinate in the set of key point coordinates corresponding to the suspected area and each point coordinate in the set of key point coordinates in the foreign object feature library / false detection feature library. If the distance value is less than the preset distance value, the point coordinates are marked as a positive key point; otherwise, they are marked as a negative key point. Count the total number of positive and negative key points, and use the ratio of the number of positive key points to the total number as the similarity. If the feature information contains multiple features such as gray-level co-occurrence vector, gradient direction feature vector, gray-level feature vector, morphological feature vector, and key point coordinate set, the resulting multiple similarities will be weighted and fused to obtain the final similarity.

5. The image foreign object detection method based on feature library search as described in claim 1, characterized in that: Step 4) also includes the following steps: I. Review the foreign object area / false detection area. When a suspected area is found to be incorrectly marked, correct the marking type of the suspected area and store its corresponding feature information in the correct feature library. II. When the number of foreign object areas / false detection areas reaches the preset value N, the recognition accuracy is calculated. If the accuracy meets the requirements, step I ends; otherwise, step I continues. Where N≥100.

6. The image foreign object detection method based on feature library search as described in claim 1, characterized in that: The set of key point coordinates includes the coordinates of the centroid of the suspected region and / or the coordinates of key points with scale invariance. The coordinates are relative coordinates with the corner of the smallest bounding rectangle of the suspected region as the starting point.

7. The image foreign object detection method based on feature library search as described in claim 1, characterized in that: The morphological feature vector includes multiple features such as the ratio of the suspected region area to the area of ​​the smallest circumscribed rectangle, the ratio of the suspected region area to the area of ​​the smallest circumscribed circle, the ratio of the suspected region area to the perimeter, and the directional features of the suspected region. Among them, the directional feature of the suspected region is the angle between the line segment from the centroid of the suspected region to the midpoint and the horizontal direction. The midpoint is: traversing any two edge points in the suspected region, taking the midpoint of the line connecting the two farthest edge points, and recording it as the midpoint.

8. The image foreign object detection method based on feature library search as described in claim 1, characterized in that: The gray-level co-occurrence vector includes energy features, contrast features, inverse difference moment features, entropy features, and autocorrelation features, which are derived from the gray-level co-occurrence matrix. The gradient direction feature vector is obtained by the following method: obtaining the gradient direction of all pixels in the suspected area, dividing 0° to 180° into multiple angle intervals according to a preset angle interval, and summing the frequency of the gradient direction of the pixel in each angle interval to form the gradient direction feature vector. The gray-level feature vector is a 256-dimensional vector, which is derived from the gray-level histogram.

9. The image foreign object detection method based on feature library search as described in claim 1, characterized in that: In step 1), the method for extracting the region of interest from the acquired image is as follows: Collect multiple images of the object under normal conditions, select the region of interest in each image and label it to form a sample dataset; use the sample dataset to train a deep learning model; In the actual detection process, the collected image of the object to be tested is input into the pre-trained deep learning model, and the model outputs the region of interest in the image and coarsely locates the region of interest. Then, template matching and image alignment are used to refine the coarsely located region of interest to obtain the region of interest.

10. The image foreign object detection method based on feature library search as described in claim 1, characterized in that: In step 1), the method for marking suspected areas is optical flow, background subtraction, or frame difference.

Images