A foreign object detection method and system based on visual recognition

Abstract

The present invention discloses a foreign object detection method and system based on visual recognition, comprising: marking a tool holder image; annotating a foreground identification area of ​​the tool holder image; photographing the tool holder using a camera, obtaining the original photo data byte[] and converting it into a Mat matrix detectionImage; preparing a tool holder image without foreign objects and converting it into a Mat matrix srcImage; setting upper and lower HSV spatial processing thresholds, performing HSV spatial processing on the Mat matrix detectionImage and the Mat matrix srcImage, and binarizing the image; performing black rectangle filling processing on the mold foreground photo Mat matrix detectionImageMask and the mold foreground original Mat matrix srcImageMask based on the marking according to DetectionTargetInfo data; performing a subtract operation on the processed mold foreground original Mat matrix srcImageMask and the mold foreground photo Mat matrix detectionImageMask; performing a contour closure operation on the mask matrix; performing an intersection operation on the closure contour Contours; and printing out the parameters and calculation results. The present invention can accurately detect foreign objects on the tool holder, and the algorithm is simple and efficient.

Description

Technical Field

[0001] The present invention relates to a visual detection method, and in particular to a foreign object detection method and system based on visual recognition. Background Art

[0002] In the existing technology, in order to standardize the use of tools, a tool management system is designed to monitor the borrowing and returning status and the in-place status of tools. Taking the aviation industry tool management system as an example, the tool usage status can be monitored through sensors, visual algorithms, etc. In actual applications, when foreign objects such as screws and nuts are left on the tool rack, it will bring safety hazards. However, the existing tool management system only has management functions such as in-place, absent, and misplaced, and lacks foreign object detection function, which cannot meet the application requirements of high safety and reliability. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a foreign object detection method and system based on visual recognition that can accurately detect foreign objects on tool racks, has a simple and efficient algorithm, and can improve the accuracy and reliability of tool management, in response to the shortcomings of the existing technology.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions.

[0005] A foreign object detection method based on visual recognition comprises the following steps: step S1, marking a tool holder image to obtain marking data, wherein the marking data includes the left margin left, the top margin top, the width width and the height height of the tool holder, and constructing marking basis DetectionTargetInfo data based on the marking data; step S2, marking the foreground recognition area of ​​the tool holder image to obtain detection area size information DetectionInfo; step S3, taking a photo of the tool holder with a camera, obtaining the original data byte[] of the photo and converting it into a Mat matrix detectionImage; step S4, preparing a tool holder image without foreign objects and converting it into a Mat matrix srcImage; step S5, setting the upper and lower limits of HSV spatial processing thresholds, performing HSV spatial processing on the Mat matrix detectionImage and the Mat matrix srcImage, and binarizing them to obtain Mat matrices detectionImageM of the mold foreground photo respectively. ask and the original Mat matrix srcImageMask of the mold foreground; step S6, based on the marking according to DetectionTargetInfo data, respectively perform black rectangle filling processing on the Mat matrix detectionImageMask of the mold foreground photo and the Mat matrix detectionImageMask of the mold foreground; step S7, perform subtract difference operation on the original Mat matrix srcImageMask of the mold foreground and the Mat matrix detectionImageMask of the mold foreground photo obtained by the step S5, and obtain the difference information mask matrix of the two foreground images; step S8, perform contour closure operation on the mask matrix in the step S7 to obtain the closure contour Contours; step S9, based on the detection area size information DetectionInfo obtained in the step S2, perform intersection operation on the closure contour Contours, and then filter out the noise outside the detection area; step S10, print out the parameters and calculation results.

[0006] Preferably, in step S5, the HSV space parameters include hue, saturation and brightness.

[0007] Preferably, in step S8, the area of ​​the closed contour is calculated, and contour noise points with an area smaller than 50 pixels are filtered out.

[0008] Preferably, in step S10, the parameters and calculation results include foreign object coordinates, foreign object outline size, and foreign object outline area.

[0009] A foreign object detection system based on visual recognition, wherein the system is used to execute the above method.

[0010] In the foreign object detection method based on visual recognition disclosed in the present invention, the EaeroQuealUnknownDetection algorithm is composed of steps S1 to S9. In actual applications, photos are obtained by taking pictures with a camera, and the photos are calculated based on the EaeroQuealUnknownDetection algorithm to identify foreign object information in the foreground. It can be effectively used in scenarios such as tool borrowing and returning, tool inventory, and various storage rack identification and detection scenarios. Compared with the existing technology, the foreign object detection method based on visual recognition of the present invention can detect whether there are foreign objects through visual recognition when applied to the detection scenario of the aviation industry tool management system. It is suitable for scenarios with engraving and the HSV space of the engraving foreground and the identified object can be distinguished. It can not only accurately detect foreign objects on the tool rack, but also the algorithm of the present invention is simple and efficient, which significantly improves the accuracy and reliability of tool borrowing and returning. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is the view after marking the tool holder image;

[0012] Figure 2 Picture of tool rack without foreign objects;

[0013] Figure 3 The detectionImageMask image is the Mat matrix of the foreground photo;

[0014] Figure 4 The original Mat matrix srcImageMask image for the foreground is engraved;

[0015] Figure 5 The image is the result of processing the Mat matrix detectionImageMask of the foreground photo of the engraving mold;

[0016] Figure 6 The image after processing the original Mat matrix srcImageMask of the engraved foreground;

[0017] Figure 7 The mask matrix image of the difference information between the two foreground images;

[0018] Figure 8 It is the closure collection Contours image;

[0019] Figure 9 This is the test result picture. DETAILED DESCRIPTION

[0020] The present invention will be described in more detail below with reference to the accompanying drawings and embodiments.

[0021] The present invention discloses a foreign object detection method based on visual recognition, which comprises the following steps:

[0022] Step S1, marking the tool rack position image to obtain marking data, the marking data includes the tool rack position ( Figure 1 The left margin left, top margin top, width width and height height of the box around the tool are constructed based on the marking data.

[0023] Step S2: annotate the foreground recognition area of ​​the tool holder image to obtain detection area size information DetectionInfo; the foreground refers to the image including the tool surface and the die surface, which is mainly used for subsequent determination of the recognition range of foreign objects.

[0024] Step S3, use the camera to take a photo of the tool holder, obtain the original data byte[] of the photo and convert it into Mat matrix detectionImage;

[0025] Step S4, prepare a tool stand image without foreign objects and convert it into a Mat matrix srcImage;

[0026] Step S5, setting the upper and lower limits of the HSV space processing threshold, performing HSV space processing on the Mat matrix detectionImage and the Mat matrix srcImage, and binarizing them to obtain the mold foreground photo Mat matrix detectionImageMask and the mold foreground original Mat matrix srcImageMask respectively;

[0027] Step S6, performing black rectangle filling processing on the mold foreground photo Mat matrix detectionImageMask and the mold foreground original Mat matrix srcImageMask based on the marking according to DetectionTargetInfo data;

[0028] Step S7, performing a subtract operation on the original Mat matrix srcImageMask of the foreground of the mold obtained by the processing in step S5 and the Mat matrix detectionImageMask of the foreground of the mold, to obtain a mask matrix of difference information of the two foreground images;

[0029] Step S8, performing a contour closure operation on the mask matrix in step S7 to obtain a closed contour Contours;

[0030] Step S9, based on the detection area size information DetectionInfo obtained in step S2, performing an intersection operation on the closure contour Contours, thereby filtering out noise points outside the detection area;

[0031] Step S10: Print out the parameters and calculation results.

[0032] In the above method, steps S1 to S9 constitute the EaeroQuealUnknownDetection algorithm. In actual applications, photos are obtained by taking pictures with a camera, and the photos are calculated based on the EaeroQuealUnknownDetection algorithm to identify foreign object information in the foreground. It can be effectively used in scenarios such as tool borrowing and returning, tool inventory, and various storage rack identification and detection scenarios. Compared with the existing technology, the foreign object detection method based on visual recognition of the present invention can detect the presence of foreign objects through visual recognition when applied to the detection scenario of the aviation industry tool management system. It is suitable for scenes with engraving and the engraving foreground and the identified object are distinguishable in HSV space. It can not only accurately detect foreign objects on the tool rack, but also the algorithm of the present invention is simple and efficient, which significantly improves the accuracy and reliability of tool borrowing and returning management.

[0033] As a preferred embodiment, in step S5, the HSV space parameters include hue, saturation, and brightness. Lighting has a significant impact on the RGB color gamut, and the algorithm is less affected by lighting because the weight of RGB hue is relatively low when processing in the HSV space.

[0034] The present invention also has a noise filtering step, specifically, in step S8, the area of ​​the closed contour is calculated, and contour noise points with an area smaller than 50 pixels are filtered out.

[0035] Regarding the result output, in step S10 , the parameters and calculation results include the coordinates of the foreign object, the size of the foreign object outline, and the area of ​​the foreign object outline.

[0036] On this basis, the present invention also proposes a foreign object detection system based on visual recognition, which is used to execute the above-mentioned method.

[0037] A specific embodiment is provided below for reference.

[0038] In the processing steps of this embodiment, a photo is obtained by taking a photo with a camera, and the EaeroQuealUnknownDetection algorithm is used to calculate the photo, so as to identify the information of foreign objects in the foreground, which can be used for subsequent business applications such as tool borrowing and returning, tool inventory, and other scenarios.

[0039] Specifically, the EaeroQuealUnknownDetection algorithm process includes:

[0040] Step 1: Mark the image to obtain the tool rack information that needs to be identified, such as Figure 1 As shown, the marking data mainly includes the left margin left, top margin top, width width, and height height of the tool holder status. These data components are used for marking based on the DetectionTargetInfo data.

[0041] Step 2: Mark the foreground recognition area of ​​the image and obtain the size information of the detection area DetectionInfo, which includes the left margin left, top margin top, width width, and height height;

[0042] Step 3: Use the camera to take a picture of the tool holder, obtain the original data byte[] of the picture and convert it into Mat matrix detectionImage to prepare for the subsequent processing of the algorithm;

[0043] Step 4: Prepare a picture without foreign objects and convert it into a Mat matrix srcImage for use in subsequent picture calculations, such as Figure 2 As shown;

[0044] Step 5, set the upper and lower limits of the HSV space processing threshold, perform HSV space processing on the Mat matrix detectionImage and the Mat matrix srcImage, and binarize them, and finally obtain the mold foreground photo Mat matrix detectionImageMask and the mold foreground original Mat matrix srcImageMask respectively;

[0045] Among them, HSV space mainly includes hue, saturation, and brightness. Lighting has a significant impact on the RGB color gamut. However, the algorithm is less affected by light because the weight of RGB hue is relatively low when processing in HSV space.

[0046] Among them, the foreground photo Mat matrix detectionImageMask is as follows Figure 3 As shown, the original Mat matrix srcImageMask of the engraved foreground is as follows Figure 4 As shown;

[0047] Step 6, by marking the detectionImageMask and srcImageMask matrices according to the DetectionTargetInfo data, respectively, the black rectangle filling process is performed. The main purpose of this step is to divide the foreground recognition area and obtain an accurate foreground detection area. The effect after processing is as follows: Figure 5 and Figure 6 As shown;

[0048] Step 7, use the srcImageMask matrix processed in step 5 and the detectionImageMask matrix to perform subtract difference operation, so that the information mask matrix of the difference between the two foreground images can be obtained, such as Figure 7 As shown;

[0049] Step 8: Perform contour closure operation on the mask matrix in step 7, find the closure set Contours in the figure, and calculate the area of ​​the closure contour, and filter out the contour noise with an area of ​​less than 50 pixels. The Contours set is as follows: Figure 8 As shown;

[0050] Step 9, using the DetectionInfo information in step 2, intersect the closed contour Contours in step 8. If they intersect, it means that the recognized contour is within the detection range of the DetectionInfo information in step 2. This step can filter out noise points outside the detection DetectionInfo area, and the final result is as follows: Figure 9 As shown;

[0051] Step 10, output results: print out relevant parameters and calculation results, including foreign object coordinates, foreign object outline size, and foreign object outline area.

[0052] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements or improvements made within the technical scope of the present invention should be included in the scope of protection of the present invention.

Claims

1. A foreign object detection method based on visual recognition, characterized in that: The steps include: Step S1: Mark the tool holder image to obtain marking data, wherein the marking data includes the left margin left, top margin top, width width and height height of the tool holder, and construct the marking basis DetectionTargetInfo data based on the marking data; Step S2: marking the foreground recognition area of ​​the tool stand image to obtain detection area size information DetectionInfo; Step S3, use the camera to take a photo of the tool holder, obtain the original data byte[] of the photo and convert it into Mat matrix detectionImage; Step S4, prepare a tool stand image without foreign objects and convert it into a Mat matrix srcImage; Step S5, setting the upper and lower limits of the HSV space processing threshold, performing HSV space processing on the Mat matrix detectionImage and the Mat matrix srcImage, and binarizing them to obtain the mold foreground photo Mat matrix detectionImageMask and the mold foreground original Mat matrix srcImageMask respectively; Step S6, performing black rectangle filling processing on the mold foreground photo Mat matrix detectionImageMask and the mold foreground original Mat matrix srcImageMask based on the marking according to DetectionTargetInfo data; Step S7, performing a subtract operation on the original Mat matrix srcImageMask of the foreground of the mold obtained by the processing in step S5 and the Mat matrix detectionImageMask of the foreground of the mold, to obtain a mask matrix of difference information of the two foreground images; Step S8, performing a contour closure operation on the mask matrix in step S7 to obtain a closed contour Contours; Step S9, based on the detection area size information DetectionInfo obtained in step S2, performing an intersection operation on the closure contour Contours, thereby filtering out noise points outside the detection area; Step S10: Print out the parameters and calculation results.

2. The foreign object detection method based on visual recognition according to claim 1, characterized in that: In step S5, the HSV space parameters include hue, saturation and brightness.

3. The foreign object detection method based on visual recognition according to claim 1, characterized in that: In step S8, the area of ​​the closed contour is calculated, and contour noise points with an area smaller than 50 pixels are filtered out.

4. The foreign object detection method based on visual recognition according to claim 1, characterized in that: In step S10 , the parameters and calculation results include the coordinates of the foreign object, the size of the foreign object outline, and the area of ​​the foreign object outline.

5. A foreign object detection system based on visual recognition, characterized in that: The system is used to execute the method according to any one of claims 1 to 4.

Images