Foreign object detection method and system based on visual identification

Abstract

The invention discloses a visual identification-based foreign object detection method and system. The method comprises the following steps of: marking a tool rack picture; labeling a foreground identification area of the tool rack position picture; the method comprises the following steps of: taking a picture of a tool rack by using a camera, acquiring original data byte [] of the picture, and converting the original data byte [] into a Mat matrix desection Image; preparing a tool rack picture without a foreign object and converting the tool rack picture into a Mat matrix srcImage; setting an upper limit and a lower limit of an HSV space processing threshold value, carrying out HSV space processing on a Mat matrix dejectionImage and a Mat matrix srcImage, and carrying out binarization processing on the Mat matrix dejectionImage and the Mat matrix srcImage; based on marking, according to the Desection TargetInfo data, black rectangle filling processing is carried out on the Mat matrix desection ImageMask of the die sinking foreground picture and the Mat matrix srcImageMask of the die sinking foreground original picture; subtrace difference set calculation is carried out on the original Mat matrix srcImage Mask of the die sinking foreground and the Mat matrix desection Image Mask of the die sinking foreground picture, which are obtained through processing, and the subtrace difference set calculation is carried out on the original Mat matrix srcImage Mask of the die sinking foreground and the Mat matrix desection Image Mask of the die sinking foreground; carrying out contour closure calculation on the mask matrix; intersection calculation is carried out on the closure contour Contours; and printing and outputting the parameters and the calculation result. The foreign objects on the tool rack can be accurately detected, and the algorithm is simple and efficient.

Description

Technical Field

[0001] The present invention relates to a visual detection method, and in particular to a foreign object detection method and system based on visual recognition. Background Art

[0002] In the prior art, in order to standardize the use of tools, a tool management system is designed to supervise the borrowing and returning status and the in-position status of tools. Taking the tool management system in the aviation industry as an example, the use status of tools can be supervised by means of sensors, visual algorithms, etc. In actual applications, when foreign objects such as screws and nuts are left on the tool rack, it will pose a safety hazard. However, the existing tool management systems only have management functions such as in-position, out-of-position, and misplacement, lacking the function of foreign object detection and unable to meet the application requirements of high safety and reliability. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a foreign object detection method and system based on visual recognition that can accurately detect foreign objects on the tool rack, has a simple and efficient algorithm, and can improve the accuracy and reliability of tool management, aiming at the deficiencies of the prior art.

[0004] To solve the above technical problems, the present invention adopts the following technical solutions.

[0005] A foreign object detection method based on visual recognition, which includes the following steps: Step S1, label the tool rack position picture to obtain labeled data. The labeled data includes the left margin left, top margin top, width width, and height height of the tool rack position. Based on the labeled data, construct the labeled basis DetectionTargetInfo data; Step S2, label the foreground recognition area of the tool rack position picture to obtain the detection area size information DetectionInfo; Step S3, use a camera to take a picture of the tool rack position, obtain the original photo data byte[] and convert it into a Mat matrix detectionImage; Step S4, prepare a tool rack position picture without foreign objects and convert it into a Mat matrix srcImage; Step S5, set the upper and lower limits of the HSV space processing threshold, perform HSV space processing on the Mat matrix detectionImage and the Mat matrix srcImage, and perform binary processing to obtain the engraved foreground photo Mat matrix detectionImageMask and the engraved foreground original Mat matrix srcImageMask respectively; Step S6, based on the labeled basis DetectionTargetInfo data, perform black rectangle filling processing on the engraved foreground photo Mat matrix detectionImageMask and the engraved foreground original Mat matrix srcImageMask respectively; Step S7, perform a subtract difference set operation on the engraved foreground original Mat matrix srcImageMask and the engraved foreground photo Mat matrix detectionImageMask obtained in Step S5 to obtain the difference information mask matrix of the foreground parts of the two pictures; Step S8, perform a contour closure operation on the mask matrix in Step S7 to obtain the closed contour Contours; Step S9, based on the detection area size information DetectionInfo obtained in Step S2, perform an intersection operation on the closed contour Contours, and then filter out the noise points outside the detection area; Step S10, print and output the parameters and calculation results.

[0006] Preferably, in Step S5, the HSV space parameters include hue, saturation, and brightness.

[0007] Preferably, in Step S8, calculate the area of the closed contour, and filter out the contour noise points with an area less than 50 pixels.

[0008] Preferably, in Step S10, the parameters and calculation results include the coordinates of foreign objects, the size of the foreign object contour, and the area of the foreign object contour.

[0009] A foreign object detection system based on visual recognition, which is used to execute the method described above.

[0010] In the foreign object detection method based on visual recognition disclosed in the present invention, the EaeroQuealUnknownDetection algorithm is composed of steps S1 to S9. In practical applications, a photo is obtained by taking a picture with a camera, and the photo is calculated based on the EaeroQuealUnknownDetection algorithm, so as to identify the foreign object information in the foreground, and it can be effectively used in scenarios such as tool borrowing and returning, tool inventory, and the identification and detection scenarios of various storage shelves. Compared with the prior art, the foreign object detection method based on visual recognition in the present invention can detect whether there are foreign objects through visual recognition when applied to the detection scenario of the tool management system in the aviation industry, and is applicable to scenarios with engraving molds and where the engraving mold foreground and the object to be recognized are distinguishable in the HSV space. It can not only accurately detect foreign objects on the tool shelf, but also the algorithm of the present invention is simple and efficient, significantly improving the accuracy and reliability of tool borrowing and returning. Description of the Drawings

[0011] Figure 1 It is a view after marking the tool shelf picture;

[0012] Figure 2 It is a picture of a tool shelf without foreign objects;

[0013] Figure 3 It is the Mat matrix detectionImageMask picture of the engraving mold foreground photo;

[0014] Figure 4 It is the original Mat matrix srcImageMask picture of the engraving mold foreground;

[0015] Figure 5 It is a picture after processing the Mat matrix detectionImageMask of the engraving mold foreground photo;

[0016] Figure 6 It is a picture after processing the original Mat matrix srcImageMask of the engraving mold foreground;

[0017] Figure 7 It is the mask matrix picture of the difference information between two pictures in the foreground part;

[0018] Figure 8 It is the Contours picture of the closure set;

[0019] Figure 9 It is the detection result picture. Detailed Embodiments

[0020] The present invention will be described in more detail below with reference to the accompanying drawings and embodiments.

[0021] The present invention discloses a foreign object detection method based on visual recognition, which includes the following steps:

[0022] Step S1, label the tool rack position picture to obtain labeling data. The labeling data includes the left margin left, top margin top, width width, and height height of the square around the tool in the tool rack position ( Figure 1 Construct the labeling basis DetectionTargetInfo data based on the labeling data;

[0023] Step S2, label the foreground recognition area of the tool rack position picture to obtain the detection area size information DetectionInfo; the foreground refers to the image including the tool surface and the mold surface, which is mainly used to judge the foreign object recognition range in the subsequent process.

[0024] Step S3, use a camera to take a picture of the tool rack position, obtain the original photo data byte[] and convert it into a Mat matrix detectionImage;

[0025] Step S4, prepare a tool rack position picture without foreign objects and convert it into a Mat matrix srcImage;

[0026] Step S5, set the upper and lower limits of the HSV space processing threshold, perform HSV space processing on the Mat matrix detectionImage and the Mat matrix srcImage, and perform binary processing to obtain the mold foreground photo Mat matrix detectionImageMask and the mold foreground original Mat matrix srcImageMask respectively;

[0027] Step S6, perform black rectangle filling processing on the mold foreground photo Mat matrix detectionImageMask and the mold foreground original Mat matrix srcImageMask respectively based on the labeling basis DetectionTargetInfo data;

[0028] Step S7, perform a subtract difference set operation on the mold foreground original Mat matrix srcImageMask and the mold foreground photo Mat matrix detectionImageMask obtained in Step S5 to obtain the difference information mask matrix of the two pictures in the foreground part;

[0029] Step S8, perform a contour closure operation on the mask matrix in Step S7 to obtain the closure contour Contours;

[0030] Step S9: Based on the detection area size information DetectionInfo obtained in Step S2, perform an intersection operation on the closure contours Contours, thereby filtering out noise points outside the detection area.

[0031] Step S10: Print and output the parameters and calculation results.

[0032] In the above method, the EaeroQuealUnknownDetection algorithm consists of Steps S1 to S9. In practical applications, a photo is obtained by taking a picture with a camera, and the photo is calculated based on the EaeroQuealUnknownDetection algorithm to identify the foreign object information in the foreground, and it can be effectively used in scenarios such as tool borrowing and returning, tool inventory, and the identification and detection scenarios of various storage shelves. Compared with the prior art, the foreign object detection method based on visual recognition of the present invention can detect whether there are foreign objects through visual recognition when applied to the detection scenario of the tool management system in the aviation industry, and is applicable to scenarios where there are engraving patterns and the HSV space of the engraving pattern foreground and the object to be recognized can be distinguished. It can not only accurately detect foreign objects on the tool rack, but also the algorithm of the present invention is simple and efficient, significantly improving the accuracy and reliability of tool borrowing and returning management.

[0033] As a preferred method, in Step S5, the HSV space parameters include hue, saturation, and brightness. Among them, the influence range of light on the RGB color gamut is obvious, and the weight of the RGB hue is relatively low when the algorithm processes in the HSV space, so it is less affected by light.

[0034] The present invention also has a noise filtering step, specifically, in Step S8, calculate the area of the closure contour, and filter out contour noise points with an area less than 50 pixels.

[0035] Regarding the result output, in Step S10, the parameters and calculation results include the coordinates of the foreign object, the size of the foreign object contour, and the area of the foreign object contour.

[0036] On this basis, the present invention also proposes a foreign object detection system based on visual recognition, and the system is used to execute the above-mentioned method.

[0037] A specific embodiment is provided below for reference.

[0038] In the processing steps of this embodiment, a photo is obtained by taking a picture with a camera, and the photo is calculated using the EaeroQuealUnknownDetection algorithm to identify the foreign object information in the foreground, which is used for applications in subsequent business scenarios such as tool borrowing and returning, tool inventory, etc.

[0039] Specifically, the process of the EaeroQuealUnknownDetection algorithm includes:

[0040] Step 1: Mark the picture to obtain the tool rack position information to be recognized. As Figure 1 shown, the marking data mainly includes the left margin left, top margin top, width width, and height height of the tool rack position status. Based on these data, the marking basis DetectionTargetInfo data is constructed;

[0041] Step 2: Annotate the foreground recognition area of the picture to obtain the size information DetectionInfo of the detection area, which includes the left margin left, top margin top, width width, and height height;

[0042] Step 3: Use the camera to take a picture of the tool rack, obtain the original photo data byte[] and convert it into a Mat matrix detectionImage to prepare for the subsequent processing of the algorithm;

[0043] Step 4: Prepare a picture without foreign objects and convert it into a Mat matrix srcImage for use in subsequent picture calculations. As Figure 2 shown;

[0044] Step 5: Set the upper and lower limits of the HSV space processing threshold, perform HSV space processing on the Mat matrix detectionImage and the Mat matrix srcImage, and binarize them to finally obtain the engraved mold foreground photo Mat matrix detectionImageMask and the engraved mold foreground original Mat matrix srcImageMask respectively;

[0045] Among them, the HSV space mainly includes hue, saturation, and brightness. The influence range of light on the RGB color gamut is obvious, and the weight of the RGB hue is relatively low when the algorithm processes in the HSV space, so it is less affected by light;

[0046] Among them, the engraved mold foreground photo Mat matrix detectionImageMask is as Figure 3 shown, and the engraved mold foreground original Mat matrix srcImageMask is as Figure 4 shown;

[0047] Step 6: Based on the DetectionTargetInfo data for marking, perform black rectangle filling on the detectionImageMask and srcImageMask matrices respectively. The main purpose of this step is to divide the foreground recognition area and obtain an accurate foreground detection area. The processing effect is as shown in Figure 5 and Figure 6 shown;

[0048] Step 7: Use the srcImageMask matrix and the detectionImageMask matrix processed in Step 5 to perform a subtract operation to find the difference set, so as to obtain the mask matrix of the difference information between the two foreground images, as shown in Figure 7 shown;

[0049] Step 8: Perform a closed contour operation on the mask matrix in Step 7 to find the closed contour set Contours in the image. At the same time, calculate the area of the closed contour and filter out the contour noise points with an area less than 50 pixels. The Contours set is as shown in Figure 8 shown;

[0050] Step 9: Use the DetectionInfo information in Step 2 to perform an intersection operation on the closed contour Contours in Step 8. If there is an intersection, it means that the recognized contour is within the detection range of the DetectionInfo information in Step 2. This step can filter out the noise points outside the DetectionInfo area to obtain the final result as shown in Figure 9 shown;

[0051] Step 10: Output the results: Print and output the relevant parameters and calculation results, including the coordinates of the foreign object, the size of the foreign object contour, and the area of the foreign object contour.

[0052] The above is only a preferred embodiment of the present invention and is not used to limit the present invention. Any modifications, equivalent replacements, or improvements made within the technical scope of the present invention shall be included within the scope protected by the present invention.

Claims

1. A foreign object detection method based on visual recognition, characterized in that: The steps include: Step S1, marking the tool rack image to obtain marking data, wherein the marking data includes the left margin left, the top margin top, the width width and the height height of the tool rack, and constructing the marking basis DetectionTargetInfo data based on the marking data; Step S2, marking the foreground recognition area of ​​the tool stand image to obtain detection area size information DetectionInfo; Step S3, using a camera to take a photo of the tool stand, obtaining the original photo data byte[] and converting it into a Mat matrix detectionImage; Step S4, prepare a tool stand image without foreign objects and convert it into a Mat matrix srcImage; Step S5, setting the upper and lower limits of the HSV space processing threshold, performing HSV space processing on the Mat matrix detectionImage and the Mat matrix srcImage, and binarizing them, to obtain the mold foreground photo Mat matrix detectionImageMask and the mold foreground original Mat matrix srcImageMask respectively; Step S6, performing black rectangle filling processing on the foreground photo Mat matrix detectionImageMask of the engraving mold and the original foreground Mat matrix srcImageMask of the engraving mold based on the marking DetectionTargetInfo data; Step S7, performing a subtract operation on the original Mat matrix srcImageMask of the foreground of the engraving mold obtained by processing in step S5 and the Mat matrix detectionImageMask of the foreground of the engraving mold, so as to obtain a mask matrix of difference information of the two foreground images; Step S8, performing contour closure operation on the mask matrix in step S7 to obtain closure contours Contours; Step S9, based on the detection area size information DetectionInfo obtained in step S2, performing intersection operation on the closure contour Contours, thereby filtering out noise points outside the detection area; Step S10, printing out the parameters and calculation results.

2. The foreign object detection method based on visual recognition according to claim 1, characterized in that: In step S5, the HSV space parameters include hue, saturation and brightness.

3. The foreign object detection method based on visual recognition according to claim 1, characterized in that: In step S8, the area of ​​the closed contour is calculated, and contour noise points with an area less than 50 pixels are filtered out.

4. The foreign object detection method based on visual recognition according to claim 1, characterized in that: In the step S10, the parameters and calculation results include the coordinates of the foreign object, the size of the foreign object outline and the area of ​​the foreign object outline.

5. A foreign object detection system based on visual recognition, characterized in that: The system is used to execute the method according to any one of claims 1 to 4.

Images