Foreign matter detection method, device and equipment and storage medium
By acquiring multi-frame images when the transparent container rotates, analyzing the position changes of target defects in adjacent images, the problem of difficult to distinguish between container surface defects and foreign objects inside high viscosity products in the prior art, and achieving high-precision foreign object detection.
Patent Information
- Application Number
- CN202510251087.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-04
- Publication Date
- 2025-08-01
AI Technical Summary
The prior art is difficult to distinguish the surface defects of transparent containers from foreign matter inside high viscosity products, resulting in high false alarm rates.
By collecting multi-frame images when the transparent container rotates, the target defect position information of two adjacent frames of images is extracted, the position of the defect in the next frame of image is predicted, and the foreign matter inside the container and the outer surface defect are distinguished by area overlap analysis.
Effectively distinguish between foreign objects inside the container and external surface defects, improve detection accuracy and reduce false alarm rate.
Smart Images
Figure CN120404730A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a foreign matter detection method, device, equipment and storage medium. Background Art
[0002] Factories producing bottled beverages, pharmaceuticals, and medical aesthetics have very high requirements for the cleanliness of the liquids canned in containers. This is especially true during the production of liquid pharmaceuticals, where transparent glass containers are inspected for foreign matter. Detection of foreign matter, such as hair or fiber, triggers an alarm in the production system. However, in actual production, the outer surface of the transparent glass containers used for canned products is prone to defects such as scratches, discoloration, and dust during transportation. Therefore, when inspecting the container for foreign matter, it is easy to mistake surface damage for internal foreign matter, leading to false alarms in the production system.
[0003] In the related art, a rotation emergency stop detection method is provided, namely: placing a glass container on a rotating device, allowing the container to rotate rapidly and then stop the container suddenly. In this way, the product inside the container will produce relative displacement with the container due to inertia, while damage to the outer surface of the container will not produce relative displacement with the container itself. Therefore, foreign matter in the product contained in the container can be detected based on the relative displacement.
[0004] However, existing methods can only be used for products with high fluidity inside the container. For highly viscous and non-fluid products (such as gel-like products), they cannot produce obvious relative displacement with the container itself after a sudden stop after high-speed rotation. Therefore, it is difficult to distinguish the internal foreign matter of the canned product in the container from the damage defect characteristics of the container's outer surface, resulting in a high false alarm rate of the system. Summary of the Invention
[0005] Embodiments of the present invention provide a foreign matter detection method, apparatus, device, and storage medium to solve the problem in the prior art of difficulty in distinguishing between surface defects of transparent containers and foreign matter defects inside highly viscous products, resulting in a high false alarm rate.
[0006] In a first aspect, an embodiment of the present invention provides a foreign body detection method, comprising:
[0007] Collect multiple frames of images of the transparent container when it rotates to obtain a set of images to be detected;
[0008] Randomly selecting two adjacent frames of images from the set of images to be detected as a first image to be detected and a second image to be detected, and the second image to be detected is a frame of image subsequent to the first image to be detected;
[0009] Extracting first position information of a target defect in a first image to be inspected;
[0010] Predict the position information of the target defect in the second image to be detected according to the first position information of the target defect, and obtain the predicted position information.
[0011] Extract the second position information of the target defect in the second image to be detected.
[0012] Determine the area overlap degree between the predicted position information and the second position information. If the area overlap degree is less than a preset threshold, determine that the target defect is a foreign object defect of the internal product in the transparent container.
[0013] In a possible implementation manner, the extracting the first position information of the target defect in the first image to be detected includes:
[0014] In the first image to be detected, extract the target area where the transparent container is located.
[0015] Based on the threshold segmentation technology, extract the features of the target defect in the target area and generate the first minimum circumscribed rectangle of the target defect.
[0016] Determine the vertex coordinate information of the first minimum circumscribed rectangle as the first position information of the target defect.
[0017] In a possible implementation manner, the predicting the position information of the target defect in the second image to be detected according to the first position information of the target defect and obtaining the predicted position information includes:
[0018] Obtain the rotation radius r of the transparent container and the rotation angle T of the transparent container from when the first image to be detected is collected to when the second image to be detected is collected.
[0019] According to the vertex coordinate information of the first minimum circumscribed rectangle, the rotation radius r, the rotation angle T, the first formula and the second formula, determine the vertex coordinates of the predicted minimum circumscribed rectangle of the target defect in the second detected image.
[0020] Determine the vertex coordinate information of the predicted minimum circumscribed rectangle as the predicted position information.
[0021] Wherein, the first formula is: Xrotate = (x - r)cosT - (y - r)sinT, and the second formula is: Yrotate = (x - r)sinT + (y - r)cosT.
[0022] Wherein, x and y respectively represent the abscissa and ordinate of the vertex of the first minimum bounding rectangle, and Xrotate and Yrotate respectively represent the abscissa and ordinate of the vertex of the predicted minimum bounding rectangle.
[0023] In a possible implementation, the second position information includes the vertex coordinate information of the actual second minimum bounding rectangle of the target defect in the second image to be detected.
[0024] Determining the area overlap degree between the predicted position information and the second position information includes:
[0025] According to the vertex coordinate information of the predicted minimum bounding rectangle and the vertex coordinate information of the second minimum bounding rectangle, determine the overlapping area and non-overlapping area between the predicted minimum bounding rectangle and the second minimum bounding rectangle.
[0026] Determine the area overlap degree according to the overlapping area and the non-overlapping area.
[0027] In a possible implementation, the extracting the features of the target defect in the target area and generating the first minimum bounding rectangle of the target defect based on the threshold segmentation technology includes:
[0028] Perform gray-scale processing on the target area to obtain a gray-scale image of the target area.
[0029] Compare the gray value of each pixel point in the gray-scale image with a preset segmentation threshold, and mark the pixel points with gray values greater than the preset segmentation threshold as the pixel points of the target defect to obtain a binary image of the target defect.
[0030] Perform contour detection on the binary image of the target defect to obtain the boundary information of the target defect.
[0031] Generate the first minimum bounding rectangle of the target defect according to the boundary information.
[0032] In a possible implementation, the collecting multiple frames of images when the transparent container rotates to obtain a set of images to be detected includes:
[0033] When the transparent container rotates around its own central axis, capture a front view of the transparent container every time it rotates a preset target angle.
[0034] Determine all the front views captured during the process of rotating the transparent container 360 degrees as the set of images to be detected.
[0035] In a possible implementation, the method further includes:
[0036] If, among the images to be detected, the area overlap degree corresponding to at least M images to be detected is less than the preset threshold, it is determined that the target defect is a foreign object defect of the internal product in the transparent container;
[0037] wherein, M is equal to 3, and the preset threshold is 80%.
[0038] In a second aspect, an embodiment of the present invention provides a foreign object detection device, which is characterized by including:
[0039] An image acquisition module, configured to acquire multiple frames of images when the transparent container rotates, and obtain a set of images to be detected;
[0040] An image extraction module, configured to arbitrarily select two adjacent images from the set of images to be detected as a first image to be detected and a second image to be detected respectively, and the second image to be detected is the next frame image of the first image to be detected;
[0041] A first information extraction module, configured to extract first position information of a target defect in the first image to be detected;
[0042] A position prediction module, configured to predict the position information of the target defect in the second image to be detected according to the first position information of the target defect, and obtain predicted position information;
[0043] A second information extraction module, configured to extract second position information of the target defect in the second image to be detected;
[0044] A foreign object detection module, configured to determine the area overlap degree between the predicted position information and the second position information. If the area overlap degree is less than the preset threshold, it is determined that the target defect is a foreign object defect of the internal product in the transparent container.
[0045] In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the method in the first aspect or any possible implementation manner of the first aspect above is implemented.
[0046] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method in the first aspect or any possible implementation manner of the first aspect above is implemented.
[0047] In a fifth aspect, an embodiment of the present invention provides a computer program product, including a computer program. When the computer program is executed by a processor, the method in the first aspect or any possible implementation manner of the first aspect above is implemented.
[0048] In the embodiments of the present invention, a set of images to be detected is obtained by collecting multiple frames of images when a transparent container rotates; the first position information of the target defect in the first image to be detected among two adjacent frames of images is extracted, and then the position information of the target defect in the next frame of image (i.e., the second image to be detected) is predicted. The area coincidence degree analysis is performed on the predicted position information and the actual position information (i.e., the second position information) of the target defect in the second image to be detected; if the target defect is a damage defect on the outer surface of the container, the predicted position and the actual position are consistent, that is, the two position areas will completely coincide; if the target defect is a foreign object defect of the internal product, due to the different rotation radii of the foreign object defect and the defect on the outer surface of the container when the container rotates, there will be a deviation between the predicted position and the actual position, that is, the two positions will not completely coincide in the second detection image; therefore, by analyzing the area overlap degree of the predicted position information and the actual position information, if the area overlap degree is lower than a preset threshold, it indicates that the deviation between the predicted position and the actual position is large, so as to determine that the target defect is a foreign object defect of the internal product of the container. This method can effectively distinguish the foreign object defect inside the container and the defect on the outer surface for highly viscous and non-flowing products, improve the detection accuracy of foreign objects in highly viscous products inside the container, and reduce the false alarm rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 FIG. is an application scenario diagram of the foreign object detection method provided by the embodiment of the present invention;
[0050] Figure 2 FIG. is a flowchart of the implementation of the foreign object detection method provided by the embodiment of the present invention;
[0051] Figure 3 FIG. is a container rotation projection diagram provided by the embodiment of the present invention;
[0052] Figure 4 FIG. is an application scenario diagram of the foreign object detection method provided by the embodiment of the present invention;
[0053] Figure 5 FIG. is a flowchart of the implementation of the foreign object detection method provided by the embodiment of the present invention;
[0054] Figure 6 FIG. is a flowchart of the implementation of the foreign object detection method provided by the embodiment of the present invention;
[0055] Figure 7 FIG. is a schematic structural diagram of the foreign object detection device provided by the embodiment of the present invention;
[0056] Figure 8 FIG. is a schematic diagram of an electronic device provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0057] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0058] Figure 1 This is an application scenario diagram of the foreign object detection method provided by the embodiments of the present invention.
[0059] As Figure 1 shown, the application scenario provided in this embodiment includes a transparent container 101, an image acquisition device 102, and an image processing device 103. Among them, the image acquisition device 102 acquires a front view of the transparent container when the transparent container rotates, and then transmits the acquired image to the image processing device 103 for analysis. When a foreign object is detected in the product inside the transparent container, an alarm message is sent to the terminal device 104.
[0060] It should be noted that the image acquisition device 102 in this application scenario can be, but is not limited to, any device that can capture images, such as a camera, a mobile phone, a tablet, etc.; the image processing device 103 can be, but is not limited to, devices such as a server and a computer, and the terminal device 104 can be, but is not limited to, devices such as a mobile phone and a tablet.
[0061] In a possible implementation manner, the rotation of the transparent container can be realized by placing the transparent container on a rotating table and controlling the rotation of the transparent container through the rotating table.
[0062] It can be understood that the application scenario provided in this embodiment is only used for exemplary purposes to facilitate relevant personnel to understand the method provided by the present invention, and does not constitute any limitation to the present invention.
[0063] The method provided by the present invention will be elaborated in detail below in combination with the application scenario.
[0064] See Figure 2 , which shows the implementation flowchart of the foreign object detection method provided by the embodiments of the present invention. The detailed steps are as follows:
[0065] S201, acquire multiple frames of images when the transparent container rotates to obtain a set of images to be detected.
[0066] In this step, the transparent container can be, but is not limited to, a transparent glass container, and the transparent glass container can be, but is not limited to, a cylindrical shape.
[0067] In a possible implementation manner, referring to Figure 1 , the transparent container is placed on a rotating table, and the transparent container rotates around its own central axis through the rotating table. Every time the container rotates a certain angle, the camera acquires one frame of image in the front of the transparent container, and finally obtains several front views of the transparent container, constituting a set of images to be detected.
[0068] S202. Arbitrarily select two adjacent images from the set of images to be detected as the first image to be detected and the second image to be detected respectively, and the second image to be detected is the next frame of the first image to be detected.
[0069] In this step, arbitrarily select two adjacent images from the set of images to be detected, namely the first image to be detected and the second image to be detected. From the acquisition of the first image to be detected to the acquisition of the second image to be detected, the transparent container has rotated by a certain angle. By analyzing these two images, it is convenient to determine the position change relationship of the defect before and after rotation.
[0070] S203. Extract the first position information of the target defect in the first image to be detected.
[0071] In this step, the threshold segmentation method can be used, but is not limited to, to extract the first position information of the target defect in the first image to be detected. The first position information can be the coordinate information of the target defect.
[0072] It should be noted that the detailed implementation method of this step will be explained in detail in the subsequent relevant embodiments.
[0073] S204. According to the first position information of the target defect, predict the position information of the target defect in the second image to be detected to obtain the predicted position information.
[0074] In this step, since the rotation path of the transparent container is a circle when it rotates, the position information of the target defect in the second image to be detected can be calculated according to the first position information of the target defect and the equation of the circle.
[0075] It should be noted that the detailed implementation method of this step will be explained in detail in the subsequent relevant embodiments.
[0076] S205. Extract the second position information of the target defect in the second image to be detected.
[0077] In this step, the second position information is the actual position information of the target defect in the second image to be detected. The extraction method of the second position information is the same as that in step S203, and will be described in detail in the subsequent relevant embodiments.
[0078] S206. Determine the area overlap degree between the predicted position information and the second position information. If the area overlap degree is less than the preset threshold, determine that the target defect is a foreign object defect of the internal product in the transparent container.
[0079] In this step, the magnitude of the area overlap reflects the deviation between the predicted position information and the actual position information of the target defect in the second image to be detected. If the target defect is a damage defect on the outer surface of the container, the predicted position is the same as the actual position, that is, the two position regions will completely overlap and there is no deviation. If the target defect is a foreign object defect in the internal product, due to the different rotation radii of the foreign object defect and the defect on the outer surface of the container when the container rotates, there will be a deviation between the predicted position and the actual position, that is, the two positions will not completely overlap in the second detection image.
[0080] To better understand the positional change relationship of the target defect before and after the rotation of the transparent container, the following combines Figure 3 and Figure 4 to illustrate with a specific example.
[0081] Exemplarily, Figure 3 is the projection of the transparent container on the Z plane. Assume that the target defect is a foreign object defect inside the transparent container product. As shown in a of Figure 3 , the orange inner circle is the rotation path of the target defect, and the blue outer circle is the rotation path of the outer surface of the container itself. Establish a two-dimensional coordinate system, and the outer circle is tangent to the horizontal axis x and the vertical axis y in the first quadrant of the two-dimensional coordinate system, and the center coordinates are (r, r), where r is the rotation radius of the transparent container. The left hollow point a1 in the inner circle represents the actual position of the defect. When collecting the front view (i.e., the first image to be detected) of the transparent container, the left solid point a2 on the outer circle is the apparent position of the defect (corresponding to the first position information of the target defect extracted from the first detection image), and the abscissas of the left hollow point a1 and the left solid point a2 are equal. According to the apparent position of the defect (i.e., the position of the left solid point a2), the predicted position after the container rotates 20 degrees is calculated, and the right solid point a3 is the predicted position, and the right hollow point a4 is the actual position of the defect after the container rotates 20 degrees. At this time, when collecting the front view (i.e., the second image to be detected) of the transparent container, the apparent position of the captured defect is the position of the middle solid point a5 on the outer circle (the position of the solid point a5 corresponds to the second position information of the target defect extracted from the second detection image to be detected). It can be seen that if the defect is a foreign object defect in the internal product of the container, due to the different rotation radii of the internal defect and the outer surface of the container during rotation, after rotation, the deviation between the predicted position and the actual position of the defect is relatively large.
[0082] Similarly, as shown in Figure 3As shown by b in [description], the projection is placed in a three-dimensional coordinate system (x, y, z). Before rotation, the hollow point b1 on the left side of the inner circle represents the actual position of the defect, and the solid point b2 on the left side of the outer circle is the apparent position of the defect when the front view is collected (corresponding to the first position information of the target defect extracted from the first image to be detected). The abscissa of the actual position of the defect is equal to the abscissa of the apparent position. After rotating 20 degrees, the predicted position after rotating 20 degrees is calculated based on the solid point b2 on the left side of the outer circle (i.e., the solid point b3 on the right side of the outer circle). The hollow point b4 on the right side of the inner circle is the actual position of the defect after the container rotates 20 degrees, and the solid point b5 in the middle of the outer circle is the apparent position of the defect after rotation (corresponding to the second position information of the target defect extracted from the second image to be detected). Through the three-dimensional coordinate system, it can also be clearly seen that if the defect is a foreign object defect of the product inside the container, the deviation between the predicted position and the actual position of the defect after rotation is relatively large. Similarly, if the defect is a defect on the outer surface of the container, through the above method of analysis, it can be clearly seen that after rotation, the predicted position and the actual position of the defect are consistent.
[0083] Exemplarily, as Figure 4 shown, Figure 4 a in [description] is the first image to be detected, Figure 4 b in [description] is the second image to be detected collected after the transparent container rotates 20 degrees; among them, the curve segment 41 represents the target defect, and the position where the minimum circumscribed rectangle 42 of the curve segment is located is the first position information; the predicted position information of the target defect in the second image to be detected after rotating 20 degrees is obtained based on the first position information (represented by the dashed box 43); then, the second position information of the target defect in the second image to be detected is determined, and the second minimum circumscribed rectangle (solid box 44) in the second image to be detected represents the second position information; in the second image to be detected, by comparing the overlap degree of the dashed box 43 and the solid box 44, it can be clearly seen that there is a large deviation between the predicted position and the actual position of the defect. Similarly, if the defect is a defect on the outer surface of the container, through the above method of analysis, it can be clearly seen that after rotation, the predicted minimum circumscribed rectangle and the actual minimum circumscribed rectangle of the defect coincide.
[0084] In this embodiment, a set of images to be detected is obtained by collecting multiple frames of images when the transparent container rotates; the first position information of the target defect in the first image to be detected among two adjacent frames of images is extracted, and then the position information of the target defect in the next frame of image (i.e., the second image to be detected) is predicted. The area coincidence degree between the predicted position information and the actual position information (i.e., the second position information) of the target defect in the second image to be detected is analyzed; if the target defect is a damage defect on the outer surface of the container, the predicted position and the actual position are consistent, that is, the two position regions will completely coincide; if the target defect is a foreign object defect of the internal product, due to the different rotation radii of the foreign object defect and the defect on the outer surface of the container when the container rotates, there will be a deviation between the predicted position and the actual position, that is, the two positions will not completely coincide in the second detection image; therefore, by analyzing the area overlap degree between the predicted position information and the actual position information, if the area overlap degree is lower than a preset threshold, it indicates that the deviation between the predicted position and the actual position is large, so as to determine that the target defect is a foreign object defect of the internal product of the container. This method can effectively distinguish foreign object defects inside the container and outer surface defects for highly viscous and non-flowing products, improve the detection accuracy of foreign objects in highly viscous products inside the container, and reduce the false alarm rate.
[0085] In a possible embodiment, as Figure 5 shown, the extracting the first position information of the target defect in the first image to be detected includes the following steps:
[0086] S501, in the first image to be detected, extract the target area where the transparent container is located.
[0087] In this step, the area where the transparent container is located needs to be cropped in the first image to be detected. Since there may be other interference features outside the area where the transparent container is located in the image, therefore, extracting the target area where the transparent container is located can directly extract the defect features in the target area more accurately, avoiding inaccurate extraction of target defect features caused by other interference features in the image.
[0088] Specifically, when actually cropping the area where the container is located, there are various implementation methods, such as cropping based on threshold segmentation, cropping based on edge detection, cropping based on manual annotation, or using a cropping method based on deep learning.
[0089] It should be noted that the detailed implementation method of cropping the target area where the transparent container is located can refer to the description in the relevant embodiments later.
[0090] S502, based on the threshold segmentation technology, extract the features of the target defect in the target area and generate the first minimum bounding rectangle of the target defect.
[0091] In a possible implementation manner, refer toFigure 6 , the above-mentioned threshold segmentation technology extracts the features of the target defect in the target area and generates the first minimum circumscribed rectangle of the target defect, including the following steps:
[0092] S5021, perform grayscale processing on the target area to obtain a grayscale image of the target area.
[0093] In this step, if the target area is a color image, it is converted into a grayscale image to reduce the data volume and computational complexity.
[0094] Specifically, according to the grayscale formula: Gray = 0.299R + 0.587G + 0.114B, the RGB value of each pixel point in the target area is converted into a grayscale value to obtain a grayscale image.
[0095] Furthermore, a filtering algorithm can be used to further denoise the grayscale image, such as mean filtering, median filtering and other algorithms. Median filtering can effectively remove salt-and-pepper noise, reduce noise interference while maintaining the image edges, and provide a clearer image for subsequent threshold segmentation.
[0096] S5022, compare the grayscale value of each pixel point in the grayscale image with a preset segmentation threshold, and mark the pixel points with grayscale values greater than the preset segmentation threshold as the pixel points of the target defect to obtain a binary image of the target defect.
[0097] In this step, a segmentation threshold can be selected according to the overall grayscale characteristics of the image; compare the grayscale value of each pixel in the grayscale image with the segmentation threshold, set the pixels greater than the segmentation threshold to white (usually represented by 255), representing the target; set the pixels less than or equal to the segmentation threshold to black (usually represented by 0), representing the background, to obtain a binary image.
[0098] Exemplarily, the determination method of the segmentation threshold can be obtained by analyzing the grayscale histogram of the image. If the histogram shows obvious bimodality, the grayscale value corresponding to the trough between the two peaks can be selected as the segmentation threshold; or the Otsu method can be used to automatically calculate a global optimal threshold as the segmentation threshold.
[0099] S5023, perform contour detection on the binary image of the target defect to obtain the boundary information of the target defect.
[0100] In this step, an image contour detection algorithm can be used to find the contour of the target defect in the binary image.
[0101] Specifically, use functions such as findContours in OpenCV to traverse all pixel points of the contour of the target defect. This function will return information about all contours in the target defect, and each contour consists of a series of points that describe the boundary of the target.
[0102] S5024, generate the first minimum bounding rectangle of the target defect according to the boundary information.
[0103] In this step, also use the findContours function. After obtaining the contour information of the target defect, traverse all pixel points in the contour and calculate and draw the minimum bounding rectangle of the target defect.
[0104] S503, determine the vertex coordinate information of the first minimum bounding rectangle as the first position information of the target defect.
[0105] In this step, determine the vertex coordinate information of the first minimum bounding rectangle by establishing a two-dimensional coordinate system and a three-dimensional coordinate system.
[0106] Exemplarily, the establishment of the coordinate system can refer to Figure 3 , taking the outer circle obtained by rotating the outer surface of the container and being tangent to the x-axis and y-axis in the first coordinate system as the standard, establish a two-dimensional coordinate system, and the center coordinates can be determined by the rotation radius r of the transparent container, that is, the center coordinates are (r, r). Correspondingly, in the three-dimensional coordinate system O(x, y, z), the outer circle obtained by rotating the outer surface of the container is tangent to the xOz plane and the yOz plane in the three-dimensional coordinate system respectively, and the center coordinates are (r, r, z), where z is the height of the defect. Whether in the two-dimensional coordinate system or the three-dimensional coordinate system, after determining the center coordinates, the vertex coordinate information of the minimum bounding rectangle can be calculated.
[0107] In this embodiment, the method of threshold segmentation can obtain the feature information of the target defect more accurately, and by determining the minimum bounding rectangle, the accurate first position information can be obtained in the first image to be detected, thereby improving the final foreign object defect detection accuracy.
[0108] In a possible embodiment, predicting the position information of the target defect in the second image to be detected based on the first position information of the target defect to obtain predicted position information includes: obtaining the rotation radius r of the transparent container and the rotation angle T of the transparent container between collecting the first image to be detected and collecting the second image to be detected; determining the vertex coordinates of the predicted minimum circumscribed rectangle of the target defect in the second detected image according to the vertex coordinate information of the first minimum circumscribed rectangle, the rotation radius r, the rotation angle T, a first formula and a second formula; determining the vertex coordinate information of the predicted minimum circumscribed rectangle as the predicted position information; where the first formula is: rotate=(x - r)cosT - (y - r)sinT, and the second formula is: Yrotate=(x - r)sinT+(y - r)cosT; where x and y respectively represent the abscissa and ordinate of the vertex of the first minimum circumscribed rectangle, and Xrotate and Yrotate respectively represent the abscissa and ordinate of the vertex of the predicted minimum circumscribed rectangle.
[0109] In this embodiment, since the first position information extracted from the first image to be detected is the apparent position information of the target defect (the solid dot a2 or b2 on the left side of the outer circle in the reference Figure 3 ), therefore, when predicting the position of the target defect in the next frame of image, it is calculated according to the outer circle radius, and the outer circle radius is the rotation radius r of the transparent container. Also, since the transparent container rotates around its own central axis, its vertical coordinate value in the z-axis direction does not change. Therefore, only by combining the abscissa x and ordinate y of the four vertices of the first minimum circumscribed rectangle, the rotation radius r, the rotation angle T, and the first formula and the second formula can the four vertex coordinates of the predicted minimum circumscribed rectangle be calculated.
[0110] Exemplarily, referring to a in the reference Figure 4 , the upper left vertex in the first minimum circumscribed rectangle frame 42 is denoted as A(x1, y1), referring to b in the reference Figure 4 , the upper left vertex of the corresponding predicted minimum circumscribed rectangle frame (dashed line frame 43) is denoted as B(Xrotate, Yrotate); substituting x1 and y1 into the first formula can obtain the abscissa Xrotate of vertex B, and substituting x1 and y1 into the second formula can obtain the ordinate Yrotate of vertex B. Similarly, the four vertex coordinates of the minimum circumscribed rectangle after rotation can be calculated by the above method, so as to generate a predicted minimum circumscribed rectangle (represented by the dashed line frame 43).
[0111] In a possible embodiment, the second position information includes vertex coordinate information of an actual second minimum bounding rectangle of the target defect in the second image to be detected; determining the area overlap degree between the predicted position information and the second position information includes: according to the vertex coordinate information of the predicted minimum bounding rectangle and the vertex coordinate information of the second minimum bounding rectangle, determining the overlapping area and the non-overlapping area between the predicted minimum bounding rectangle and the second minimum bounding rectangle; and determining the area overlap degree according to the overlapping area and the non-overlapping area.
[0112] In this embodiment, the method for determining the vertex coordinate information of the actual second minimum bounding rectangle in the second image to be detected can refer to Figure 5 the specific description of the method embodiment shown, which will not be repeated here.
[0113] Exemplarily, referring to Figure 4 b in, the actual second minimum bounding rectangle in the second image to be detected is represented by the solid line frame 44, the overlapping area between the predicted minimum bounding rectangle and the second minimum bounding rectangle is represented by the shadow, calculate the area of the shadow part (denoted as s1), and the blank area of the predicted minimum bounding rectangle or the second minimum bounding rectangle is the non-overlapping area (denoted as s2), then the area overlap degree is equal to s1÷(s1 + s2).
[0114] In a possible implementation manner, the method for calculating the area can be but is not limited to the pixel point statistics method, that is, first determining the actual physical area s represented by each pixel point in the image, taking the product of the number of pixel points N1 in the shadow area and s as the overlapping area (i.e., s1 = s*N1), and taking the product of the number of pixel points N2 in the blank area of the predicted minimum bounding rectangle or the second minimum bounding rectangle and s as the non-overlapping area (i.e., s2 = s*N2).
[0115] In this embodiment, by analyzing the area overlap degree between the predicted minimum bounding rectangle and the actual second minimum bounding rectangle of the target defect in the second image to be detected, the deviation degree between the predicted position and the actual position of the target defect before and after the rotation of the transparent container can be more clearly and accurately reflected, and it can more accurately distinguish whether the defect is a foreign object defect inside the transparent container or a surface defect, further improving the accuracy of foreign object detection and thus reducing the false alarm rate.
[0116] In a possible embodiment, collecting multiple frames of images when the transparent container rotates to obtain a set of images to be detected includes: when the transparent container rotates around its own central axis, collecting a front view of the transparent container every time it rotates a preset target angle; determining all the front views collected during the process of the transparent container rotating 360 degrees as the set of images to be detected.
[0117] In this embodiment, the value range of the preset target angle can be, but is not limited to, [10, 20] degrees.
[0118] Exemplarily, when the target angle is 20 degrees, during the rotation of the transparent container by 360 degrees, the camera captures one frame of image every 20 degrees of rotation of the transparent container, and finally an image set to be detected composed of 18 images can be obtained; when the target angle is 15 degrees, an image set to be detected composed of 24 images can be obtained finally; if the target angle is 12 degrees, an image set to be detected composed of 30 images can be obtained finally.
[0119] It should be noted that the setting of the target angle in this embodiment can be determined according to the actual situation, ensuring that the finally obtained image set to be detected can cover the 360-degree rotation of the transparent container, and the rotation angle of the corresponding transparent container between every two adjacent images is within the range of [10, 20].
[0120] In this embodiment, by capturing the front view during the 360-degree rotation of the transparent container and capturing one frame of image at a certain rotation angle, it is ensured that the finally obtained image set to be detected can cover the 360-degree rotation of the transparent container and there is a certain rotation angle interval between every two adjacent images, ensuring more accurate subsequent analysis of the change in the position of the defect before and after rotation, being able to better distinguish the internal foreign object defect and the defect on the outer surface of the transparent container, further improving the detection accuracy of foreign object defects and reducing the false alarm rate.
[0121] In a possible embodiment, the method further includes: if, among the images to be detected in the image set to be detected, there are at least M frames of images to be detected corresponding to the area overlap degree less than the preset threshold, then it is determined that the target defect is a foreign object defect of the internal product in the transparent container; where M is equal to 3 and the preset threshold is 80%.
[0122] In this embodiment, defect analysis and defect prediction are performed on all the images in the image set to be detected. If the area overlap degree between the actual position of the defect in at least 3 frames of images and the predicted position obtained from the previous frame of image is less than 80%, then it is determined that the defect is a foreign object defect of the internal product in the transparent container.
[0123] It should be noted that the selection of the relevant values in this embodiment is a data obtained through a large number of actual experiments. By setting M equal to 3 and the preset threshold to 80%, the final detection accuracy of foreign objects is higher.
[0124] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
[0125] The following is an apparatus embodiment of the present invention. For details not described in detail, reference may be made to the corresponding method embodiment above.
[0126] Figure 7 FIG. shows a schematic structural diagram of a foreign object detection apparatus provided by an embodiment of the present invention. For the sake of convenience of description, only the parts related to the embodiment of the present invention are shown and are described in detail as follows:
[0127] As Figure 7 shown, the foreign object detection apparatus includes: an image acquisition module 701, an image extraction module 702, a first information extraction module 703, a position prediction module 704, a second information extraction module 705, and a foreign object detection module 706, wherein,
[0128] The image acquisition module 701 is configured to acquire multiple frames of images when the transparent container rotates to obtain a set of images to be detected;
[0129] The image extraction module 702 is configured to arbitrarily select two adjacent images from the set of images to be detected as a first image to be detected and a second image to be detected respectively, and the second image to be detected is the next frame image of the first image to be detected;
[0130] The first information extraction module 703 is configured to extract first position information of a target defect in the first image to be detected;
[0131] The position prediction module 704 is configured to predict the position information of the target defect in the second image to be detected according to the first position information of the target defect to obtain predicted position information;
[0132] The second information extraction module 705 is configured to extract second position information of the target defect in the second image to be detected;
[0133] The foreign object detection module 706 is configured to determine the area overlap degree between the predicted position information and the second position information. If the area overlap degree is less than a preset threshold, it is determined that the target defect is a foreign object defect of the internal product in the transparent container.
[0134] In a possible implementation manner, the first information extraction module 703 is specifically configured to: in the first image to be detected, extract the target area where the transparent container is located;
[0135] Based on threshold segmentation technology, extract the features of the target defect in the target area and generate a first minimum bounding rectangle of the target defect;
[0136] Determine the vertex coordinate information of the first minimum bounding rectangle as the first position information of the target defect.
[0137] In a possible implementation, the position prediction module 704 is specifically configured to: obtain the rotation radius r of the transparent container and the rotation angle T of the transparent container from when the first image to be detected is acquired to when the second image to be detected is acquired;
[0138] According to the vertex coordinate information of the first minimum bounding rectangle, the rotation radius r, the rotation angle T, the first formula, and the second formula, determine the vertex coordinates of the predicted minimum bounding rectangle of the target defect in the second detection image;
[0139] Determine the vertex coordinate information of the predicted minimum bounding rectangle as the predicted position information;
[0140] Wherein, the first formula is: Xrotate = (x - r)cosT - (y - r)sinT, and the second formula is: Yrotate = (x - r)sinT + (y - r)cosT;
[0141] Wherein, x and y respectively represent the abscissa and ordinate of the vertex of the first minimum bounding rectangle, and Xrotate and Yrotate respectively represent the abscissa and ordinate of the vertex of the predicted minimum bounding rectangle.
[0142] In a possible implementation, the second position information includes the vertex coordinate information of the actual second minimum bounding rectangle of the target defect in the second image to be detected; the foreign object detection module 706 is specifically configured to: according to the vertex coordinate information of the predicted minimum bounding rectangle and the vertex coordinate information of the second minimum bounding rectangle, determine the overlapping area and the non-overlapping area between the predicted minimum bounding rectangle and the second minimum bounding rectangle;
[0143] Determine the area overlapping degree according to the overlapping area and the non-overlapping area.
[0144] In a possible implementation, the first information extraction module 703 is specifically configured to: perform grayscale processing on the target area to obtain a grayscale image of the target area;
[0145] Compare the grayscale value of each pixel point in the grayscale image with a preset segmentation threshold, and mark the pixel points with grayscale values greater than the preset segmentation threshold as the pixel points of the target defect to obtain a binary image of the target defect;
[0146] Perform contour detection on the binary image of the target defect to obtain the boundary information of the target defect;
[0147] Generate the first minimum bounding rectangle of the target defect according to the boundary information.
[0148] In a possible implementation, the image acquisition module 701 is specifically configured to: when the transparent container rotates around its central axis, acquire a front view of the transparent container every time it rotates a preset target angle;
[0149] Determine all the front views acquired during the process of rotating the transparent container 360 degrees as the set of images to be detected.
[0150] In a possible implementation, the foreign object detection module 706 is further configured to: if, in the set of images to be detected, there are at least M frames of the images to be detected corresponding to an area overlap degree less than the preset threshold, determine that the target defect is a foreign object defect of the internal product in the transparent container; where M is equal to 3 and the preset threshold is 80%.
[0151] Figure 8 It is a schematic diagram of an electronic device provided by an embodiment of the present invention. As Figure 8 shown, the electronic device 8 of this embodiment includes: a processor 80 and a memory 81. The memory 81 stores a computer program 82. When the processor 80 executes the computer program 82, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 80 executes the computer program 82, the functions of each module / unit in the above-mentioned device embodiments are implemented.
[0152] Exemplarily, the computer program 82 can be divided into one or more modules / units, and the one or more modules / units are stored in the memory 81 and executed by the processor 80 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 82 in the electronic device 8.
[0153] The electronic device 8 may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art can understand that Figure 8 merely an example of the electronic device 8, which does not constitute a limitation on the electronic device 8, and may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the electronic device 8 may further include input / output devices, network access devices, buses, etc.
[0154] The processor 80 may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0155] The memory 81 may be an internal storage unit of the electronic device 8, such as the hard disk or memory of the electronic device 8. The memory 81 may also be an external storage device of the electronic device 8, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. equipped on the electronic device 8. Further, the memory 81 may also include both the internal storage unit and the external storage device of the electronic device 8. The memory 81 is used to store the computer program 82 and other programs and data required by the electronic device 8. The memory 81 may also be used to temporarily store the data that has been output or will be output.
[0156] For the convenience and simplicity of description, only the above division of each functional module / unit is used as an example for illustration. In actual applications, the above functions may be assigned to different functional modules / units according to needs. The above modules / units may be implemented in the form of hardware, may also be implemented in the form of software, or may be implemented in the form of a combination of hardware and software.
[0157] The embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the methods in the above method embodiments are implemented.
[0158] The embodiment of the present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the methods in the above method embodiments are implemented.
[0159] Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.
[0160] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. Without special instructions and logical conflicts, the terms and / or descriptions between different embodiments are consistent and can be mutually referred to, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.
[0161] The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. A foreign object detection method, characterized in that, Including: Collecting multiple frames of images when the transparent container rotates to obtain a set of images to be detected; Arbitrarily selecting two adjacent images from the set of images to be detected as the first image to be detected and the second image to be detected respectively, and the second image to be detected is the next frame image of the first image to be detected; Extracting the first position information of the target defect in the first image to be detected; Predicting the position information of the target defect in the second image to be detected according to the first position information of the target defect to obtain predicted position information; Extracting the second position information of the target defect in the second image to be detected; Determining the area overlap degree between the predicted position information and the second position information, and if the area overlap degree is less than a preset threshold, determining that the target defect is a foreign object defect of the internal product in the transparent container.
2. The method according to claim 1, characterized in that, The extracting the first position information of the target defect in the first image to be detected includes: In the first image to be detected, extracting the target area where the transparent container is located; Based on the threshold segmentation technology, extracting the features of the target defect in the target area and generating the first minimum bounding rectangle of the target defect; Determining the vertex coordinate information of the first minimum bounding rectangle as the first position information of the target defect.
3. The method according to claim 2, wherein The predicting the position information of the target defect in the second image to be detected according to the first position information of the target defect to obtain predicted position information includes: Obtaining the rotation radius r of the transparent container and the rotation angle T of the transparent container from collecting the first image to be detected to collecting the second image to be detected; According to the vertex coordinate information of the first minimum bounding rectangle, the rotation radius r, the rotation angle T, the first formula and the second formula, determining the vertex coordinates of the predicted minimum bounding rectangle of the target defect in the second detected image; Determining the vertex coordinate information of the predicted minimum bounding rectangle as the predicted position information; Wherein, the first formula is: Xrotate = (x - r)cosT - (y - r)sinT, and the second formula is: Yrotate = (x - r)sinT + (y - r)cosT; Wherein, x and y respectively represent the abscissa and ordinate of the vertex of the first minimum bounding rectangle, and Xrotate and Yrotate respectively represent the abscissa and ordinate of the vertex of the predicted minimum bounding rectangle.
4. The method according to claim 3, characterized in that The second position information includes the vertex coordinate information of the actual second minimum bounding rectangle of the target defect in the second image to be detected; The determining the area overlap degree between the predicted position information and the second position information includes: According to the vertex coordinate information of the predicted minimum bounding rectangle and the vertex coordinate information of the second minimum bounding rectangle, determining the overlapping area and the non - overlapping area between the predicted minimum bounding rectangle and the second minimum bounding rectangle; Determining the area overlap degree according to the overlapping area and the non - overlapping area.
5. The method according to claim 2, characterized in that, The based on the threshold segmentation technology, extracting the features of the target defect in the target area and generating the first minimum bounding rectangle of the target defect includes: Perform grayscale processing on the target area to obtain a grayscale image of the target area; Compare the grayscale value of each pixel point in the grayscale image with a preset segmentation threshold, and mark the pixel points with grayscale values greater than the preset segmentation threshold as pixel points of the target defect to obtain a binary image of the target defect; Perform contour detection on the binary image of the target defect to obtain boundary information of the target defect; Generate a first minimum bounding rectangle of the target defect according to the boundary information.
6. The method according to any one of claims 1-5, characterized in that Collect multiple frames of images when the transparent container rotates to obtain a set of images to be detected, including: When the transparent container rotates around its central axis, capture a front view of the transparent container every time it rotates a preset target angle; Determine all the front views captured during the process of rotating the transparent container 360 degrees as the set of images to be detected.
7. The method according to claim 1, characterized in that Further include: If, in the set of images to be detected, there are at least M frames of images to be detected corresponding to an area overlap degree less than the preset threshold, then determine that the target defect is a foreign object defect of the internal product in the transparent container; Wherein, M is equal to 3, and the preset threshold is 80%.
8. A foreign object detection device, characterized in that, Include: An image acquisition module, configured to collect multiple frames of images when the transparent container rotates to obtain a set of images to be detected; An image extraction module, configured to arbitrarily select two adjacent images from the set of images to be detected as a first image to be detected and a second image to be detected respectively, and the second image to be detected is the subsequent frame of the first image to be detected; A first information extraction module, configured to extract first position information of a target defect in the first image to be detected; A position prediction module, configured to predict the position information of the target defect in the second image to be detected according to the first position information of the target defect to obtain predicted position information; A second information extraction module, configured to extract second position information of the target defect in the second image to be detected; A foreign object detection module, configured to determine the area overlap degree between the predicted position information and the second position information. If the area overlap degree is less than the preset threshold, then determine that the target defect is a foreign object defect of the internal product in the transparent container.
9. An electronic device, characterized in that, Include a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the method described in any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 7 is implemented.