Flat part visual inspection method, system and equipment, storage medium and product

Abstract

The invention discloses a flat part visual inspection method, system and device, a storage medium and a product. The method comprises the following steps: acquiring image information of a part supply table; empty disk detection is carried out based on the image information; wherein if the flat piece is placed on the piece supply table, it is judged that the empty disc is normal; if the flat piece is not placed on the piece supply table, judging that the empty disc is abnormal, and ending the detection; based on the image information, performing at least one of stack detection and hyperedge detection; wherein if the condition that two or more flat pieces are overlapped on the piece supply table does not exist, it is judged that the pieces are overlapped normally; and if two or more than two flat pieces are overlapped on the piece supply table, judging that the overlapped pieces are abnormal, and ending the detection. According to the flat part visual detection method, detection of more abnormal form types is completed.

Description

Technical Field

[0001] This application relates to the field of visual inspection technology, and in particular to a visual inspection method, system, device, storage medium and product for flat parts. Background Technology

[0002] In modern industrial production and e-commerce logistics, the demand for automated sorting of flat items (such as envelopes) is increasing. A typical sorting system usually includes a loading device, a feed table, and feed crates. The loading device includes a robot and a conveyor mechanism. The conveyor mechanism transports the scanned flat items, and the robot sequentially delivers the flat items to the feed table. The feed table has Z-axis lifting and Y-axis forward and backward movement functions, and integrates a pusher plate that moves along the X-axis to push the flat items into the feed crates. Feed crates are located on both sides of the feed table to receive the pushed flat items.

[0003] To ensure accurate and efficient sorting, the shape and size of flat parts must be inspected before they are pushed. However, current visual inspection methods for flat parts have limited capacity to detect a wide range of abnormal shapes, leading to missed detections of irregularly shaped flat parts and impacting sorting efficiency. Summary of the Invention Therefore, this application proposes a visual inspection method for flat parts, which can detect more types of abnormal shapes.

[0004] This application also proposes a visual inspection system for flat parts.

[0005] This application also proposes a visual inspection device for flat parts.

[0006] This application also proposes a computer-readable storage medium.

[0007] This application also proposes a computer program product.

[0008] The visual inspection method for flat parts according to the first aspect of this application includes the following steps: Acquire image information of the feeding station; Based on the image information, an empty disk detection is performed; if the flat component is placed on the feeding platform, the empty disk is considered normal; if the flat component is not placed on the feeding platform, the empty disk is considered abnormal, and the detection ends. Based on the image information, at least one of overlapping component detection and hyper-edge detection is performed; wherein... If there are no two or more flat parts overlapping on the feeding platform, the stacking is considered normal; if there are two or more flat parts overlapping on the feeding platform, the stacking is considered abnormal, and the detection ends. If the size of the flat part placed on the feeding platform that exceeds the effective area of ​​the feeding platform is within a set range, it is determined to be normal for the edge to be outside the edge; if the size of the flat part placed on the feeding platform that exceeds the effective area is not within the set range, it is determined to be abnormal for the edge to be outside the edge, and the detection ends.

[0009] The visual inspection method for flat parts according to the embodiments of this application has at least the following beneficial effects: by performing empty tray detection on the feeding table, it can be determined whether there is an empty tray abnormality; in addition, by performing at least one of stacking detection and edge-exceeding detection, it can be detected whether the flat part has at least one of stacking abnormality and edge-exceeding abnormality; by detecting at least two types of abnormality, the detection rate of flat parts in a defective state can be effectively improved, thereby helping to ensure smooth sorting and thus improving sorting efficiency.

[0010] According to some embodiments of this application, the "acquiring image information of the supply table" includes the following steps: The process involves registering a reference plane, which includes obtaining a first depth map of the feed stage in an empty disk state, performing plane fitting on the first depth map, and obtaining a leveling matrix. And the reference depth of the feeding stage was measured. The pixel width of the effective region and the pixel height of the effective region ; The feeder station obtains its working status. Image and second depth map; The semantic segmentation model is used to analyze the above. The image is segmented to obtain the mask of the flat component. Mask of the push plate and the mask of the baffle ; The instance segmentation model is used to analyze the... The image is segmented to obtain the instance mask of the flat component located at the top layer. .

[0011] According to some embodiments of this application, the step of "if the flat component is placed on the feeding table, the empty tray is determined to be normal; if the flat component is not placed on the feeding table, the empty tray is determined to be abnormal, and the detection ends" includes the following steps: calculate ,in, These are adjustable parameters; like If the disk is empty, the test is considered abnormal and the test ends; otherwise, the disk is considered normal.

[0012] According to some embodiments of this application, the step of "if there are no two or more overlapping flat parts on the feeding table, the stacking is determined to be normal; if there are two or more overlapping flat parts on the feeding table, the stacking is determined to be abnormal, and the detection ends" includes the following steps: Calculate the differential mask ; Morphological processing of the difference image; calculate ,in, These are adjustable parameters; like If the condition is not met, the stacking is considered abnormal and the detection ends; otherwise, the stacking is considered normal.

[0013] According to some embodiments of this application, the step of "if the size of the flat part placed on the feeding table that exceeds the effective area of ​​the feeding table is within a set range, it is determined to be normal for the edge to extend beyond the edge; if the size of the flat part placed on the feeding table that exceeds the effective area is not within the set range, it is determined to be abnormal for the edge to extend beyond the edge, and the detection ends" includes the following steps: The effective area includes the area enclosed by the push plate and the baffle plate, and the mask of the effective area is obtained. ; Calculate the mask for the excess portion. ; like It is zero, or the length of the short side of the largest portion exceeding the limit. , If the parameter is adjustable, the edge is considered normal; otherwise, the edge is considered abnormal, and the detection ends.

[0014] According to some embodiments of this application, the "effective area includes the area enclosed by the push plate and the baffle, and a mask for the effective area is obtained." "Includes the following steps:" If the push plate and the baffle are not blocked, then by and Calculate the coordinates of the four vertices of the effective region, which are respectively , , and Thus, the mask is obtained. ; If at least one of the push plate and the baffle is obstructed, then in and Based on, utilize and The effective region is completed, and then the coordinates of the four vertices of the effective region are calculated as follows: , , and Thus, the mask is obtained. .

[0015] According to some embodiments of this application, after "performing empty disk detection based on the image information", the following steps are also included: Based on the image information, warping detection is performed; If the degree of warping of the flat part placed on the feeding platform is not greater than a set threshold, it is determined to be normal warping; if the degree of warping of the flat part placed on the feeding platform is greater than the set threshold, it is determined to be abnormal warping, and the detection ends.

[0016] According to some embodiments of this application, the step of "if the degree of warping of the flat part placed on the feeding table is not greater than a set threshold, it is determined that the warping is normal; if the degree of warping of the flat part placed on the feeding table is greater than the set threshold, it is determined that the warping is abnormal, and the detection ends" includes the following steps: mask Mapping to the second depth map yields a depth map of the specific region. ; By leveling matrix Depth map of the specific region Adjust to horizontal to obtain the third depth map. ; Based on the third depth map Analyze the edge points of the flat component. If the height of a certain edge point is greater than the height of the push plate, then the edge point is determined to be a height anomaly point. Calculate the proportion of height outliers to the total number of edge points. ,in This is an indicator function that sets the value to 1 when the depth value is greater than the threshold, and 0 otherwise. Based on the third depth map For each row containing an edge point of the flat component, within a specified search range, calculate the angle of the line connecting the lowest and highest points relative to a reference plane; if the angle is greater than a set threshold... If so, the edge point is identified as an angle anomaly point; Calculate the proportion of outliers in the total number of edge points. ,in This is an indicator function that sets the value to 1 when the calculated angle is greater than the threshold, and 0 otherwise. If at least one of the abnormal edge height percentage and abnormal edge angle percentage exceeds the set threshold, it is judged as an abnormal lifting and the detection ends; otherwise, it is judged as normal lifting.

[0017] According to some embodiments of this application, the following steps are also included: If the conditions are determined to be normal (empty disk, stacked parts, excessive edges, and warping), then the dimensions of the flat part are inspected; wherein, based on The four vertices of the region where the flat component is located are obtained by fitting the bounding rectangle with the smallest area. Map the midpoints of the left and right line segments of the circumscribed rectangle, as well as the midpoints of the front and back line segments, back to three-dimensional space. Calculate the physical length of the line connecting the midpoints of the left and right line segments to obtain the length of the flat piece. Calculate the physical length of the line connecting the midpoints of the front and back line segments to obtain the width of the flat piece. based on The average height and maximum height are obtained by statistically analyzing all depth values ​​of the mask area of ​​the flat component.

[0018] A visual inspection system for flat parts according to a second aspect of this application includes: The acquisition module is used to acquire image information of the feeding station; The first detection module is used to perform empty disk detection based on the image information; wherein, if the flat component is placed on the feeding platform, the empty disk is determined to be normal; if the flat component is not placed on the feeding platform, the empty disk is determined to be abnormal, and the detection ends. The second detection module is used to perform at least one of overlapping component detection and hyper-edge detection based on the image information; wherein, If there are no two or more flat parts overlapping on the feeding platform, the stacking is considered normal; if there are two or more flat parts overlapping on the feeding platform, the stacking is considered abnormal, and the detection ends. If the size of the flat part placed on the feeding platform that exceeds the effective area of ​​the feeding platform is within a set range, it is determined to be normal for the edge to be outside the edge; if the size of the flat part placed on the feeding platform that exceeds the effective area is not within the set range, it is determined to be abnormal for the edge to be outside the edge, and the detection ends.

[0019] The visual inspection system for flat parts according to the embodiments of this application has at least the following beneficial effects: by performing empty tray detection on the feeding table, it can be determined whether there is an empty tray abnormality; in addition, by performing at least one of stacking detection and edge-exceeding detection, it can be detected whether the flat part has at least one of stacking abnormality and edge-exceeding abnormality; by detecting at least two types of abnormality, the detection rate of flat parts in a defective state can be effectively improved, thereby helping to ensure smooth sorting and thus improving sorting efficiency.

[0020] A visual inspection device for flat parts according to a third aspect of this application, characterized in that it comprises: Memory, which stores computer programs; A processor that, when executing the computer program, can implement the steps of the flat part visual inspection method described above.

[0021] The visual inspection device for flat parts according to the embodiments of this application has at least the following beneficial effects: by implementing the visual inspection method for flat parts as described above, the detection rate of flat parts in defective condition can be effectively improved, which helps to ensure smooth sorting and thus improves sorting efficiency.

[0022] A computer-readable storage medium according to a fourth aspect of this application is characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the steps of the flat part visual inspection method as described above.

[0023] The computer-readable storage medium according to the embodiments of this application has at least the following beneficial effects: by implementing the flat part visual inspection method as described above, the detection rate of flat parts in defective condition can be effectively improved, thereby helping to ensure smooth sorting and thus improving sorting efficiency.

[0024] A computer program product according to a fifth aspect of this application includes a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of the flat part visual inspection method as described above.

[0025] The computer program product according to the embodiments of this application has at least the following beneficial effects: by implementing the visual inspection method for flat parts as described above, the detection rate of flat parts in defective condition can be effectively improved, thereby helping to ensure smooth sorting and thus improving sorting efficiency.

[0026] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0027] The present application will be further described below with reference to the accompanying drawings and embodiments, wherein: Figure 1 This is a flowchart of a visual inspection method for flat parts according to an embodiment of this application; Figure 2 A schematic diagram of the feeding station; Figure 3 This is a flowchart of a visual inspection method for flat parts according to another embodiment of this application; Figure 4 Image of the feeder before registration on the reference plane; Figure 5 The image shows the result after registering the reference plane. Figure 6 The diagram to be tested shows an abnormal empty tray at the supply station; Figure 7 The image shows the detection results of an abnormal empty tray at the feeding station; Figure 8 This is the normal test diagram for the supply station; Figure 9 This is a diagram showing the normal inspection results of the parts supply station; Figure 10 The diagram to be tested shows an abnormal stacking of parts on the feeding platform; Figure 11 This is a diagram showing the detection results of abnormal stacking of parts on the feeding station; Figure 12 The image to be tested shows an abnormal tilting of the feeding table; Figure 13 The image shows the results of the inspection for the abnormal tilting of the feeding table; Figure 14 The image to be tested is for the edge-excess anomaly of the feeding table; Figure 15 This is a diagram showing the detection results of the edge-exceeding anomaly at the feeding station; Figure 16 This is a schematic diagram of a flat component visual inspection system according to an embodiment of this application; Figure 17 This is a schematic diagram of a flat part visual inspection device according to an embodiment of this application.

[0028] Reference numerals: feed table 100, push plate 110, baffle 120, effective area 130; Flat part 200; Flat part visual inspection system 300, acquisition module 310, first inspection module 320, second inspection module 330; Flat part visual inspection device 400, memory 410, processor 420. Detailed Implementation

[0029] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0030] In the description of this application, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0031] In the description of this application, "several" means one or more, "multiple" means two or more, "greater than," "less than," "exceeding," etc. are understood to exclude the stated number, and "above," "below," "within," etc. are understood to exclude the stated number. If the description mentions "first" or "second," it is only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance or implicitly indicating the number of indicated technical features or the order of the indicated technical features.

[0032] In the description of this application, unless otherwise expressly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0033] In the description of this application, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0034] Reference Figure 1 and Figure 2 The visual inspection method for a flat component 200 according to the first aspect of this application includes the following steps: S100: Obtain image information of the supply table 100; S200. Based on image information, perform empty disk detection; if a flat part 200 is placed on the feeding table 100, the empty disk is considered normal; if no flat part 200 is placed on the feeding table 100, the empty disk is considered abnormal, and the detection ends. S300, based on image information, performs at least one of stacked component detection and hyper-edge detection; wherein... If there are no two or more flat parts 200 overlapping on the feeding table 100, the stacking is considered normal; if there are two or more flat parts 200 overlapping on the feeding table 100, the stacking is considered abnormal and the detection ends. If the size of the flat part 200 placed on the feeding table 100 exceeds the effective area 130 of the feeding table 100 within the set range, it is determined to be normal for the edge to exceed the edge; if the size of the flat part 200 placed on the feeding table 100 exceeds the effective area 130 but is not within the set range, it is determined to be abnormal for the edge to exceed the edge, and the detection ends.

[0035] The visual inspection method for flat parts 200 according to the embodiments of this application has at least the following beneficial effects: by performing empty tray inspection on the feeding table 100, it can be determined whether there is an empty tray abnormality; in addition, by performing at least one of stacking inspection and edge-exceeding inspection, it can be detected whether the flat parts 200 have at least one of stacking abnormality and edge-exceeding abnormality; by detecting at least two types of abnormality, the detection rate of flat parts 200 in a defective state can be effectively improved, thereby helping to ensure smooth sorting and thus improving sorting efficiency.

[0036] Reference Figure 2 In some embodiments of this application, "acquiring image information of the supply table 100" includes the following steps: The process involves registering a reference plane, which includes obtaining a first depth map of the feed stage 100 in an empty disk state, performing plane fitting on the first depth map, and obtaining a leveling matrix. The reference depth of the feeding platform 100 was measured. The effective area has a width of 130 pixels. And the effective area of ​​130 pixels in height ; The working status of the supply station 100 Image and second depth map; Using semantic segmentation models The image is segmented to obtain the mask for flat part 200. Mask of push plate 110 and the mask of baffle 120 ; Using instance segmentation model The image is segmented to obtain the instance mask of the topmost flat component 200. .

[0037] The dimensional calibration of the feeding stage 100 was achieved through datum plane registration, providing a unified reference datum for subsequent depth information analysis and improving the accuracy and consistency of the test results.

[0038] Furthermore, by using a dual segmentation model combining semantic segmentation and instance segmentation, the overall mask of the flat component 200, the mask of the push plate, and the mask of the baffle were obtained, as well as the instance mask of the topmost flat component 200, thus achieving accurate separation between the detected target and the background.

[0039] Finally, the segmentation results of the RGB image are mapped to the depth image, which integrates two-dimensional visual features and three-dimensional depth information, providing multi-dimensional data support for subsequent detection of empty disks, stacked components, and super-edges.

[0040] Reference Figure 1 and Figure 2 In the improved embodiment described above, "S200, if a flat component 200 is placed on the feeding table 100, the empty tray is determined to be normal; if no flat component 200 is placed on the feeding table 100, the empty tray is determined to be abnormal, and the detection ends" includes the following steps: S210, Calculation ,in, These are adjustable parameters; S220, if If the disk is empty, the test is considered abnormal and the test ends; otherwise, the disk is considered normal.

[0041] The method of quantization based on mask area and threshold, and the introduction of adjustable parameters ( This allows for accurate detection of empty disk anomalies.

[0042] Specifically, The default value is 0.1. The value can also be other values, such as 0.11, 0.12 or 0.13.

[0043] Reference Figure 1 and Figure 2 In the improved embodiment described above, the step of "if there are no two or more overlapping flat parts 200 on the feeding table 100, the stacking is determined to be normal; if there are two or more overlapping flat parts 200 on the feeding table 100, the stacking is determined to be abnormal, and the detection ends" includes the following steps: S310, Calculate the differential mask ; S320. Perform morphological processing on the difference image; S330, Calculation ,in, These are adjustable parameters; S340, if If the condition is not met, the stacking is considered abnormal and the detection ends; otherwise, the stacking is considered normal.

[0044] By performing a differential operation between the overall mask of the flat component and the mask of the topmost instance, the mask of the overlapping area is directly located, thus accurately realizing the stacking anomaly of the flat component.

[0045] Furthermore, in the specific computation process, morphological processing is used to optimize the differential mask, which eliminates the interference of image noise on the detection results; adjustable parameters ( By associating the mask area of ​​the topmost flat component, the stacking threshold for determining the stacked components is dynamically adjusted, thereby adapting to the stacking detection requirements of flat components of different sizes.

[0046] Reference Figure 1 and Figure 2 In the improved embodiment described above, the step of "if the size of the flat part 200 placed on the feeding table 100 exceeding the effective area 130 of the feeding table 100 is within a set range, it is determined to be normal for the edge to extend beyond the edge; if the size of the flat part 200 placed on the feeding table 100 exceeding the effective area 130 is not within the set range, it is determined to be abnormal for the edge to extend beyond the edge, and the detection is terminated" includes the following steps: S350, the effective area 130 includes the area enclosed by the push plate 110 and the baffle 120, and the mask of the effective area 130 is obtained. ; S360, Calculate the mask for the excess portion. ; S370, if It is zero, or the length of the short side of the largest excess portion. , If the parameter is adjustable, the edge is considered normal; otherwise, the edge is considered abnormal, and the detection ends.

[0047] By subtracting from the mask, the part of the flat part 200 that exceeds the effective area 130 can be directly found, thus achieving accurate quantitative detection of edge-exceeding anomalies.

[0048] Furthermore, during the inspection process, a dual judgment logic is adopted: either "the area exceeding the limit is zero" or "the length of the shorter side of the exceeding portion is not greater than the threshold." This balances the tolerance for slight exceeding of the limit with the strict interception of severe exceeding of the limit, avoiding the reduction in sorting efficiency caused by over-inspection while ensuring that the placement of the flat parts 200 complies with regulations. Adjustable parameters can be set ( The method can adjust the edge judgment criteria according to different production precision requirements, thereby improving the method's adaptability to different scenarios.

[0049] Reference Figure 1 and Figure 2 In the improved embodiment described above, "S350, the effective area 130 includes the area enclosed by the push plate 110 and the baffle 120, and the mask of the effective area 130 is obtained." "Includes the following steps:" S351. If the push plate 110 and the baffle 120 are not blocked, then... and The coordinates of the four vertices of the effective region 130 are calculated as follows: , , and This leads to the mask. ; S352. If at least one of the push plate 110 and the baffle 120 is blocked, then in and Based on, utilize and The effective region 130 is completed, and then the coordinates of the four vertices of the effective region 130 are calculated as follows: , , and Thus, the mask is obtained. .

[0050] In the case where the mask information of the effective region 130 is incomplete, the reference width obtained by registering the reference plane is used. ) and reference height ( The method completes the effective area, solving the problem of effective area determination failure caused by occlusion and improving the robustness of the detection method.

[0051] Furthermore, regardless of whether the push plate 110 and the baffle 120 are obscured, the mask of the effective area can be obtained through a unified vertex coordinate calculation method, ensuring the consistency and reliability of the super-edge detection under different working conditions.

[0052] Reference Figure 1 and Figure 2 In the improved version of the above embodiment, after "performing empty disk detection based on image information", the following steps are also included: S400: Based on image information, perform warping detection; If the degree of warping of the flat part 200 placed on the feeding table 100 is not greater than the set threshold, it is determined to be normal warping; if the degree of warping of the flat part 200 placed on the feeding table 100 is greater than the set threshold, it is determined to be abnormal warping, and the detection ends.

[0053] When flat component 200 tilts, it can easily lead to pushing failure and damage to the component. Adding a tilting detection step after empty tray inspection can further improve the detection rate of abnormally shaped flat components 200, protect the component 200, and reduce the sorting failure rate. Reference Figure 1 and Figure 2 In the improved embodiment described above, the step of "if the degree of warping of the flat part 200 placed on the feeding table 100 is not greater than a set threshold, it is determined that the warping is normal; if the degree of warping of the flat part 200 placed on the feeding table 100 is greater than the set threshold, it is determined that the warping is abnormal, and the detection ends" includes the following steps: S410, Mask Mapping to the second depth map yields a depth map of the specific region. ; S420, via leveling matrix Depth map of a specific region Adjust to horizontal to obtain the third depth map. ; S430, based on third depth map Analyze the edge points of the flat part 200. If the height of a certain edge point is greater than the height of the push plate 110, then the edge point is determined to be a height abnormal point. S440. Calculate the proportion of height anomalies to the total number of edge points. ,in This is an indicator function that sets the value to 1 when the depth value is greater than the threshold, and 0 otherwise. S450, based on third depth map For each row containing an edge point of the flat component 200, within a specified search range, calculate the angle of the line connecting the lowest and highest points relative to the reference plane; if the angle is greater than a set threshold... If so, the edge point is identified as an angle anomaly point; S460, Calculate the proportion of angle outliers to the total number of edge points. ,in This is an indicator function that sets the value to 1 when the calculated angle is greater than the threshold, and 0 otherwise. S470. If at least one of the abnormal edge height percentage and abnormal edge angle percentage exceeds the set threshold, it is determined to be an abnormal lifting and the detection ends; otherwise, it is determined to be normal lifting.

[0054] By employing a dual judgment index of the proportion of height anomalies and the proportion of angle anomalies, the degree of warping of the flat part 200 is quantified from two dimensions: "edge height difference" and "edge tilt angle". This achieves accurate and objective judgment of warping anomalies and avoids the risk of misjudgment caused by using a single index.

[0055] In addition, by adjusting the flat part 200 mask to a horizontal position using a leveling matrix, the influence of the feed stage tilt on the depth information was eliminated, providing an accurate three-dimensional data basis for the quantitative analysis of the degree of warping.

[0056] Reference Figure 1 and Figure 2 In the improved embodiment described above, the visual inspection method for flat parts further includes the following steps: S500: If the conditions of empty disk, stacked parts, edge protrusion, and warping are determined to be normal, then the dimensional inspection of flat part 200 is performed; among which, based on By fitting the bounding rectangle with the smallest area, we obtain the four vertices of the region where the flat component 200 is located; Map the midpoints of the left and right line segments of the circumscribed rectangle, as well as the midpoints of the front and back line segments, back to three-dimensional space. Calculate the physical length of the line connecting the midpoints of the left and right line segments to obtain the length of the flat component 200. Calculate the physical length of the line connecting the midpoints of the front and back line segments to obtain the width of the flat component 200. based on The average height and maximum height are obtained by statistically analyzing all depth values ​​of the mask area of ​​the flat part 200.

[0057] After all basic inspections (empty discs, stacked parts, excessive edges, warping) are passed, dimensional inspection of flat parts is added, realizing integrated inspection of "abnormal screening + parameter measurement". No additional dimensional measurement equipment is required, which reduces the equipment cost of the production line.

[0058] The following combination Figures 3 to 15 This application provides a specific embodiment of a visual inspection method for flat components.

[0059] It should be noted that, Figure 4 The image of the feeding stage 100 before registration on the reference plane needs to be manually selected during algorithm execution to mark the reference plane so that the warping anomaly can be determined using the reference plane. Figure 5 This is a schematic diagram after performing ROI calibration of the reference plane.

[0060] Figure 6 The diagram to be tested shows an abnormal empty tray on the feeding station 100. Figure 7 The image shows the detection results of an abnormal empty disk in the supply station 100. The abnormal data information is displayed on the control panel.

[0061] Figure 8 This is the normal test diagram for supply table 100. Figure 9 The image shows the normal inspection results for the feeding station 100. The dashed box indicates that the flat part 200 is normal, and the key data information is displayed on the control panel.

[0062] Figure 10 The diagram to be tested shows an abnormal stacking of parts on the feeding table 100. Figure 11 The image shows the detection results of stacked parts abnormality on the feeding table 100. The dashed box indicates that the flat part 200 is abnormal, and the abnormal data information is displayed on the control panel.

[0063] Figure 12 The image shown is for testing the abnormal warping of the feeding table 100. Figure 13 The image shows the detection results of the warping abnormality of the feeding table 100. The dashed box indicates the abnormality of the flat part 200. The abnormal data information is displayed on the control panel.

[0064] Figure 14The image to be tested is for the edge-excess anomaly of the feeding table 100. Figure 15 The image shows the detection results of the edge-exceeding anomaly of the feeding table 100. The dashed box indicates the anomaly of the flat part 200, and the anomaly data information is displayed on the control panel.

[0065] Step 1: System Hardware and Installation Function: Used to provide basic image data support for algorithms.

[0066] Implementation: The flat parts sorting machine includes a feeding device, a feeding table 100, a material frame, and a vision inspection component. The vision inspection component is fixed directly above the feeding table 100 by a mounting bracket. The flat parts sorting machine mainly includes a three-dimensional sensor, a light source unit, an industrial control computer (IPC), and an adjustable height mounting bracket.

[0067] 1.1 Three-dimensional sensor (3D sensor): Speckle structured light depth camera, with parameters example as "640×480 depth resolution, measurement range 200 mm~1500 mm, depth accuracy ±1 mm, frame rate 8 fps, communication with IPC via GigE port"; 1.2 Light source unit: 4 near-infrared panel light sources with a bandwidth of 600 nm to 900 nm, supporting 5 kHz strobe, respectively installed in front, behind, left and right of the depth camera, with a downward angle of about 45°, and the projection range covers an area of ​​800 mm × 600 mm on the feeding stage 100; 1.3 Industrial Control Computer (IPC), equipped with high-performance GPUs and inference frameworks such as CUDA and TensorRT; 1.4 Adjustable height mounting bracket, the column height is adjustable from 200 mm to 1200 mm, and the crossbeam with slide rail can be finely adjusted along the X-axis.

[0068] 1.5 The robotic arm places the scanned flat part 200 on the feeding table 100, and the industrial control computer captures the RGB image and the second depth image through the 3D sensor (takes about 175ms).

[0069] Step 2: Flat Part 200 Recognition and Rule Detection Functions: Used to implement reference plane registration, real-time semantic segmentation of flat component 200, real-time instance segmentation, and rule judgment for each detection item.

[0070] Principle: The RGB image and second depth map acquired by the hardware are input into the detection algorithm. The algorithm segments the flat component 200 in the entire RGB image according to category based on the semantic model and outputs the segmented mask image A. Based on the instance segmentation model, the topmost flat component 200 in the RGB image is segmented and the segmented mask image B is output. The difference between A and B is used to determine whether there are stacked components or empty disk anomalies. If there are no stacked components or empty disk anomalies, the mask image of the segmented single flat component 200 is mapped to the second depth map at the pixel scale. Based on the rule-based judgment algorithm built by the CUDA framework, the specified depth map region is solved to determine whether the current flat component 200 has super-edge or warping anomalies.

[0071] Implementation method: 2.1, Reference plane registration: IPC performs RANSAC plane fitting on the first depth map to obtain the leveling matrix. And measure the reference depth of the feeding stage 100. Pixel width of the effective area pixel height of the effective area ; 2.2 Real-time semantic segmentation: The PP-Lite-Seg model, accelerated on TensorRT, performs pixel-level segmentation on RGB images and outputs... , , ,Will Mapping to a second depth map yields a depth map of a specific region. ; 2.3 Real-time instance segmentation: The YOLOv11-Seg model accelerated by the CUDA framework utilizes the GPU to extract the instance mask of the "topmost flat component". ; 2.4 Rule-based judgment: A parallel program written based on the CUDA framework implements rule-based judgment for empty disk detection, stacked component detection, edge protrusion detection, warping detection, and dimensional measurement. 2.4.1 Empty disk test, if

[0072]

[0073] It is then determined to be an empty disk, where This is an adjustable parameter; the default value is 0.1. 2.4.2 Stacked component detection, calculation of differential mask

[0074] Morphological processing is performed on the difference image, if

[0075]

[0076] Then it is determined to be a stacked piece, among which This is an adjustable parameter; the default value is 0.05. 2.4.3. Excess edge detection, through and The four vertices of the effective area of ​​the feeding table 100, which consists of the left and right push plates and the front and rear baffles, are calculated as follows: , , and If there is severe obstruction by flat parts, the effective area width and height of the feeder platform 100 obtained during registration will be used. and The occluded area is filled in to obtain an effective area mask. ; Calculate the mask for the excess portion

[0077] If the area of ​​the excess portion is not zero, calculate the length of the shorter side of the largest excess portion. ,like

[0078] Then it is determined to be a superedge, where This is an adjustable parameter; the default value is 30, in mm. 2.4.4 Warping detection, via leveling matrix Depth map of the specific region Adjust to horizontal to obtain the third depth map. ; Calculate the percentage of abnormal heights on the left and right edges: If the height of a point on the edge of a flat part is greater than the height of the push plate, it is considered an abnormal height point.

[0079] The edge height anomaly feature is the proportion of height anomalies to the total number of edge points.

[0080] in This is an indicator function that sets the value to 1 when the depth value is greater than the threshold, and 0 otherwise. Calculate the percentage of abnormal angles on the left and right edges: Among the edge points of a flat component, if the row of data corresponding to that edge point is within the specified search range (start, end), and the angle between the line connecting the lowest and highest points and the reference plane is greater than a threshold t, it is considered an angle anomaly point. The percentage of abnormal edge angles is the proportion of angle anomalies to the total number of edge points.

[0081] in This is an indicator function that takes the value 1 when the calculated angle is greater than the threshold, and 0 otherwise; if the percentage of abnormal edge height or abnormal edge angle exceeds the set threshold, it is determined to be a raised edge. 2.4.5 Dimensional Measurement: If none of the first four anomaly detections are triggered, the dimensional measurement process will proceed, based on... The four vertices of the flattened part region are obtained by fitting the bounding rectangle with the smallest area. The midpoints of the front / back and left / right line segments are mapped back to 3D space using system parameters. The physical lengths of the lines connecting the midpoints of the left / right and front / back line segments are calculated, representing the length and width of the flattened part. This is based on the warping detection... The average height and maximum height are obtained by statistically analyzing all depth values ​​of the flat component mask area. In terms of communication control, the IPC sends the corresponding detection results to the sorting main control system via industrial Ethernet. The main control system then drives the three-axis and X-axis tracks of the feeding table 100 to complete the discharge or feeding frame operation.

[0082] The entire process (model reasoning + rule judgment) is executed in parallel on the GPU, taking ≤ 25ms, totaling ≤ 200ms, and supporting a sorting cycle of up to 4000 pieces / hour. Reference Figure 16 A visual inspection system 300 for flat parts according to a second aspect of this application includes an acquisition module 310, a first detection module 320, and a second detection module 330. The acquisition module 310 is used to acquire image information of a feeding station 100. The first detection module 320 is used to perform empty tray detection based on the image information; wherein, if a flat part 200 is placed on the feeding station 100, the empty tray is determined to be normal; if no flat part 200 is placed on the feeding station 100, the empty tray is determined to be abnormal, and the detection ends.

[0083] The second detection module 330 is used to perform at least one of stacking detection and edge detection based on image information. Specifically, if there are no two or more overlapping flat parts 200 on the feeding platform 100, the stacking is considered normal; if there are two or more overlapping flat parts 200 on the feeding platform 100, the stacking is considered abnormal, and the detection ends. If the size of a flat part 200 placed on the feeding platform 100 that exceeds the effective area 130 of the feeding platform 100 is within a set range, the edge detection is considered normal; if the size of a flat part 200 placed on the feeding platform 100 that exceeds the effective area 130 is not within a set range, the edge detection is considered abnormal, and the detection ends.

[0084] The flat part visual inspection system 300 according to the embodiments of this application has at least the following beneficial effects: by performing empty tray detection on the feeding table 100, it can be determined whether there is an empty tray abnormality; in addition, by performing at least one of stacking detection and edge-exceeding detection, it can be detected whether the flat part 200 has at least one of stacking abnormality and edge-exceeding abnormality; by detecting at least two types of abnormality, the detection rate of flat parts 200 in a defective state can be effectively improved, thereby helping to ensure smooth sorting and thus improving sorting efficiency.

[0085] A visual inspection device 400 for flat parts according to a third aspect of this application is characterized by comprising a memory 410 and a processor 420. The memory 410 stores a computer program. When the processor 420 executes the computer program, it is able to implement the steps of the visual inspection method for flat parts 200 as described above.

[0086] The flat part visual inspection device 400 according to the embodiments of this application has at least the following beneficial effects: by implementing the visual inspection method for flat part 200 as described above, the detection rate of flat part 200 in a defective state can be effectively improved, which helps to ensure smooth sorting and thus improves sorting efficiency.

[0087] The memory 410 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 410 may be an internal storage unit of the flat component vision inspection device 400, such as the hard disk or memory of the flat component vision inspection device 400. In other embodiments, the memory 410 may also be an external storage device of the flat component vision inspection device 400, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the flat component vision inspection device 400. Of course, the memory 410 may also include both the internal storage unit and the external storage device of the flat component vision inspection device 400. In this embodiment, the memory 410 is typically used to store the operating system and various application software installed on the flat part visual inspection device 400, such as the program code of the flat part visual inspection method. In addition, the memory 410 can also be used to temporarily store various types of data that have been output or will be output.

[0088] In some embodiments, processor 420 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 420 is typically used to control the overall operation of the flat part visual inspection device 400. In this embodiment, processor 420 is used to run program code stored in memory 410 or process data, such as program code for a flat part visual inspection method.

[0089] In addition, the flat part vision inspection device 400 typically includes a display, a communication interface, and a bus. The memory 410, processor 420, display, and communication interface can communicate with each other via the bus. The display screen is configured to show the user interface preset in the initial setup mode, and can also display a process control window. The communication interface may include a wireless network interface or a wired network interface.

[0090] A computer-readable storage medium according to a fourth aspect of the present application is characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor 420, it implements the steps of the visual inspection method for flat parts 200 as described above.

[0091] The computer-readable storage medium according to the embodiments of this application has at least the following beneficial effects: by implementing the visual inspection method for flat parts 200 as described above, the detection rate of flat parts 200 in a defective state can be effectively improved, thereby helping to ensure smooth sorting and thus improving sorting efficiency.

[0092] The computer-readable storage medium provided in the embodiments of this application may be a USB flash drive, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific embodiments of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CDROM), optical storage device, or magnetic storage device, or any suitable combination thereof.

[0093] In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0094] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being loaded into the electronic device. Computer programs for executing this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0095] A computer program product according to a fifth aspect of this application includes a computer program, characterized in that, when executed by a processor 420, the computer program implements the steps of the visual inspection method for flat parts 200 as described above.

[0096] The computer program product according to the embodiments of this application has at least the following beneficial effects: by implementing the visual inspection method for flat parts 200 as described above, the detection rate of flat parts 200 in defective condition can be effectively improved, thereby helping to ensure smooth sorting and thus improving sorting efficiency.

[0097] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application. Furthermore, unless otherwise specified, the embodiments and features described in the embodiments of this application can be combined with each other.

Claims

1. A visual inspection method for flat parts, characterized in that, Includes the following steps: Acquire image information of the feeding station; Based on the image information, an empty disk detection is performed; if the flat component is placed on the feeding platform, the empty disk is considered normal; if the flat component is not placed on the feeding platform, the empty disk is considered abnormal, and the detection ends. Based on the image information, at least one of overlapping component detection and hyper-edge detection is performed; wherein... If there are no two or more flat parts overlapping on the feeding platform, the stacking is considered normal; if there are two or more flat parts overlapping on the feeding platform, the stacking is considered abnormal, and the detection ends. If the size of the flat part placed on the feeding platform that exceeds the effective area of ​​the feeding platform is within a set range, it is determined to be normal for the edge to be outside the edge; if the size of the flat part placed on the feeding platform that exceeds the effective area is not within the set range, it is determined to be abnormal for the edge to be outside the edge, and the detection ends.

2. The visual inspection method for flat parts according to claim 1, characterized in that, The "acquiring image information of the supply station" includes the following steps: The process involves registering a reference plane, which includes obtaining a first depth map of the feed stage in an empty disk state, performing plane fitting on the first depth map, and obtaining a leveling matrix. And the reference depth of the feeding stage was measured. The pixel width of the effective region and the pixel height of the effective region ; The feeder station obtains its working status. Image and second depth map; The semantic segmentation model is used to analyze the above. The image is segmented to obtain the mask of the flat component. Mask of the push plate and the mask of the baffle ; The instance segmentation model is used to analyze the... The image is segmented to obtain the instance mask of the flat component located at the top layer. .

3. The visual inspection method for flat parts according to claim 2, characterized in that, The statement "If the flat component is placed on the feeding table, the empty tray is considered normal; if the flat component is not placed on the feeding table, the empty tray is considered abnormal, and the test ends" includes the following steps: calculate ,in, These are adjustable parameters; like If the disk is empty, the test is considered abnormal and the test ends; otherwise, the disk is considered normal.

4. The visual inspection method for flat parts according to claim 2, characterized in that, The statement "If there are no two or more overlapping flat parts on the feeding platform, the stacking is considered normal; if there are two or more overlapping flat parts on the feeding platform, the stacking is considered abnormal, and the detection ends" includes the following steps: Calculate the differential mask ; Morphological processing of the difference image; calculate ,in, These are adjustable parameters; like If the condition is not met, the stacking is considered abnormal and the detection ends; otherwise, the stacking is considered normal.

5. The visual inspection method for flat parts according to claim 2, characterized in that, The statement "If the size of the flat part placed on the feeding table that exceeds the effective area of ​​the feeding table is within a set range, it is determined to be normal for the edge to extend beyond the edge; if the size of the flat part placed on the feeding table that exceeds the effective area is not within the set range, it is determined to be abnormal for the edge to extend beyond the edge, and the detection ends" includes the following steps: The effective area includes the area enclosed by the push plate and the baffle plate, and the mask of the effective area is obtained. ; Calculate the mask for the excess portion. ; like It is zero, or the length of the short side of the largest portion exceeding the limit. , If the parameter is adjustable, the edge is considered normal; otherwise, the edge is considered abnormal, and the detection ends.

6. The visual inspection method for flat parts according to claim 5, characterized in that, The effective area includes the area enclosed by the push plate and the baffle, and the mask of the effective area is obtained. "Includes the following steps:" If the push plate and the baffle are not blocked, then by and Calculate the coordinates of the four vertices of the effective region, which are respectively , , and Thus, the mask is obtained. ; If at least one of the push plate and the baffle is obstructed, then in and Based on, utilize and The effective region is completed, and then the coordinates of the four vertices of the effective region are calculated as follows: , , and Thus, the mask is obtained. .

7. The visual inspection method for flat parts according to claim 2, characterized in that, Following the step of "performing empty disk detection based on the image information", the following steps are also included: Based on the image information, warping detection is performed; If the degree of warping of the flat part placed on the feeding platform is not greater than a set threshold, it is determined to be normal warping; if the degree of warping of the flat part placed on the feeding platform is greater than the set threshold, it is determined to be abnormal warping, and the detection ends.

8. The visual inspection method for flat parts according to claim 7, characterized in that, The statement "If the degree of warping of the flat part placed on the feeding table is not greater than a set threshold, it is determined that the warping is normal; if the degree of warping of the flat part placed on the feeding table is greater than the set threshold, it is determined that the warping is abnormal, and the detection ends" includes the following steps: mask Mapping to the second depth map yields a depth map of the specific region. ; By leveling matrix Depth map of the specific region Adjust to horizontal to obtain the third depth map. ; Based on the third depth map Analyze the edge points of the flat component. If the height of a certain edge point is greater than the height of the push plate, then the edge point is determined to be a height anomaly point. Calculate the proportion of height outliers to the total number of edge points. ,in This is an indicator function that sets the value to 1 when the depth value is greater than the threshold, and 0 otherwise. Based on the third depth map For each row containing an edge point of the flat component, within a specified search range, calculate the angle of the line connecting the lowest and highest points relative to a reference plane; if the angle is greater than a set threshold... If so, the edge point is identified as an angle anomaly point; Calculate the proportion of outliers in the total number of edge points. ,in This is an indicator function that sets the value to 1 when the calculated angle is greater than the threshold, and 0 otherwise. If at least one of the abnormal edge height percentage and abnormal edge angle percentage exceeds the set threshold, it is judged as an abnormal lifting and the detection ends; otherwise, it is judged as normal lifting.

9. The visual inspection method for flat parts according to claim 8, characterized in that, It also includes the following steps: If the conditions are determined to be normal (empty disk, stacked parts, excessive edges, and warping), then the dimensions of the flat part are inspected; wherein, based on The four vertices of the region where the flat component is located are obtained by fitting the bounding rectangle with the smallest area. Map the midpoints of the left and right line segments of the circumscribed rectangle, as well as the midpoints of the front and back line segments, back to three-dimensional space. Calculate the physical length of the line connecting the midpoints of the left and right line segments to obtain the length of the flat piece. Calculate the physical length of the line connecting the midpoints of the front and back line segments to obtain the width of the flat piece. based on The average height and maximum height are obtained by statistically analyzing all depth values ​​of the mask area of ​​the flat component.

10. A visual inspection system for flat parts, characterized in that, include: The acquisition module is used to acquire image information of the feeding station; The first detection module is used to perform empty disk detection based on the image information; wherein, if the flat component is placed on the feeding platform, the empty disk is determined to be normal; if the flat component is not placed on the feeding platform, the empty disk is determined to be abnormal, and the detection ends. The second detection module is used to perform at least one of overlapping component detection and hyper-edge detection based on the image information; wherein, If there are no two or more flat parts overlapping on the feeding platform, the stacking is considered normal; if there are two or more flat parts overlapping on the feeding platform, the stacking is considered abnormal, and the detection ends. If the size of the flat part placed on the feeding platform that exceeds the effective area of ​​the feeding platform is within a set range, it is determined to be normal for the edge to be outside the edge; if the size of the flat part placed on the feeding platform that exceeds the effective area is not within the set range, it is determined to be abnormal for the edge to be outside the edge, and the detection ends.

11. A visual inspection device for flat parts, characterized in that, include: Memory, which stores computer programs; A processor, which, when executing the computer program, is capable of implementing the steps of the visual inspection method for flat parts as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the visual inspection method for flat parts as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the visual inspection method for flat parts as described in any one of claims 1 to 9.

Images