Workpiece detection method, device, visual inspection equipment and storage medium

Abstract

The present application relates to the technical field of equipment manufacturing, and provides a workpiece detection method, device, visual inspection equipment and storage medium. Applied to visual inspection equipment, the method includes: when it is detected that there is a workpiece to be inspected at the inspection position, multiple point cloud data of a preset inspection area are obtained, and point cloud segmentation algorithm processing is performed to obtain a point cloud cluster corresponding to each workpiece to be inspected; the workpiece type of the workpiece to be inspected is determined according to the point cloud cluster; the standard point cloud data corresponding to each workpiece type is determined, and the contour shape variable of each workpiece to be inspected is determined based on the point cloud data and the standard point cloud data; for each point cloud cluster, the point cloud data in each point cloud cluster are clustered to obtain the plane point cloud data belonging to the upper surface of the workpiece to be inspected in each point cloud cluster, and the plane point cloud data in each point cloud cluster are fitted to determine the flatness data of each workpiece to be inspected; the workpiece quality of each workpiece to be inspected is determined based on the contour shape variable and the flatness data.

Description

Technical Field

[0001] The present application relates to the field of equipment manufacturing technology, and specifically to a workpiece detection method, device, visual inspection equipment and storage medium. Background Art

[0002] In the equipment manufacturing industry, the processing of large-scale equipment requires quality inspection of various types and specifications of extra-long plate parts. To prevent substandard parts from entering the next process and to meet the requirements of subsequent welding processes, structural strength, and product appearance, these extra-long plate parts are subject to quality inspection requirements for flatness and dimensional measurement. The current method for correcting large-scale workpieces is to overlap and compare the measured workpiece with a standard workpiece. Workers use a feeler gauge to visually identify the dimensional deviation of the measured workpiece to obtain the flatness value, and then use a long ruler to measure the dimensional information to determine whether the workpiece is qualified. Manual inspection has the pain points of low efficiency, large errors, and low precision. Existing industrial automated visual inspection solutions, while meeting the accuracy requirements, have a small detection range and can only meet the automated inspection requirements of small workpieces. Machine vision automatic measurement systems for large-scale workpieces can only detect the dimensional information of the workpiece, without height information, and have low accuracy. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a workpiece detection method, device, visual inspection equipment and storage medium to solve the technical defect in the prior art that the workpiece quality of large-sized workpieces cannot be accurately detected.

[0004] In order to achieve the above-mentioned objectives, the present application provides, in a first aspect, a workpiece detection method, which is applied to a visual inspection device, and the method comprises:

[0005] When it is detected that there is a workpiece to be inspected at the inspection position, multiple point cloud data of the preset inspection area are obtained;

[0006] Processing multiple point cloud data using a point cloud segmentation algorithm to obtain a point cloud cluster corresponding to each workpiece to be inspected;

[0007] Determine the workpiece type of each workpiece to be inspected based on the point cloud cluster;

[0008] Determine the standard point cloud data corresponding to each workpiece type, and determine the contour deformation amount of each workpiece to be inspected based on the point cloud data of each workpiece to be inspected and the standard point cloud data;

[0009] For each point cloud cluster, cluster the point cloud data in each point cloud cluster to obtain the plane point cloud data belonging to the upper surface of the workpiece to be inspected in each point cloud cluster, and fit the plane point cloud data in each point cloud cluster to determine the flatness data of each workpiece to be inspected;

[0010] The workpiece quality of each workpiece to be inspected is determined based on the contour deformation amount and flatness data of each workpiece to be inspected.

[0011] In an embodiment of the present application, determining the workpiece quality of each workpiece to be inspected based on the contour deformation amount and flatness data of each workpiece to be inspected includes: for any workpiece to be inspected, when the flatness per unit of the upper surface of the workpiece to be inspected is less than a first value, determining that the flatness of the workpiece to be inspected is qualified; when the contour deformation amount per unit of the upper surface of the workpiece to be inspected is less than a second value and the contour deformation amount of each edge of the upper surface is less than a third value, determining that the contour size of the workpiece to be inspected is qualified; for any workpiece to be inspected, when the flatness of the workpiece to be inspected is qualified and the contour size is qualified, determining that the workpiece quality of the workpiece to be inspected is qualified.

[0012] In an embodiment of the present application, the visual inspection equipment includes a truss and a coding module fixed on the truss, and the method also includes: for any workpiece to be inspected, when the outline size of the workpiece to be inspected is qualified, the workpiece to be inspected is marked with a first pattern by the coding module; when the flatness of the workpiece to be inspected is qualified, the workpiece to be inspected is marked with a second pattern by the coding module; for any workpiece to be inspected, when the workpiece to be inspected includes the first pattern and the second pattern, determining that the workpiece quality of the workpiece to be inspected is qualified.

[0013] In an embodiment of the present application, the visual inspection equipment also includes a display device, and the method also includes: for any workpiece to be inspected, the three-dimensional image, workpiece type, flatness data and contour data of each workpiece to be inspected are displayed in real time through the display device, and when the workpiece quality of the workpiece to be inspected is unqualified, the unqualified area is rendered.

[0014] In an embodiment of the present application, a point cloud segmentation algorithm is performed on a plurality of point cloud data to obtain a point cloud cluster corresponding to each workpiece to be detected, including: performing spatial division processing on the plurality of point cloud data based on an octree algorithm to screen key point cloud data belonging to the workpiece to be detected in the plurality of point cloud data; constructing a tree structure diagram based on a Euclidean clustering algorithm, and storing the key point cloud data in the tree structure diagram; using the key point cloud data as the data to be clustered; selecting any one key point cloud data from the data to be clustered as the target point cloud data, obtaining a plurality of key point cloud data adjacent to the target point cloud data based on the tree structure diagram, and determining the distance between the target point cloud data and each adjacent key point cloud data. the straight-line distance between the key point cloud data and the target point cloud data; determining the key point cloud data whose straight-line distance is less than the first preset distance threshold as the same type of data of the target point cloud data; clustering the same type of data and the target point cloud data into a point cloud cluster; adding a visited mark to the target point cloud data and the same type of data of the target point cloud data; re-using the point cloud data of all key point cloud data except the target point cloud data and the same type of data of the target point cloud data as the data to be clustered; executing the step of selecting any key point cloud data from the data to be clustered as the target point cloud data again until all the key point cloud data are added with the visited mark; segmenting each point cloud cluster so that each workpiece to be detected forms an independent point cloud cluster.

[0015] In an embodiment of the present application, determining the workpiece type of each workpiece to be inspected based on the point cloud cluster includes: for any point cloud cluster, constructing a directed rectangular box for the point cloud cluster, and determining multiple vertex data of the directed rectangular box; determining the contour data of the workpiece to be inspected corresponding to each point cloud cluster based on the multiple vertex data of each directed rectangular box; and determining the workpiece type of each workpiece to be inspected based on the contour data of each workpiece to be inspected.

[0016] In an embodiment of the present application, determining the standard point cloud data corresponding to each workpiece type, and determining the contour shape variable of each workpiece to be inspected based on the point cloud data of each workpiece to be inspected and the standard point cloud data includes: after determining the workpiece type of each workpiece to be inspected, matching the workpiece drawings corresponding to each workpiece type; for any workpiece to be inspected, parsing the workpiece drawings of the workpiece to be inspected into standard point cloud data for the workpiece to be inspected, and performing point cloud data alignment processing on the point cloud data of the point cloud cluster corresponding to the workpiece to be inspected and the standard point cloud data; performing an edge line feature extraction operation on the point cloud cluster to extract the endpoints of high curvature line segments and neighborhood depth discontinuity points as edge key points, and determining the three-dimensional space straight line for the workpiece to be inspected based on the edge key points; and determining the contour shape variable of each workpiece to be inspected based on the three-dimensional space straight line and the standard point cloud data.

[0017] A second aspect of the present application provides a workpiece detection device, the device comprising:

[0018] a memory configured to store instructions;

[0019] The processor is configured to call the instructions from the memory and implement the above-mentioned workpiece detection method when executing the instructions.

[0020] A third aspect of the present application provides a visual inspection device, comprising:

[0021] The workpiece detection device described above;

[0022] Guide rail, fixed to the equipment body;

[0023] A workstation table is fixed to the equipment body and connected to the incoming material area, and is used to carry at least one workpiece to be inspected that is transferred from the incoming material area;

[0024] A plurality of line scan profilers are mounted on a truss through a bracket. The truss is used to carry the plurality of line scan profilers and move along the guide rail to scan the workpiece to be inspected.

[0025] In an embodiment of the present application, the device further includes:

[0026] The inkjet coding module is fixed on the truss and is used to perform a first pattern inkjet coding mark on the workpiece to be inspected when the outline size of the workpiece to be inspected is qualified, and to perform a second pattern inkjet coding mark on the workpiece to be inspected when the flatness of the workpiece to be inspected is qualified;

[0027] The display device is used to display the three-dimensional image, workpiece type, flatness data and contour data of each workpiece to be inspected in real time, and to render the unqualified area when the workpiece quality of the workpiece to be inspected is unqualified.

[0028] A fourth aspect of the present application provides a machine-readable storage medium having instructions stored thereon. When the instructions are executed by a processor, the processor is configured to execute the above-mentioned workpiece detection method.

[0029] The above technical solution provides a workpiece detection method for use in visual inspection equipment, including: upon detecting the presence of a workpiece to be inspected at the inspection position, obtaining multiple point cloud data of a preset inspection area; performing a point cloud segmentation algorithm on the multiple point cloud data to obtain a point cloud cluster corresponding to each workpiece to be inspected; determining the workpiece type of each workpiece to be inspected based on the point cloud cluster; determining the standard point cloud data corresponding to each workpiece type, and determining the contour shape variable of each workpiece to be inspected based on the point cloud data of each workpiece to be inspected and the standard point cloud data; clustering the point cloud data in each point cloud cluster to obtain planar point cloud data belonging to the upper surface of the workpiece to be inspected in each point cloud cluster, and fitting the planar point cloud data in each point cloud cluster to determine the flatness data of each workpiece to be inspected; thereby determining the workpiece quality of each workpiece to be inspected based on the contour shape variable and flatness data of each workpiece to be inspected. This method creates a dedicated intelligent visual inspection device for large-scale inspection scenarios through intelligent and integrated means. Based on the intelligent measurement of the three-dimensional dimensions of the workpiece by the device, accurate inspection of the workpiece quality of large-scale workpieces is achieved.

[0030] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present application, but do not constitute a limitation of the embodiments of the present application. In the accompanying drawings:

[0032] Figure 1 The following schematically shows a structural diagram of a visual inspection device according to an embodiment of the present application;

[0033] Figure 2 The following schematically shows a flow chart of a workpiece detection method according to an embodiment of the present application;

[0034] Figure 3 A schematic diagram of a large-sized workpiece according to an embodiment of the present application is schematically shown;

[0035] Figure 4 A schematic diagram of transporting a workpiece to be inspected according to an embodiment of the present application is shown schematically;

[0036] Figure 5 The internal structure diagram of a computer device according to an embodiment of the present application is schematically shown.

[0037] Description of Reference Numerals

[0038] 101, guide rail; 102, workstation workbench;

[0039] 103, a workpiece to be inspected; 104, multiple line scan profilometers;

[0040] 105, bracket; 106, truss;

[0041] 107, inkjet coding module; 108, display device;

[0042] 109, first direction moving module; 110, second direction moving module;

[0043] 111, flip pedal; 112, electrical control module;

[0044] 113, industrial computer; 114, calibration board. DETAILED DESCRIPTION

[0045] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0046] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), such directional indications are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0047] In addition, if there are descriptions involving "first" and "second" in the embodiments of the present application, the descriptions of "first" and "second" are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0048] The workpiece detection method provided in this application can be applied to Figure 1 In the application environment shown in the figure, Figure 1 As shown, a visual inspection device is provided, the device comprising:

[0049] Guide rail 101, fixed to the equipment body;

[0050] The workstation table 102 is fixed to the equipment body and connected to the incoming material area, and is used to carry at least one workpiece 103 to be inspected that is transferred from the incoming material area;

[0051] A plurality of line scan profilers 104 are mounted on a truss 106 via a bracket 105 . The truss 106 is used to carry the plurality of line scan profilers 104 and move along the guide rail 101 to scan the workpiece 103 to be inspected.

[0052] In the embodiments of this application, Figure 1 The equipment shown also includes:

[0053] The coding module 107 is fixed on the truss 106 and is used to perform a first pattern coding mark on the workpiece 103 when the outline size of the workpiece 103 is qualified, and to perform a second pattern coding mark on the workpiece 103 when the flatness of the workpiece 103 is qualified;

[0054] The display device 108 is used to display the three-dimensional image, workpiece type, flatness data and contour data of each workpiece 103 to be inspected in real time, and to render the unqualified area when the workpiece quality of the workpiece 103 to be inspected is unqualified;

[0055] The first direction moving module 109 is used to control the truss 106 carrying the multiple line scan profilers 104 to move along the guide rail 101 to scan the workpiece 103 to be inspected;

[0056] The second direction moving module 110 is used to control the coding module 107 to move along the second direction to perform coding marking on the workpiece 103 to be inspected;

[0057] The flip pedal 111 is used to ensure the user's safe operation and prevent foreign matter and dust from falling onto the guide rail 101;

[0058] The electrical control module 112 includes an electrical control cabinet, a photoelectric sensor, a motor, a switching power supply, a PLC, and electrical control components. The electrical control module 112 is used to control the motion of the device.

[0059] The industrial computer 113 is used to maintain the stable operation of the processor and perform real-time processing and analysis of image data;

[0060] The calibration plate 114 is used to calibrate the motion distortion angle and position posture of the multi-view profiler.

[0061] Figure 2 The following schematically shows a flow chart of a workpiece detection method according to an embodiment of the present application. Figure 2 As shown, the embodiment of the present application provides a method for detecting a workpiece, which can be applied to Figure 1 The visual inspection device shown in FIG. 1 , the method may include the following steps.

[0062] Step 201 : When it is detected that there is a workpiece to be inspected at the inspection position, a plurality of point cloud data of a preset inspection area are obtained.

[0063] In the embodiment of the present application, it should be noted that, in this solution, the workpiece to be detected may refer to Figure 3 For the large-sized workpiece shown, the inspection position may refer to a workstation workbench. The incoming material area is connected to the workstation workbench to transport the workpiece to be inspected to the workstation workbench for inspection. The placement of each workpiece to be inspected is such that the length direction is consistent with the guide rail direction. For example, taking the guide rail direction as the X direction, the length direction of each workpiece to be inspected should be in the X direction and the width direction in the Y direction. The workstation workbench in this solution can accommodate multiple workpieces to be inspected and placed side by side at the same time, and the interval between each workpiece to be inspected meets the requirements. For example, the interval can be set to 0.1m.

[0064] In this embodiment, it should be noted that when a workpiece to be inspected is detected on the workstation, a line scan profiler can be used to obtain multiple point cloud data of the preset area to be inspected. The line scan profiler can refer to a 3D line scan profiler, and multiple line scan profilers can be mounted in parallel along the Y direction on the truss of the visual inspection equipment. Specifically, Figure 4 The figure shows a schematic diagram of the transport of workpieces to be inspected. The incoming material area ① transports the workpieces to be inspected to the workstation workbench. Multiple line scan profilers ② move in direction ③ to scan a preset inspection area. Each workpiece to be inspected has a corresponding QR code. An intelligent code scanning module is installed above the incoming material area ① to identify the QR code on each workpiece to identify the code or model of each workpiece to be inspected. The length direction of the workpiece to be inspected is consistent with direction ③. The preset inspection area can refer to the area above the workstation workbench. It should be noted that the preset inspection area should be able to cover the entire area of ​​the workstation workbench surface to ensure that the multiple line scan profilers can scan all workpieces to be inspected. In this technical solution, multiple line scan profilers can be configured to cover a field of view of 10m in the X direction (the X direction is the direction in which the length of the workpiece to be inspected is placed), and a field of view of 2m in the Y direction (the direction in which the multiple workpieces to be inspected are placed side by side). In other words, the preset inspection area has a detection range of 2m*10m. After the multiple line scan profilers ② move in direction ③ to scan the preset inspection area, point cloud data of the preset inspection area can be obtained. At the same time, after scanning the workpiece to be inspected, it will be transferred to the next process ⑤ to continue the operation, and multiple line scan profilers will return to the starting point position in the direction of ④ to scan the next round of workpieces to be inspected transferred from the incoming area ①.

[0065] In this embodiment, it is important to note that a full-scale calibration block is designed to calibrate the rigid body pose of the multi-view line scan profiler using 3D control points. The calibration results are then used to integrate the scan data from multiple line scan profilers into a reference coordinate system. Distortion compensation is then applied to the line scan profiler data using a distortion correction and rigid body relationship calibration algorithm. The point cloud information from adjacent line scan profilers is then spliced ​​onto the point cloud information from the reference camera to form the overall point cloud data for the multi-line scan profiler scanning area.

[0066] Step 202 : performing point cloud segmentation algorithm processing on the plurality of point cloud data to obtain a point cloud cluster corresponding to each workpiece to be inspected.

[0067] In an embodiment of the present application, after acquiring multiple point cloud data, the multiple point cloud data can be smoothed first to remove noise generated by scanning or other reasons, and then the multiple point cloud data can be processed by a point cloud segmentation algorithm to obtain a point cloud cluster corresponding to each workpiece to be inspected.

[0068] In an embodiment of the present application, a point cloud segmentation algorithm is performed on a plurality of point cloud data to obtain a point cloud cluster corresponding to each workpiece to be detected, including: performing spatial division processing on the plurality of point cloud data based on an octree algorithm to screen key point cloud data belonging to the workpiece to be detected in the plurality of point cloud data; constructing a tree structure diagram based on a Euclidean clustering algorithm, and storing the key point cloud data in the tree structure diagram; using the key point cloud data as the data to be clustered; selecting any one key point cloud data from the data to be clustered as the target point cloud data, obtaining a plurality of key point cloud data adjacent to the target point cloud data based on the tree structure diagram, and determining the distance between the target point cloud data and each adjacent key point cloud data. the straight-line distance between the key point cloud data and the target point cloud data; determining the key point cloud data whose straight-line distance is less than the first preset distance threshold as the same type of data of the target point cloud data; clustering the same type of data and the target point cloud data into a point cloud cluster; adding a visited mark to the target point cloud data and the same type of data of the target point cloud data; re-using the point cloud data of all key point cloud data except the target point cloud data and the same type of data of the target point cloud data as the data to be clustered; executing the step of selecting any key point cloud data from the data to be clustered as the target point cloud data again until all the key point cloud data are added with the visited mark; segmenting each point cloud cluster so that each workpiece to be detected forms an independent point cloud cluster.

[0069] In this embodiment, it should be noted that the octree algorithm may refer to octree, which is a tree-like data structure used to describe three-dimensional space. Each node of the octree represents a volume element of a cube, and each node has eight child nodes. The volume elements represented by the eight child nodes added together are equal to the volume of the parent node. The octree is an extension of the quadtree in three-dimensional space. There are four quadrants in two dimensions and eight quadrants in three dimensions. The octree is mainly used for space division and nearest neighbor search. Since the collected point cloud data includes the entire preset detection area, the acquired point cloud data includes both the workpiece to be detected and the space area above the workstation workbench. In this technical solution, the set operation of the point cloud target can be quickly performed based on the octree, and the point cloud search of the dense point cloud to be detected area can be quickly carried out according to the point cloud space division to screen the key point cloud data belonging to the workpiece to be detected in multiple point cloud data.

[0070] In this embodiment, it should be noted that the Euclidean clustering algorithm is a clustering algorithm based on the Euclidean distance height metric, and the nearest neighbor query algorithm based on KD-Tree is an important preprocessing method for accelerating the Euclidean clustering algorithm. Therefore, when processing the key point cloud data belonging to the workpiece to be detected, it is necessary to first construct a tree structure diagram and store the key point cloud data in the tree structure diagram. The tree structure diagram may refer to a KD-Tree. The key point cloud data is used as the data to be clustered, and any key point cloud data is selected from the data to be clustered as the target point cloud data. Based on the tree structure diagram, multiple key point cloud data adjacent to the target point cloud data are obtained, and the straight-line distance between the target point cloud data and each adjacent key point cloud data is determined, and the key point cloud data whose straight-line distance is less than a first preset distance threshold is determined as the same type of data as the target point cloud data, and the same type of data and the target point cloud data are clustered into a point cloud cluster. For example, taking a point O from the key point cloud data as the target point cloud data, kdtree is used to find the P points closest to point O. The distance from these P points to point O is determined, and all points with distances less than the set clustering threshold x are placed in set M, thereby clustering all point cloud data in set M into a point cloud cluster. After clustering a point cloud cluster, the target point cloud data and similar data of the target point cloud data are marked as visited. All key point cloud data except the target point cloud data and similar data of the target point cloud data are re-classified as data to be clustered. The step of selecting any key point cloud data from the data to be clustered as the target point cloud data is repeated until all key point cloud data are marked as visited, that is, all point cloud data are clustered into different point cloud clusters. Finally, each point cloud cluster is divided into independent parts so that each workpiece to be inspected forms an independent point cloud cluster.

[0071] Step 203 : determining the workpiece type of each workpiece to be inspected based on the point cloud cluster.

[0072] In an embodiment of the present application, after obtaining a point cloud cluster corresponding to each workpiece to be inspected, point cloud boundary extraction can be performed on each point cloud cluster to obtain contour data of each workpiece to be inspected, so that the workpiece type of each workpiece to be inspected can be further identified based on the contour data of each workpiece to be inspected.

[0073] In an embodiment of the present application, determining the workpiece type of each workpiece to be inspected based on the point cloud cluster includes: for any point cloud cluster, constructing a directed rectangular box for the point cloud cluster, and determining multiple vertex data of the directed rectangular box; determining the contour data of the workpiece to be inspected corresponding to each point cloud cluster based on the multiple vertex data of each directed rectangular box; and determining the workpiece type of each workpiece to be inspected based on the contour data of each workpiece to be inspected.

[0074] In this embodiment, it should be noted that point cloud boundary extraction refers to a process involving extracting boundary points from three-dimensional point cloud data. These boundary points typically represent the external or internal contours of an object. There are various methods and techniques for extracting boundary points, including projection-based methods, image processing techniques, and direct extraction from three-dimensional point cloud data. Therefore, after segmenting and obtaining point cloud clusters corresponding to each workpiece to be inspected, point cloud boundary extraction is performed on each point cloud cluster to obtain the contour data of each workpiece to be inspected. Specifically, in this technical solution, the contour data of each workpiece to be inspected can be extracted based on an OBB bounding box algorithm combined with an edge detection algorithm. The OBB bounding box algorithm can refer to an algorithm for calculating the minimum bounding box of an object in three-dimensional space. A minimum bounding box is a rectangular box that completely encloses an object and has minimal boundaries. It can be used to describe the object's position, orientation, and size. The OBB bounding box algorithm calculates the minimum bounding box by finding the object's principal orientation and boundary points. The OBB bounding box algorithm is a rotatable rectangular box that can better adapt to rotation and misaligned shapes for irregularly shaped objects, better fitting and covering the object's actual shape. Specifically, for any point cloud cluster, a directed rectangular box is constructed based on the OBB bounding box algorithm to enclose the feature point cloud cluster of each workpiece to be inspected. This box then contains eight vertices for the cluster. Based on the vertex information of each vertex, the feature information of the object within the bounding box can be quickly extracted. The bounding box's features are then calculated by calculating the length, width, and height of the bounding box, as well as the distances and angles between vertices and the direction vector of the principal axis. Statistical analysis of the OBB bounding box feature data reveals the three-dimensional shape of each workpiece to be inspected. This, combined with the edge detection algorithm, can then be used to extract the contour data of each workpiece to be inspected.

[0075] Step 204 : determining the standard point cloud data corresponding to each workpiece type, and determining the contour deformation amount of each workpiece to be inspected based on the point cloud data of each workpiece to be inspected and the standard point cloud data.

[0076] In this embodiment of the present application, after obtaining the workpiece type of each workpiece to be inspected, a workpiece drawing of the corresponding model can be searched based on the workpiece type and workpiece code. The standard point cloud data corresponding to each workpiece type can be determined from the workpiece drawing. Feature extraction is performed on the standard point cloud data of each workpiece to be inspected and the point cloud cluster to obtain the contour deformation of each workpiece to be inspected.

[0077] In an embodiment of the present application, determining the standard point cloud data corresponding to each workpiece type, and determining the contour shape variable of each workpiece to be inspected based on the point cloud data of each workpiece to be inspected and the standard point cloud data includes: after determining the workpiece type of each workpiece to be inspected, matching the workpiece drawings corresponding to each workpiece type; for any workpiece to be inspected, parsing the workpiece drawings of the workpiece to be inspected into standard point cloud data for the workpiece to be inspected, and performing point cloud data alignment processing on the point cloud data of the point cloud cluster corresponding to the workpiece to be inspected and the standard point cloud data; performing an edge line feature extraction operation on the point cloud cluster to extract the endpoints of high curvature line segments and neighborhood depth discontinuity points as edge key points, and determining a three-dimensional space straight line for the workpiece to be inspected based on the edge key points; and determining the contour shape variable of each workpiece to be inspected based on the three-dimensional space straight line and the standard point cloud data.

[0078] In this technical solution, it should be noted that the workpiece drawing can refer to a CAD drawing of the workpiece. Therefore, after determining the workpiece type of each workpiece to be inspected, a matching CAD drawing is searched for based on the workpiece type and workpiece code. The CAD drawing is then parsed to obtain the standard point cloud data for each workpiece to be inspected. Ultra-high-precision point cloud registration technology based on PCA and ICP is then used to align the CAD point cloud data with the point cloud data of the point cloud cluster. PCA can refer to principal component analysis, which utilizes the concept of dimensionality reduction to transform multiple metrics into a few comprehensive indicators. For each point cloud cluster, an edge line feature extraction operation is performed on the point cloud cluster to extract the endpoints of high-curvature line segments and neighboring depth discontinuities as edge key points. 2D line extraction is then performed on the edge key points using a random sample consensus algorithm. The 2D lines are then back-projected into 3D space to obtain 3D spatial lines, thereby obtaining the 3D spatial lines for each workpiece to be inspected. Information related to the 3D spatial lines is then calculated from the standard CAD point cloud data space to calculate the contour shape variables of each workpiece to be inspected.

[0079] In step 205 , for each point cloud cluster, the point cloud data in each point cloud cluster are clustered to obtain the planar point cloud data belonging to the upper surface of the workpiece to be inspected in each point cloud cluster, and the planar point cloud data in each point cloud cluster are fitted to determine the flatness data of each workpiece to be inspected.

[0080] In an embodiment of the present application, for each point cloud cluster, the point cloud data within each cluster can be further clustered based on the Euclidean clustering algorithm and the K-means algorithm to cluster the planar point cloud data within each point cloud cluster that belongs to the upper surface of the workpiece to be inspected. The K-means algorithm, also known as the k-means clustering algorithm, is an iterative cluster analysis algorithm. Its steps are: pre-dividing the data into K groups, randomly selecting K objects as initial cluster centers, then calculating the distance between each object and each seed cluster center, and assigning each object to the cluster center closest to it. The cluster centers and the objects assigned to them represent a cluster. With each sample assigned, the cluster center is recalculated based on the existing objects in the cluster. This process is repeated until a termination condition is met. The termination condition can be that no (or a minimum number of) objects are reassigned to different clusters, no (or a minimum number of) cluster centers change again, and the sum of squared errors reaches a local minimum. After clustering to obtain the upper surface of each workpiece to be inspected, a plane fitting algorithm can be used to calculate the flatness data of each area and the whole of each workpiece to be inspected. Specifically, for each workpiece to be inspected, the RANSAC random sample consensus algorithm can be combined with the least squares calculation method to fit all points in the upper surface area of ​​the workpiece to be inspected to obtain a best-fit plane. Calculate the maximum and minimum distances from all points in the upper surface area of ​​the workpiece to be inspected to the best-fit plane, and add up the absolute values ​​to obtain the maximum flatness. It should be noted that the workpiece to be inspected in this technical solution is an extra-long plate-like workpiece, and whether the flatness of the workpiece is flat has a great influence on the accuracy of various parts subsequently processed into equipment. If the workpiece has poor flatness, it will cause uneven force on the workpiece and poor appearance of the product. Therefore, quality inspection of the flatness of extra-long plate-like workpieces can effectively prevent workpieces that do not meet the precision from flowing into the next process to meet the requirements of later welding technology, structural strength and product appearance.

[0081] Step 206 : Determine the workpiece quality of each workpiece to be inspected based on the contour deformation amount and flatness data of each workpiece to be inspected.

[0082] In the embodiment of the present application, after obtaining the contour deformation amount and flatness data of each workpiece to be inspected, the workpiece quality of each workpiece to be inspected can be evaluated based on the contour deformation amount and the flatness data.

[0083] In an embodiment of the present application, determining the workpiece quality of each workpiece to be inspected based on the contour deformation amount and flatness data of each workpiece to be inspected includes: for any workpiece to be inspected, when the flatness per unit of the upper surface of the workpiece to be inspected is less than a first value, determining that the flatness of the workpiece to be inspected is qualified; when the contour deformation amount per unit of the upper surface of the workpiece to be inspected is less than a second value and the contour deformation amount of each edge of the upper surface is less than a third value, determining that the contour size of the workpiece to be inspected is qualified; for any workpiece to be inspected, when the flatness of the workpiece to be inspected is qualified and the contour size is qualified, determining that the workpiece quality of the workpiece to be inspected is qualified.

[0084] In this embodiment, it should be noted that the first numerical value, the second numerical value and the third numerical value can be set according to, but not limited to, historical experience values ​​and actual needs. For example, in this technical solution, the first numerical value can be 0.2 mm / m², the second numerical value can be 1.5 mm / m, and the third numerical value can be 3 mm / full length. Specifically, for any workpiece to be detected, when the flatness per square meter of the upper surface of the workpiece to be detected is less than 0.2 mm / m², the flatness of the workpiece to be detected can be considered to be qualified. For any workpiece to be detected, when the contour deformation variable per meter of the upper surface of the workpiece to be detected, that is, the difference between the actual contour size and the size set by the CAD model is less than 1.5 mm / m, and the contour deformation variable of each periphery of the upper surface of the workpiece to be detected is less than 3 mm / full length, the contour deformation variable of the workpiece to be detected can be considered to be qualified. For any workpiece to be detected, if and only if the flatness and contour deformation variables of the workpiece to be detected are both qualified, the workpiece quality of the workpiece to be detected is considered to be qualified. When either the flatness or the contour deformation of the workpiece to be inspected is unqualified, the workpiece to be inspected will be transferred to the corresponding unqualified product frame for manual correction.

[0085] In an embodiment of the present application, the visual inspection equipment includes a truss and a coding module fixed on the truss, and the method also includes: for any workpiece to be inspected, when the outline size of the workpiece to be inspected is qualified, the workpiece to be inspected is marked with a first pattern by the coding module; when the flatness of the workpiece to be inspected is qualified, the workpiece to be inspected is marked with a second pattern by the coding module; for any workpiece to be inspected, when the workpiece to be inspected includes the first pattern and the second pattern, determining that the workpiece quality of the workpiece to be inspected is qualified.

[0086] In this embodiment, it should be noted that the inkjet coding model may include inkjet coding system equipment, inkjet coding anti-collision structure and connecting plate, the first pattern may refer to a red circular pattern, and the second pattern may refer to a yellow triangular pattern. Therefore, for any workpiece to be inspected, if the flatness of the workpiece to be inspected is unqualified, the inkjet coding device will inkjet a red circular pattern mark on the workpiece to be inspected, and if the contour deformation of the workpiece to be inspected is unqualified, the inkjet coding device will inkjet a yellow triangular pattern mark on the workpiece to be inspected. Among them, only when the workpiece to be inspected has both red circular pattern and yellow triangular pattern inkjet coding, can the workpiece to be inspected be determined as a workpiece with qualified workpiece quality. When the workpiece to be inspected lacks any one of the inkjet codings, it will be transmitted to the corresponding guidance manual to perform the corresponding workpiece repair work.

[0087] In an embodiment of the present application, the visual inspection equipment also includes a display device, and the method also includes: for any workpiece to be inspected, the three-dimensional image, workpiece type, flatness data and contour data of each workpiece to be inspected are displayed in real time through the display device, and when the workpiece quality of the workpiece to be inspected is unqualified, the unqualified area is rendered.

[0088] In this embodiment, it should be noted that the display device includes a human-computer interaction interface and an image display interface. For any workpiece to be detected, the detection results of each workpiece to be detected will be displayed on the image display interface. Specifically, the displayed content may include the 3D image of each workpiece to be detected, the workpiece type, flatness data such as flatness per square meter and overall flatness, contour data such as contour shape variables per meter and contour shape variables of each periphery. For any workpiece to be detected, when the workpiece quality of the workpiece to be detected is unqualified, the unqualified data will be marked in red. At the same time, the unqualified area will also be rendered in the 3D model with a red circle and the position to the edge.

[0089] The above technical solution creates a dedicated intelligent visual inspection equipment for large-scale inspection scenarios through intelligence and integration. Based on the intelligent measurement of the three-dimensional dimensions of the workpiece by this equipment, it realizes the accurate detection of the workpiece quality of large-scale workpieces.

[0090] This technical solution calibrates the rigid body pose of multi-view line scan cameras through three-dimensional control points. The calibration results are used to integrate the scanning data of multiple line laser cameras into a reference coordinate system. It is compatible with multi-size workpiece inspection and can achieve high-precision inspection of very long workpieces. It uses advanced technologies such as point cloud segmentation technology based on 3D octree and Euclidean clustering, high-precision partitioning flatness calculation technology based on line scan point clouds, point cloud classification technology based on directed bounding boxes, and ultra-high-precision point cloud registration technology based on PCA and ICP. Combined with CAD drawing analysis, it combines workpiece flatness, contour deformation, and workpiece size detection algorithms to achieve high-precision inspection of large-scale workpieces of various specifications. The integrated inkjet printer module, vision module, and electrical control module create a multifunctional inspection device that can be applied to scenarios such as flatness inspection, workpiece sorting, and size measurement, addressing the pain point of large-scale workpieces that cannot be automatically inspected.

[0091] An embodiment of the present application provides a storage medium on which a program is stored. When the program is executed by a processor, the above-mentioned workpiece detection method is implemented.

[0092] The present invention provides a workpiece detection device, comprising:

[0093] a memory configured to store instructions; and

[0094] The processor is configured to call the instructions from the memory and implement the workpiece detection method when executing the instructions.

[0095] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor A01, a network interface A02, a memory (not shown in the figure), and a database (not shown in the figure) connected via a system bus. The processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02, and a database (not shown in the figure). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A04. The database of the computer device is used to store workpiece detection method data. The network interface A02 of the computer device is used to communicate with an external terminal via a network connection. When the computer program B02 is executed by the processor A01, a workpiece detection method is implemented.

[0096] Those skilled in the art will understand that Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0097] An embodiment of the present application provides a device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, steps of a workpiece detection method are implemented.

[0098] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing the program steps of the initialization workpiece detection method.

[0099] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0100] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0101] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0103] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0104] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0105] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0106] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0107] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included within the scope of the claims of the present application.

Claims

1. A workpiece detection method, characterized in that: Applied to visual inspection equipment, the method comprises: When it is detected that there is a workpiece to be inspected at the inspection position, multiple point cloud data of the preset inspection area are obtained; Processing the multiple point cloud data with a point cloud segmentation algorithm to obtain a point cloud cluster corresponding to each workpiece to be inspected; Determine the workpiece type of each workpiece to be inspected according to the point cloud cluster; Match the artifact drawings corresponding to each artifact type; For any workpiece to be inspected, the workpiece drawing of the workpiece to be inspected is parsed into standard point cloud data for the workpiece to be inspected, and the point cloud data of the point cloud cluster corresponding to the workpiece to be inspected is aligned with the standard point cloud data; an edge straight line feature extraction operation is performed on the corresponding point cloud cluster to extract the endpoints of the high curvature line segment and the neighborhood depth discontinuity points as edge key points, and a three-dimensional space straight line for the workpiece to be inspected is determined based on the edge key points; and a contour shape variable of each workpiece to be inspected is determined based on the three-dimensional space straight line and the standard point cloud data; For each point cloud cluster, cluster the point cloud data in each point cloud cluster to obtain the plane point cloud data belonging to the upper surface of the workpiece to be inspected in each point cloud cluster, and fit the plane point cloud data in each point cloud cluster to determine the flatness data of each workpiece to be inspected; The workpiece quality of each workpiece to be inspected is determined based on the contour deformation amount and flatness data of each workpiece to be inspected.

2. The workpiece detection method according to claim 1, characterized in that: Determining the workpiece quality of each workpiece to be inspected based on the contour deformation amount and flatness data of each workpiece to be inspected includes: For any workpiece to be inspected, when the flatness per unit of the upper surface of the workpiece to be inspected is less than a first value, it is determined that the flatness of the workpiece to be inspected is qualified; When the contour deformation amount per unit of the upper surface of the workpiece to be inspected is less than the second value and the contour deformation amount of each edge of the upper surface is less than the third value, it is determined that the contour size of the workpiece to be inspected is qualified; For any workpiece to be inspected, when the flatness and contour dimensions of the workpiece to be inspected are qualified, it is determined that the workpiece quality of the workpiece to be inspected is qualified.

3. The workpiece detection method according to claim 1, characterized in that: The visual inspection device includes a truss and a coding module fixed on the truss, and the method further includes: For any workpiece to be inspected, when the outline size of the workpiece to be inspected is qualified, the workpiece to be inspected is marked with a first pattern by the coding module; When the flatness of the workpiece to be inspected is qualified, the workpiece to be inspected is marked with a second pattern by the coding module; For any workpiece to be inspected, when the workpiece to be inspected includes a first pattern and a second pattern, it is determined that the workpiece quality of the workpiece to be inspected is qualified.

4. The workpiece detection method according to claim 1, characterized in that: The visual inspection device further includes a display device, and the method further includes: For any workpiece to be inspected, the display device displays the three-dimensional image, workpiece type, flatness data and contour data of each workpiece to be inspected in real time, and renders the unqualified area when the workpiece quality of the workpiece to be inspected is unqualified.

5. The workpiece detection method according to claim 1, characterized in that: The step of performing point cloud segmentation algorithm processing on the plurality of point cloud data to obtain a point cloud cluster corresponding to each workpiece to be detected includes: Performing spatial division processing on the plurality of point cloud data based on an octree algorithm to screen key point cloud data belonging to the workpiece to be inspected from the plurality of point cloud data; Constructing a tree structure diagram based on a Euclidean clustering algorithm, and storing the key point cloud data in the tree structure diagram; Using the key point cloud data as data to be clustered; Select any key point cloud data from the data to be clustered as target point cloud data, obtain multiple key point cloud data adjacent to the target point cloud data based on the tree structure diagram, and determine the straight-line distance between the target point cloud data and each adjacent key point cloud data; Determine the key point cloud data whose straight-line distance is less than a first preset distance threshold as the same type of data as the target point cloud data; Clustering the similar data and the target point cloud data into a point cloud cluster; Adding a visited mark to the target point cloud data and data of the same type as the target point cloud data; Re-using all key point cloud data except the target point cloud data and the same type of data as the target point cloud data as data to be clustered; The step of selecting any key point cloud data from the data to be clustered as the target point cloud data is performed again until all the key point cloud data are marked as visited; Each point cloud cluster is segmented so that each workpiece to be inspected forms an independent point cloud cluster.

6. The workpiece detection method according to claim 1, characterized in that: Determining the workpiece type of each workpiece to be detected according to the point cloud cluster includes: For any point cloud cluster, construct a directed rectangular frame for the point cloud cluster, and determine a plurality of vertex data of the directed rectangular frame; Determine contour data of the workpiece to be detected corresponding to each point cloud cluster based on multiple vertex data of each directed rectangular frame; The workpiece type of each workpiece to be inspected is determined based on the contour data of each workpiece to be inspected.

7. A workpiece detection device, characterized in that: The device comprises: a memory configured to store instructions; A processor is configured to call the instructions from the memory and implement the workpiece detection method according to any one of claims 1 to 6 when executing the instructions.

8. A visual inspection device, characterized in that: The device comprises: The workpiece detection device according to claim 7; Guide rails, fixed on the equipment body; A workstation workbench is fixed on the equipment body, connected to the incoming material area, and used to carry at least one workpiece to be inspected transmitted through the incoming material area; A plurality of line scan profilers are mounted on a truss via a bracket, and the truss is used to carry the plurality of line scan profilers and move along a guide rail to scan a workpiece to be inspected.

9. The visual inspection device according to claim 8, characterized in that: The device also includes: A coding module is fixed on the truss, and is used to perform a first pattern coding mark on the workpiece to be detected when the outline size of the workpiece to be detected is qualified, and to perform a second pattern coding mark on the workpiece to be detected when the flatness of the workpiece to be detected is qualified; The display device is used to display the three-dimensional image, workpiece type, flatness data and contour data of each workpiece to be inspected in real time, and render the unqualified area when the workpiece quality of the workpiece to be inspected is unqualified.

10. A machine-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the processor is configured to perform the workpiece detection method according to any one of claims 1 to 6.

Images