Workpiece detection method and device, visual detection equipment and storage medium
Through the combination of point cloud data processing technology and inkjet module, the accuracy and efficiency of quality detection of large-size workpieces are solved, and high-precision automatic detection and real-time result display of large-size workpieces are realized.
Patent Information
- Application Number
- CN202411491195.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-24
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-10-24
AI Technical Summary
The prior art cannot accurately detect the quality of large-sized workpieces, and the manual inspection efficiency is low and the error is large. The automated visual inspection scheme has low accuracy in large-sized workpiece detection and cannot fully detect height information.
Point cloud data processing technology is adopted to determine the contour variables and planarity data of the workpiece through point cloud segmentation algorithm, clustering and fitting, and combine the injection module and display device to realize automatic detection of workpiece quality.
Accurate quality inspection of large-sized workpieces is realized, detection efficiency and accuracy are improved, detection results can be displayed in real time and unqualified areas are rendered.
Smart Images

Figure CN119006470A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of equipment manufacturing technology, and in particular to a workpiece detection method, device, visual inspection equipment and storage medium. Background Art
[0002] In the large-scale equipment processing in the equipment manufacturing industry, the production of various types and specifications of super-long plate parts needs to be quality inspected. In order to prevent parts that do not meet the precision from flowing into the next process and meet the requirements of the later welding process, structural strength, and product appearance, super-long plate parts have quality inspection requirements for flatness and size measurement. The current correction method for large-size workpieces is to overlap and compare the measured workpiece with the standard workpiece. Workers use a feeler gauge to find the size deviation of the measured workpiece with the naked eye to obtain the flatness value, and use a long ruler to measure the size information to determine whether the workpiece is qualified. Manual inspection has pain points such as low efficiency, large errors, and low precision. The existing industrial automated visual inspection solutions have a small detection range while meeting the accuracy requirements, and can only meet the automated inspection of small workpieces; while the large-size workpiece automatic measurement system of machine vision can only detect the size information of the workpiece, without height information, and has low accuracy. Summary of the invention
[0003] The purpose of the embodiments of the present application is to provide a workpiece detection method, device, visual inspection equipment and storage medium to solve the technical defect of the prior art that the workpiece quality of large-sized workpieces cannot be accurately detected.
[0004] In order to achieve the above-mentioned purpose, the first aspect of the present application provides a workpiece detection method, which is applied to a visual inspection device, and the method comprises: When it is detected that there is a workpiece to be inspected at the inspection position, multiple point cloud data of the preset inspection area are obtained; Processing the multiple point cloud data with a point cloud segmentation algorithm to obtain a point cloud cluster corresponding to each workpiece to be inspected; Determine the workpiece type of each workpiece to be inspected according to the point cloud cluster; Determine the standard point cloud data corresponding to each workpiece type, and determine the contour deformation amount of each workpiece to be inspected based on the point cloud data of each workpiece to be inspected and the standard point cloud data; For each point cloud cluster, cluster the point cloud data in each point cloud cluster to obtain the plane point cloud data belonging to the upper surface of the workpiece to be inspected in each point cloud cluster, and fit the plane point cloud data in each point cloud cluster to determine the flatness data of each workpiece to be inspected; The workpiece quality of each workpiece to be inspected is determined based on the contour deformation amount and flatness data of each workpiece to be inspected.
[0005] In an embodiment of the present application, determining the workpiece quality of each workpiece to be inspected based on the contour deformation amount and flatness data of each workpiece to be inspected includes: for any workpiece to be inspected, when the flatness per unit of the upper surface of the workpiece to be inspected is less than a first value, determining that the flatness of the workpiece to be inspected is qualified; when the contour deformation amount per unit of the upper surface of the workpiece to be inspected is less than a second value and the contour deformation amount of each edge of the upper surface is less than a third value, determining that the contour size of the workpiece to be inspected is qualified; for any workpiece to be inspected, when the flatness of the workpiece to be inspected is qualified and the contour size is qualified, determining that the workpiece quality of the workpiece to be inspected is qualified.
[0006] In an embodiment of the present application, the visual inspection equipment includes a truss and a coding module fixed on the truss, and the method also includes: for any workpiece to be inspected, when the contour size of the workpiece to be inspected is qualified, the workpiece to be inspected is marked with a first pattern by the coding module; when the flatness of the workpiece to be inspected is qualified, the workpiece to be inspected is marked with a second pattern by the coding module; for any workpiece to be inspected, when the workpiece to be inspected includes the first pattern and the second pattern, it is determined that the workpiece quality of the workpiece to be inspected is qualified.
[0007] In an embodiment of the present application, the visual inspection equipment also includes a display device, and the method also includes: for any workpiece to be inspected, the three-dimensional image, workpiece type, flatness data and contour data of each workpiece to be inspected are displayed in real time through the display device, and when the workpiece quality of the workpiece to be inspected is unqualified, the unqualified area is rendered.
[0008] In an embodiment of the present application, a point cloud segmentation algorithm is performed on a plurality of point cloud data to obtain a point cloud cluster corresponding to each workpiece to be detected, including: performing spatial division processing on the plurality of point cloud data based on an octree algorithm to screen key point cloud data belonging to the workpiece to be detected in the plurality of point cloud data; constructing a tree structure diagram based on a Euclidean clustering algorithm, and storing the key point cloud data in the tree structure diagram; using the key point cloud data as data to be clustered; selecting any one key point cloud data from the data to be clustered as target point cloud data, obtaining a plurality of key point cloud data adjacent to the target point cloud data based on the tree structure diagram, and determining the distance between the target point cloud data and each adjacent key point cloud data. The straight-line distance between the key point cloud data and the target point cloud data is determined as the same type of data as the target point cloud data; the same type of data and the target point cloud data are clustered into a point cloud cluster; the target point cloud data and the same type of data of the target point cloud data are marked as visited; all the point cloud data except the target point cloud data and the same type of data of the target point cloud data are re-used as the data to be clustered; the step of selecting any key point cloud data from the data to be clustered as the target point cloud data is performed again until all the key point cloud data are marked as visited; each point cloud cluster is segmented so that each workpiece to be detected forms an independent point cloud cluster.
[0009] In an embodiment of the present application, determining the workpiece type of each workpiece to be inspected based on the point cloud cluster includes: for any point cloud cluster, constructing a directed rectangular box for the point cloud cluster, and determining multiple vertex data of the directed rectangular box; determining the contour data of the workpiece to be inspected corresponding to each point cloud cluster based on the multiple vertex data of each directed rectangular box; and determining the workpiece type of each workpiece to be inspected based on the contour data of each workpiece to be inspected.
[0010] In an embodiment of the present application, determining the standard point cloud data corresponding to each workpiece type, and determining the contour shape variable of each workpiece to be detected based on the point cloud data of each workpiece to be detected and the standard point cloud data includes: after determining the workpiece type of each workpiece to be detected, matching the workpiece drawing corresponding to each workpiece type; for any workpiece to be detected, parsing the workpiece drawing of the workpiece to be detected into standard point cloud data for the workpiece to be detected, and performing point cloud data alignment processing on the point cloud data of the point cloud cluster corresponding to the workpiece to be detected and the standard point cloud data; performing an edge line feature extraction operation on the point cloud cluster to extract the endpoints of the high curvature line segments and the neighborhood depth discontinuity points as edge key points, and determining the three-dimensional space straight line for the workpiece to be detected based on the edge key points; determining the contour shape variable of each workpiece to be detected based on the three-dimensional space straight line and the standard point cloud data.
[0011] A second aspect of the present application provides a workpiece detection device, the device comprising: a memory configured to store instructions; The processor is configured to call the instructions from the memory and implement the workpiece detection method described above when executing the instructions.
[0012] A third aspect of the present application provides a visual inspection device, the device comprising: The workpiece detection device described above; Guide rails, fixed on the equipment body; A workstation workbench is fixed on the equipment body, connected to the incoming material area, and used to carry at least one workpiece to be inspected transmitted through the incoming material area; A plurality of line scanning profilers are mounted on a truss through a bracket, and the truss is used to carry the plurality of line scanning profilers and move them along the guide rail direction to scan the workpiece to be inspected.
[0013] In an embodiment of the present application, the device further includes: The coding module is fixed on the truss and is used to perform a first pattern coding mark on the workpiece to be detected when the outline size of the workpiece to be detected is qualified, and to perform a second pattern coding mark on the workpiece to be detected when the flatness of the workpiece to be detected is qualified; The display device is used to display the three-dimensional image, workpiece type, flatness data and contour data of each workpiece to be inspected in real time, and render the unqualified area when the workpiece quality of the workpiece to be inspected is unqualified.
[0014] A fourth aspect of the present application provides a machine-readable storage medium, on which instructions are stored. When the instructions are executed by a processor, the processor is configured to execute the above-mentioned workpiece detection method.
[0015] The above technical scheme provides a workpiece detection method, which is applied to visual inspection equipment, including: when the presence of a workpiece to be detected is detected at the detection position, multiple point cloud data of a preset detection area are obtained; the multiple point cloud data are processed by a point cloud segmentation algorithm to obtain a point cloud cluster corresponding to each workpiece to be detected; the workpiece type of each workpiece to be detected is determined according to the point cloud cluster; the standard point cloud data corresponding to each workpiece type is determined, and the contour shape variable of each workpiece to be detected is determined based on the point cloud data of each workpiece to be detected and the standard point cloud data; for each point cloud cluster, the point cloud data in each point cloud cluster are clustered to obtain the plane point cloud data belonging to the upper surface of the workpiece to be detected in each point cloud cluster, and the plane point cloud data in each point cloud cluster are fitted to determine the flatness data of each workpiece to be detected; thereby determining the workpiece quality of each workpiece to be detected based on the contour shape variable and flatness data of each workpiece to be detected. This method creates a dedicated intelligent visual inspection device for large-size inspection scenes through intelligence and integration, and based on the intelligent measurement of the three-dimensional size measurement of the workpiece by the device, the accurate detection of the workpiece quality of large-size workpieces is achieved.
[0016] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings: Figure 1 A schematic diagram of the structure of a visual inspection device according to an embodiment of the present application is shown; Figure 2 A schematic diagram of a process flow of a workpiece detection method according to an embodiment of the present application is schematically shown; Figure 3 A schematic diagram of a large-size workpiece according to an embodiment of the present application is schematically shown; Figure 4 A schematic diagram of transporting a workpiece to be inspected according to an embodiment of the present application is schematically shown; Figure 5 The internal structure diagram of a computer device according to an embodiment of the present application is schematically shown.
[0018] Description of Reference Numerals 101, guide rail; 102, workstation workbench; 103, a workpiece to be inspected; 104, a plurality of line scan profilers; 105, bracket; 106, truss; 107, inkjet coding module; 108, display device; 109, a first direction moving module; 110, a second direction moving module; 111, a turning pedal; 112, an electrical control module; 113, industrial computer; 114, calibration board. DETAILED DESCRIPTION
[0019] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0020] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), such directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.
[0021] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.
[0022] The workpiece detection method provided in this application can be applied to Figure 1 In the application environment shown in the figure, Figure 1 As shown, a visual inspection device is provided, the device comprising: The guide rail 101 is fixed on the equipment body; A workstation workbench 102 is fixed to the equipment body and connected to the incoming material area, and is used to carry at least one workpiece 103 to be inspected transmitted from the incoming material area; A plurality of line scan profilers 104 are mounted on a truss 106 via a bracket 105 . The truss 106 is used to carry the plurality of line scan profilers 104 and move along the guide rail 101 to scan the workpiece 103 to be inspected.
[0023] In the embodiments of the present application, Figure 1 The equipment shown also includes: The coding module 107 is fixed on the truss 106 and is used to perform a first pattern coding mark on the workpiece 103 to be detected when the outline size of the workpiece 103 to be detected is qualified, and to perform a second pattern coding mark on the workpiece 103 to be detected when the flatness of the workpiece 103 to be detected is qualified; The display device 108 is used to display the three-dimensional image, workpiece type, flatness data and contour data of each workpiece 103 to be inspected in real time, and render the unqualified area when the workpiece quality of the workpiece 103 to be inspected is unqualified; The first direction moving module 109 is used to control the truss 106 carrying the multiple line scanning profilers 104 to move along the guide rail 101 to scan the workpiece 103 to be inspected; The second direction moving module 110 is used to control the coding module 107 to move along the second direction to perform coding marking on the workpiece 103 to be inspected; The flip pedal 111 is used to ensure the safety of the user's operation and prevent foreign matter and dust from falling onto the guide rail 101; The electrical control module 112 includes an electrical control cabinet, a photoelectric sensor, a motor, a switching power supply, a PLC and electrical control elements. The electrical control module 112 is used to control the motion of the device; The industrial computer 113 is used to maintain the stable operation of the processor and to process and analyze the image data in real time; The calibration plate 114 is used to calibrate the motion distortion angle and position posture of the multi-view profiler.
[0024] Figure 2 The following schematically shows a flow chart of a workpiece detection method according to an embodiment of the present application. Figure 2 As shown, the embodiment of the present application provides a method for workpiece detection, which can be applied to Figure 1 The visual inspection device shown, the method may include the following steps.
[0025] Step 201 , when it is detected that there is a workpiece to be inspected at the inspection position, a plurality of point cloud data of a preset inspection area are obtained.
[0026] In the embodiment of the present application, it should be noted that, in this solution, the workpiece to be detected may refer to Figure 3For the large-size workpiece shown, the detection position may refer to a workstation workbench. The incoming material area is connected to the workstation workbench to transport the workpiece to be detected to the workstation workbench for detection, wherein the placement posture of each workpiece to be detected is consistent with the direction of the guide rail in the length direction. For example, taking the guide rail direction as the X direction, the length direction of each workpiece to be detected should be in the X direction, and the width direction should be in the Y direction. Among them, the workstation workbench in this solution can accommodate multiple workpieces to be detected and placed side by side at the same time, and the interval between each workpiece to be detected meets the requirements, for example, the interval can be set to 0.1m.
[0027] In this embodiment, it should be noted that when a workpiece to be inspected is detected on the workstation, a line scan profiler can be used to obtain multiple point cloud data of a preset area to be inspected. The line scan profiler may refer to a 3D line scan profiler, and multiple line scan profilers may be mounted on the truss of the visual inspection device in parallel along the Y direction. Specifically, Figure 4 As shown, a schematic diagram of the transportation of workpieces to be detected is provided. The incoming material area ① conveys the workpieces to be detected to the workstation workbench, and multiple line scanning profilers ② move in the direction of ③ to scan the preset detection area, wherein each workpiece to be detected has a corresponding QR code, and an intelligent code scanning module is set up above the incoming material area ①, which can be used to identify the QR code on each workpiece to be detected, so as to identify the code or model of each workpiece to be detected. The length direction of the workpiece to be detected is consistent with the direction of ③, and the preset detection area can refer to the area above the workstation workbench. It should be noted that the preset detection area should be able to cover the entire area of the upper surface of the workstation workbench to ensure that multiple line scanning profilers can scan all the workpieces to be detected. In this technical solution, multiple line scanning profilers can be set to cover a field of view of 10m in length in the X direction, and the X direction is the placement direction of the length of the workpiece to be detected, and can cover a field of view of 2m in length in the Y direction, and the Y direction is the direction in which multiple workpieces to be detected are placed side by side, that is, the detectable range of the preset detection area is a detection area of 2m*10m. After multiple line scanning profilers ② move in the direction of ③ to scan the preset detection area, the point cloud data of the preset detection area can be obtained. At the same time, after the workpiece to be inspected is scanned, it will be transferred to the next process ⑤ to continue the operation, and multiple line scanning profilers will return to the starting point position in the direction of ④ to scan the next round of workpieces to be inspected transmitted from the incoming area ①.
[0028] In this embodiment, it should be noted that a full-size calibration block is designed in this solution, and the rigid body posture of the multi-view line scan profiler is calibrated through three-dimensional control points, and the scanning data of multiple line scan profilers are integrated into the reference coordinate system through the calibration results. Then, based on the distortion correction and rigid body relationship calibration algorithm, the scanning data of the line scan profiler is subjected to distortion compensation, and the point cloud information of adjacent line scan profilers is spliced onto the point cloud information of the reference camera to form the overall point cloud data of the multi-line scan profiler scanning area.
[0029] Step 202 : Process the plurality of point cloud data using a point cloud segmentation algorithm to obtain a point cloud cluster corresponding to each workpiece to be inspected.
[0030] In an embodiment of the present application, after acquiring multiple point cloud data, the multiple point cloud data can be smoothed first to remove noise generated by scanning or other reasons, and then the multiple point cloud data can be processed by a point cloud segmentation algorithm to obtain a point cloud cluster corresponding to each workpiece to be inspected.
[0031] In an embodiment of the present application, a point cloud segmentation algorithm is performed on a plurality of point cloud data to obtain a point cloud cluster corresponding to each workpiece to be detected, including: performing spatial division processing on the plurality of point cloud data based on an octree algorithm to screen key point cloud data belonging to the workpiece to be detected in the plurality of point cloud data; constructing a tree structure diagram based on a Euclidean clustering algorithm, and storing the key point cloud data in the tree structure diagram; using the key point cloud data as data to be clustered; selecting any one key point cloud data from the data to be clustered as target point cloud data, obtaining a plurality of key point cloud data adjacent to the target point cloud data based on the tree structure diagram, and determining the distance between the target point cloud data and each adjacent key point cloud data. The straight-line distance between the key point cloud data and the target point cloud data is determined as the same type of data as the target point cloud data; the same type of data and the target point cloud data are clustered into a point cloud cluster; the target point cloud data and the same type of data of the target point cloud data are marked as visited; all the point cloud data except the target point cloud data and the same type of data of the target point cloud data are re-used as the data to be clustered; the step of selecting any key point cloud data from the data to be clustered as the target point cloud data is performed again until all the key point cloud data are marked as visited; each point cloud cluster is segmented so that each workpiece to be detected forms an independent point cloud cluster.
[0032] In this embodiment, it should be noted that the octree algorithm may refer to octree, which is a tree data structure used to describe three-dimensional space. Each node of the octree represents a volume element of a cube, and each node has eight child nodes. The volume elements represented by the eight child nodes added together are equal to the volume of the parent node. The octree is an extension of the quadtree in three-dimensional space. There are four quadrants in two dimensions and eight quadrants in three dimensions. The octree is mainly used for space division and nearest neighbor search. Since the collected point cloud data includes the entire preset detection area, the acquired point cloud data includes both the workpiece to be detected and the space area above the workstation workbench. In this technical solution, the set operation of the point cloud target can be quickly performed based on octree, and the point cloud search of the dense point cloud to be detected area can be quickly carried out according to the point cloud space division to screen the key point cloud data belonging to the workpiece to be detected in multiple point cloud data.
[0033] In this embodiment, it should be noted that the Euclidean clustering algorithm is a clustering algorithm based on the Euclidean distance height quantity, and the neighbor query algorithm based on KD-Tree is an important preprocessing method for accelerating the Euclidean clustering algorithm. Therefore, when processing the key point cloud data belonging to the workpiece to be detected, it is necessary to first construct a tree structure diagram and store the key point cloud data in the tree structure diagram. Among them, the tree structure diagram can refer to KD-Tree. The key point cloud data is used as the data to be clustered, and any key point cloud data is selected from the data to be clustered as the target point cloud data. Based on the tree structure diagram, multiple key point cloud data adjacent to the target point cloud data are obtained, and the straight-line distance between the target point cloud data and each adjacent key point cloud data is determined, and the key point cloud data whose straight-line distance is less than the first preset distance threshold is determined as the same type of the target point cloud data, and the same type of data and the target point cloud data are clustered into a point cloud cluster. For example, taking a point O in the key point cloud data as the target point cloud data, kdtree is used to find the P points closest to point O, and the distance from these P points to point O is determined. Points whose distance is less than the set clustering threshold x are all put into the set M, so that all point cloud data in the set M are clustered into a point cloud cluster. After clustering a point cloud cluster, the target point cloud data and the same type of target point cloud data are marked as visited, and all the key point cloud data except the target point cloud data and the same type of target point cloud data are re-classified as the data to be clustered, and the step of selecting any key point cloud data from the data to be clustered as the target point cloud data is performed again until all the key point cloud data are marked as visited, that is, all the point cloud data are clustered into different point cloud clusters. Finally, each point cloud cluster is divided into independent parts so that each workpiece to be detected forms an independent point cloud cluster.
[0034] Step 203: determining the workpiece type of each workpiece to be inspected according to the point cloud cluster.
[0035] In an embodiment of the present application, after obtaining a point cloud cluster corresponding to each workpiece to be inspected, point cloud boundary extraction can be performed on each point cloud cluster to obtain contour data of each workpiece to be inspected, so that the workpiece type of each workpiece to be inspected can be further identified based on the contour data of each workpiece to be inspected.
[0036] In an embodiment of the present application, determining the workpiece type of each workpiece to be inspected based on the point cloud cluster includes: for any point cloud cluster, constructing a directed rectangular box for the point cloud cluster, and determining multiple vertex data of the directed rectangular box; determining the contour data of the workpiece to be inspected corresponding to each point cloud cluster based on the multiple vertex data of each directed rectangular box; and determining the workpiece type of each workpiece to be inspected based on the contour data of each workpiece to be inspected.
[0037] In this embodiment, it should be noted that point cloud boundary extraction may refer to a process involving extracting boundary points from three-dimensional point cloud data, and these boundary points generally represent the external or internal contours of an object. There are various methods and techniques for extracting boundary points, including projection-based methods, image processing techniques, and direct extraction from three-dimensional point cloud data. Therefore, after segmenting and obtaining the point cloud cluster corresponding to each workpiece to be detected, point cloud boundary extraction is performed on each point cloud cluster to obtain the contour data of each workpiece to be detected. Specifically, in this technical solution, the contour data of each workpiece to be detected can be extracted based on the OBB bounding box algorithm combined with the edge detection algorithm. Among them, the OBB bounding box algorithm may refer to an algorithm for calculating the minimum bounding box of an object in three-dimensional space. The minimum bounding box is a rectangular box that can completely surround an object and has the smallest boundary, which can be used to describe the position, direction and size of an object. The OBB bounding box algorithm calculates the minimum bounding box by finding the main direction and boundary points of the object. The OBB bounding box algorithm is a rotatable rectangular box that can better adapt to the rotation and non-aligned shape of irregularly shaped objects, and can better fit and cover the actual shape of the object. Specifically, for any point cloud cluster, a directed rectangular box for the point cloud cluster is constructed based on the OBB bounding box algorithm to enclose the feature point cloud cluster of each workpiece to be detected in the rectangular box, thereby obtaining 8 vertices for the point cloud cluster. Based on the vertex information of each vertex, the feature information of the object in the bounding box can be quickly extracted, and the features of the bounding box are obtained by calculating the length, width, height of the bounding box, the distance and angle between the vertices, and the direction vector of the main axis. The three-dimensional morphology of each workpiece to be detected is obtained by statistical analysis of the OBB bounding box feature data, and then the contour data of each workpiece to be detected can be extracted in combination with the edge detection algorithm.
[0038] Step 204 : determining the standard point cloud data corresponding to each workpiece type, and determining the contour deformation amount of each workpiece to be inspected based on the point cloud data of each workpiece to be inspected and the standard point cloud data.
[0039] In the embodiment of the present application, after obtaining the workpiece type of each workpiece to be detected, the workpiece drawing of the corresponding model can be searched based on the workpiece type and workpiece code of each workpiece to be detected, and the standard point cloud data corresponding to each workpiece type can be determined through the workpiece drawing. The standard point cloud data of each workpiece to be detected and the point cloud cluster are subjected to feature extraction, and the contour shape variable of each workpiece to be detected can be obtained.
[0040] In an embodiment of the present application, determining the standard point cloud data corresponding to each workpiece type, and determining the contour shape variable of each workpiece to be detected based on the point cloud data of each workpiece to be detected and the standard point cloud data includes: after determining the workpiece type of each workpiece to be detected, matching the workpiece drawing corresponding to each workpiece type; for any workpiece to be detected, parsing the workpiece drawing of the workpiece to be detected into standard point cloud data for the workpiece to be detected, and performing point cloud data alignment processing on the point cloud data of the point cloud cluster corresponding to the workpiece to be detected and the standard point cloud data; performing an edge line feature extraction operation on the point cloud cluster to extract the endpoints of the high curvature line segments and the neighborhood depth discontinuity points as edge key points, and determining the three-dimensional space straight line for the workpiece to be detected based on the edge key points; determining the contour shape variable of each workpiece to be detected based on the three-dimensional space straight line and the standard point cloud data.
[0041] In the present technical solution, it should be noted that the workpiece drawing may refer to the CAD drawing of the workpiece. Therefore, after determining the workpiece type of each workpiece to be detected, the matching CAD drawing is searched according to the workpiece type and workpiece code of each workpiece to be detected, and the CAD drawing is parsed to obtain the standard point cloud data of each workpiece to be detected. Then, the ultra-high precision point cloud registration technology based on PCA and ICP realizes the alignment of the CAD point cloud data with the point cloud data of the point cloud cluster. Among them, PCA may refer to the principal component analysis technology, which aims to use the idea of dimensionality reduction to transform multiple indicators into a few comprehensive indicators. For each point cloud cluster, the point cloud cluster is subjected to edge straight line feature extraction operation to extract the endpoints of the high curvature line segment and the neighborhood depth discontinuity points as edge key points, and the edge key points are subjected to 2D straight line extraction according to the random sample consistency algorithm, and the 2D straight line is back-projected to the 3D space to obtain the 3D space straight line, so as to obtain the 3D space straight line of each workpiece to be detected. Calculate the relevant information of the 3D space straight line to the CAD standard point cloud data space to calculate the contour shape variable of each workpiece to be detected.
[0042] Step 205, for each point cloud cluster, clustering the point cloud data in each point cloud cluster to obtain the plane point cloud data belonging to the upper surface of the workpiece to be inspected in each point cloud cluster, and fitting the plane point cloud data in each point cloud cluster to determine the flatness data of each workpiece to be inspected.
[0043] In an embodiment of the present application, for each point cloud cluster, the point cloud data in each point cloud cluster can be further clustered based on the Euclidean clustering algorithm and the K-means algorithm to cluster the plane point cloud data belonging to the upper surface of the workpiece to be detected in each point cloud cluster. Among them, the K-means algorithm, also known as the k-means clustering algorithm, is an iterative clustering analysis algorithm, and its steps are to pre-divide the data into K groups, then randomly select K objects as the initial cluster centers, and then calculate the distance between each object and each seed cluster center, and assign each object to the cluster center closest to it. The cluster centers and the objects assigned to them represent a cluster. For each sample assigned, the cluster center of the cluster will be recalculated based on the existing objects in the cluster. This process will be repeated until a certain termination condition is met. The termination condition can be that no (or the minimum number) of objects are reassigned to different clusters, no (or the minimum number) of cluster centers change again, and the error sum of squares is locally minimized. After clustering to obtain the upper surface of each workpiece to be detected, a plane fitting algorithm can be used to calculate the flatness data of each area and the whole of each workpiece to be detected. Specifically, for each workpiece to be detected, the RANSAC random sample consensus algorithm combined with the least squares calculation method can be used to fit all points in the upper surface area of the workpiece to be detected to obtain a best fitting plane. The maximum and minimum values of the distances from all points in the upper surface area of the workpiece to be detected to the best fitting plane are calculated, and the absolute values are added to obtain the maximum flatness. It should be noted that the workpiece to be detected in the present technical solution is an extra-long plate-like workpiece, and whether the flatness of the workpiece is flat has a great influence on the accuracy of various parts on the subsequent processing equipment. If the workpiece flatness is poor, it will cause uneven force on the workpiece and poor appearance of the product. Therefore, quality inspection of the flatness of extra-long plate-like workpieces can effectively prevent workpieces that do not meet the precision from flowing into the next process to meet the requirements of later welding process, structural strength and product appearance.
[0044] Step 206 : determining the workpiece quality of each workpiece to be inspected based on the contour deformation amount and flatness data of each workpiece to be inspected.
[0045] In the embodiment of the present application, after the contour deformation amount and flatness data of each workpiece to be inspected are obtained, the workpiece quality of each workpiece to be inspected can be evaluated based on the contour deformation amount and the flatness data.
[0046] In an embodiment of the present application, determining the workpiece quality of each workpiece to be inspected based on the contour deformation amount and flatness data of each workpiece to be inspected includes: for any workpiece to be inspected, when the flatness per unit of the upper surface of the workpiece to be inspected is less than a first value, determining that the flatness of the workpiece to be inspected is qualified; when the contour deformation amount per unit of the upper surface of the workpiece to be inspected is less than a second value and the contour deformation amount of each edge of the upper surface is less than a third value, determining that the contour size of the workpiece to be inspected is qualified; for any workpiece to be inspected, when the flatness of the workpiece to be inspected is qualified and the contour size is qualified, determining that the workpiece quality of the workpiece to be inspected is qualified.
[0047] In the present embodiment, it should be noted that the first value, the second value and the third value may be set including but not limited to historical experience values and actual needs. For example, in the present technical solution, the first value may be 0.2 mm / m², the second value may be 1.5 mm / m, and the third value may be 3 mm / full length. Specifically, for any workpiece to be detected, when the flatness per square meter of the upper surface of the workpiece to be detected is less than 0.2 mm / m², the flatness of the workpiece to be detected can be considered to be qualified. For any workpiece to be detected, when the contour deformation variable per meter of the upper surface of the workpiece to be detected, that is, the difference between the actual contour size and the size set by the CAD model is less than 1.5 mm / m, and the contour deformation variable of each periphery of the upper surface of the workpiece to be detected is less than 3 mm / full length, the contour deformation variable of the workpiece to be detected can be considered to be qualified. For any workpiece to be detected, only when the flatness and contour deformation variables of the workpiece to be detected are qualified, the workpiece quality of the workpiece to be detected is considered to be qualified. When any one of the flatness or contour deformation of the workpiece to be inspected is unqualified, the workpiece to be inspected will be transferred to the corresponding unqualified product frame for manual correction.
[0048] In an embodiment of the present application, the visual inspection equipment includes a truss and a coding module fixed on the truss, and the method also includes: for any workpiece to be inspected, when the contour size of the workpiece to be inspected is qualified, the workpiece to be inspected is marked with a first pattern by the coding module; when the flatness of the workpiece to be inspected is qualified, the workpiece to be inspected is marked with a second pattern by the coding module; for any workpiece to be inspected, when the workpiece to be inspected includes the first pattern and the second pattern, it is determined that the workpiece quality of the workpiece to be inspected is qualified.
[0049] In this embodiment, it should be noted that the inkjet model may include an inkjet system device, an inkjet anti-collision structure, and a connecting plate. The first pattern may refer to a red circular pattern, and the second pattern may refer to a yellow triangular pattern. Therefore, for any workpiece to be inspected, if the flatness of the workpiece to be inspected is unqualified, the inkjet device will inkjet the red circular pattern mark on the workpiece to be inspected, and if the contour deformation of the workpiece to be inspected is unqualified, the inkjet device will inkjet the yellow triangular pattern mark on the workpiece to be inspected. Among them, only when the workpiece to be inspected has both red circular pattern and yellow triangular pattern inkjet, can the workpiece to be inspected be determined as a workpiece with qualified workpiece quality. When the workpiece to be inspected lacks any one of the inkjet codes, it will be transmitted to the corresponding guidance manual to perform the corresponding workpiece rework.
[0050] In an embodiment of the present application, the visual inspection equipment also includes a display device, and the method also includes: for any workpiece to be inspected, a three-dimensional image, workpiece type, flatness data and contour data of each workpiece to be inspected are displayed in real time through the display device, and when the workpiece quality of the workpiece to be inspected is unqualified, the unqualified area is rendered.
[0051] In the present embodiment, it should be noted that the display device includes a human-computer interaction interface and an image display interface. For any workpiece to be detected, the detection result of each workpiece to be detected will be displayed on the image display interface. Specifically, the displayed content may include the 3D image of each workpiece to be detected, the workpiece type, the flatness per square meter and the overall flatness and other flatness data, the contour data such as the contour deformation per meter and the contour deformation of each periphery. For any workpiece to be detected, when the workpiece quality of the workpiece to be detected is unqualified, the unqualified data will be marked in red, and at the same time, the unqualified area will also be rendered in the 3D model with a red circle and the position to the edge.
[0052] The above technical solution creates a dedicated intelligent visual inspection equipment for large-size inspection scenarios through intelligence and integration. Based on the intelligent measurement of the three-dimensional size of the workpiece by the equipment, it realizes the accurate detection of the workpiece quality of large-size workpieces.
[0053] This technical solution calibrates the rigid body posture of the multi-view line scan camera through three-dimensional control points, and integrates the scanning data of multiple line laser cameras into the reference coordinate system through the calibration results. It is compatible with multi-size workpiece detection and can achieve high-precision detection of super-long workpieces. It adopts advanced technologies such as point cloud segmentation technology based on 3D octree and Euclidean clustering, high-precision partition flatness calculation technology based on line scan point cloud, point cloud classification technology based on directed bounding box, and ultra-high-precision point cloud registration technology based on PCA and ICP, combined with CAD drawing analysis, workpiece flatness, contour shape variables, and workpiece size detection algorithms to achieve high-precision detection of multi-specification large-size workpieces. The integrated inkjet module, visual module and electrical control module create a multi-functional detection equipment that can be applied to scenarios such as flatness detection, workpiece sorting and size measurement, and can make up for the pain point that large-size workpieces cannot be automatically detected.
[0054] An embodiment of the present application provides a storage medium on which a program is stored. When the program is executed by a processor, the above-mentioned workpiece detection method is implemented.
[0055] The present application provides a workpiece detection device, the device comprising: a memory configured to store instructions; and The processor is configured to call the instructions from the memory and implement the workpiece detection method when executing the instructions.
[0056] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor A01, a network interface A02, a memory (not shown in the figure) and a database (not shown in the figure) connected through a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02 and a database (not shown in the figure). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A04. The database of the computer device is used to store workpiece detection method data. The network interface A02 of the computer device is used to communicate with an external terminal through a network connection. When the computer program B02 is executed by the processor A01, a workpiece detection method is implemented.
[0057] Those skilled in the art will understand that Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0058] An embodiment of the present application provides a device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, steps of a workpiece detection method are implemented.
[0059] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing the program of the steps of the initialization workpiece detection method.
[0060] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0061] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0062] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0063] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0064] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0065] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0066] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0067] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0068] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A workpiece detection method, characterized in that: Applied to visual inspection equipment, the method comprises: When it is detected that there is a workpiece to be inspected at the inspection position, multiple point cloud data of the preset inspection area are obtained; Processing the multiple point cloud data with a point cloud segmentation algorithm to obtain a point cloud cluster corresponding to each workpiece to be inspected; Determine the workpiece type of each workpiece to be inspected according to the point cloud cluster; Determine the standard point cloud data corresponding to each workpiece type, and determine the contour deformation amount of each workpiece to be inspected based on the point cloud data of each workpiece to be inspected and the standard point cloud data; For each point cloud cluster, cluster the point cloud data in each point cloud cluster to obtain the plane point cloud data belonging to the upper surface of the workpiece to be inspected in each point cloud cluster, and fit the plane point cloud data in each point cloud cluster to determine the flatness data of each workpiece to be inspected; The workpiece quality of each workpiece to be inspected is determined based on the contour deformation amount and flatness data of each workpiece to be inspected.
2. The workpiece detection method according to claim 1, characterized in that: Determining the workpiece quality of each workpiece to be inspected based on the contour deformation amount and flatness data of each workpiece to be inspected includes: For any workpiece to be inspected, when the flatness per unit of the upper surface of the workpiece to be inspected is less than a first value, it is determined that the flatness of the workpiece to be inspected is qualified; When the contour deformation amount per unit of the upper surface of the workpiece to be inspected is less than the second value and the contour deformation amount of each edge of the upper surface is less than the third value, it is determined that the contour size of the workpiece to be inspected is qualified; For any workpiece to be inspected, when the flatness and contour dimensions of the workpiece to be inspected are qualified, it is determined that the workpiece quality of the workpiece to be inspected is qualified.
3. The workpiece detection method according to claim 1, characterized in that: The visual inspection device includes a truss and a coding module fixed on the truss, and the method further includes: For any workpiece to be inspected, when the outline size of the workpiece to be inspected is qualified, the workpiece to be inspected is marked with a first pattern by the coding module; When the flatness of the workpiece to be inspected is qualified, the workpiece to be inspected is marked with a second pattern by the coding module; For any workpiece to be inspected, when the workpiece to be inspected includes a first pattern and a second pattern, it is determined that the workpiece quality of the workpiece to be inspected is qualified.
4. The workpiece detection method according to claim 1, characterized in that: The visual inspection device further includes a display device, and the method further includes: For any workpiece to be inspected, the display device displays the three-dimensional image, workpiece type, flatness data and contour data of each workpiece to be inspected in real time, and renders the unqualified area when the workpiece quality of the workpiece to be inspected is unqualified.
5. The workpiece detection method according to claim 1, characterized in that: The step of performing point cloud segmentation algorithm processing on the plurality of point cloud data to obtain a point cloud cluster corresponding to each workpiece to be detected includes: Performing spatial division processing on the plurality of point cloud data based on an octree algorithm to screen key point cloud data belonging to the workpiece to be inspected from the plurality of point cloud data; Constructing a tree structure diagram based on a Euclidean clustering algorithm, and storing the key point cloud data in the tree structure diagram; Using the key point cloud data as data to be clustered; Select any key point cloud data from the data to be clustered as target point cloud data, obtain multiple key point cloud data adjacent to the target point cloud data based on the tree structure diagram, and determine the straight-line distance between the target point cloud data and each adjacent key point cloud data; Determine the key point cloud data whose straight-line distance is less than a first preset distance threshold as the same type of data as the target point cloud data; Clustering the similar data and the target point cloud data into a point cloud cluster; Adding a visited mark to the target point cloud data and data of the same type as the target point cloud data; Re-using all key point cloud data except the target point cloud data and the same type of data as the target point cloud data as data to be clustered; The step of selecting any key point cloud data from the data to be clustered as the target point cloud data is performed again until all the key point cloud data are marked as visited; Each point cloud cluster is segmented so that each workpiece to be inspected forms an independent point cloud cluster.
6. The workpiece detection method according to claim 1, characterized in that: Determining the workpiece type of each workpiece to be detected according to the point cloud cluster includes: For any point cloud cluster, construct a directed rectangular frame for the point cloud cluster, and determine a plurality of vertex data of the directed rectangular frame; Determine contour data of the workpiece to be detected corresponding to each point cloud cluster based on multiple vertex data of each directed rectangular frame; The workpiece type of each workpiece to be inspected is determined based on the contour data of each workpiece to be inspected.
7. The workpiece detection method according to claim 1, characterized in that: The step of determining the standard point cloud data corresponding to each workpiece type, and determining the contour deformation amount of each workpiece to be detected based on the point cloud data of each workpiece to be detected and the standard point cloud data includes: After determining the workpiece type of each workpiece to be inspected, matching the workpiece drawing corresponding to each workpiece type; For any workpiece to be inspected, the workpiece drawing of the workpiece to be inspected is parsed into standard point cloud data for the workpiece to be inspected, and the point cloud data of the point cloud cluster corresponding to the workpiece to be inspected is aligned with the standard point cloud data; Performing an edge straight line feature extraction operation on the point cloud cluster to extract high curvature line segment endpoints and neighborhood depth discontinuity points as edge key points, and determining a three-dimensional space straight line for the workpiece to be detected based on the edge key points; The contour deformation amount of each workpiece to be inspected is determined based on the three-dimensional space straight line and the standard point cloud data.
8. A workpiece detection device, characterized in that: The device comprises: a memory configured to store instructions; A processor is configured to call the instructions from the memory and implement the workpiece detection method according to any one of claims 1 to 7 when executing the instructions.
9. A visual inspection device, characterized in that: The device comprises: The workpiece detection device according to claim 8; Guide rails, fixed on the equipment body; A workstation workbench is fixed on the equipment body, connected to the incoming material area, and used to carry at least one workpiece to be inspected transmitted through the incoming material area; A plurality of line scan profilers are mounted on a truss via a bracket, and the truss is used to carry the plurality of line scan profilers and move along a guide rail to scan a workpiece to be inspected.
10. The visual inspection device according to claim 9, characterized in that: The device also includes: A coding module is fixed on the truss, and is used to perform a first pattern coding mark on the workpiece to be detected when the outline size of the workpiece to be detected is qualified, and to perform a second pattern coding mark on the workpiece to be detected when the flatness of the workpiece to be detected is qualified; The display device is used to display the three-dimensional image, workpiece type, flatness data and contour data of each workpiece to be inspected in real time, and render the unqualified area when the workpiece quality of the workpiece to be inspected is unqualified.
11. A machine-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the processor is configured to perform the workpiece detection method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Injection molding impeller warping defect detection method based on point cloud feature comparison
CN111709934A
Engine combustion chamber depth dimension detection method and system based on visual detection
CN114241034A
Visual detection method for crimping quality of electric power fittings
CN115937098A
Point cloud cluster labeling method and device, storage medium and electronic device
CN117830998A
Container ship cargo hold three-dimensional point cloud feature data extraction and precision test method
CN118135214A
Cited By
Point cloud segmentation method and device, and storage medium
CN119919427A
A point cloud segmentation method, device and storage medium
CN119919427B
Intelligent detection method and system for workpiece machining quality of numerical control machine tool
CN120147680A
AFM scanning wafer structure analysis method based on image and data processing
CN120451141A
