Automatic workpiece inspection methods, devices, and related media based on 2D and 3D vision

Abstract

This invention discloses an automatic workpiece inspection method, apparatus, and related media based on 2D and 3D vision. The method includes acquiring 2D and 3D information of the workpiece according to control commands; preprocessing the 2D and 3D information to obtain information processing results; wherein the information processing results include standard workpiece data and workpiece data to be inspected; extracting features from the workpiece data to be inspected using a feature detection algorithm to obtain feature information data; comparing the feature information data with the standard workpiece data to obtain a comparison deviation value; determining whether the comparison deviation value is within a set error threshold; if yes, the workpiece is judged as qualified; if no, it is judged as unqualified, and a manual re-inspection is prompted. This invention, by processing the 2D and 3D information and then using a feature detection algorithm for feature extraction, enables the vision inspection system to simultaneously describe the geometric shape and surface features of the workpiece.

Description

Technical Field

[0001] This invention relates to the field of information detection technology, and in particular to automatic workpiece detection methods, devices and related media based on 2D and 3D vision. Background Technology

[0002] In modern manufacturing, the requirements for automated inspection and quality control of workpieces are becoming increasingly stringent. Traditional workpiece inspection methods typically use a single 2D or 3D vision system, but these methods have limitations in certain situations. 2D vision systems can only provide surface information of the workpiece and struggle to capture its three-dimensional shape and depth information, thus limiting their accuracy for complex or geometrically varied workpieces. 3D vision systems can acquire three-dimensional shape and depth information, but their feature extraction and comparative analysis are less effective for workpieces with surface features such as texture and color. Therefore, there is an urgent need for a solution that can simultaneously describe the geometric shape and surface features of a workpiece. Summary of the Invention

[0003] The present invention provides an automatic workpiece inspection method, device and related media based on 2D and 3D vision, which aims to solve the problem that the existing workpiece inspection system is single and cannot simultaneously describe the geometric shape and surface features of the workpiece.

[0004] In a first aspect, embodiments of the present invention provide an automatic workpiece detection method based on 2D and 3D vision, comprising:

[0005] The 2D and 3D information of the workpiece are obtained according to the control instructions.

[0006] The 2D and 3D information are preprocessed to obtain information processing results; wherein, the information processing results include standard workpiece data and workpiece data to be inspected;

[0007] Feature information data is obtained by extracting features from the workpiece data to be inspected using a feature detection algorithm.

[0008] The feature information data is compared with the standard workpiece data to obtain the comparison deviation value;

[0009] Determine whether the comparison deviation value is within the set error threshold. If yes, it is judged as a qualified workpiece; otherwise, it is judged as a non-qualified workpiece, and a manual re-inspection is required.

[0010] Secondly, embodiments of the present invention provide an automatic workpiece inspection device based on 2D and 3D vision, comprising:

[0011] The information acquisition unit is used to acquire the 2D and 3D information of the workpiece according to the control instructions.

[0012] An information processing unit is used to preprocess the 2D and 3D information to obtain information processing results; wherein, the information processing results include standard workpiece data and workpiece data to be inspected;

[0013] The information extraction unit is used to extract features from the workpiece data to be detected using a feature detection algorithm to obtain feature information data;

[0014] An information comparison unit is used to compare the feature information data with the standard workpiece data to obtain a comparison deviation value.

[0015] The information judgment unit is used to determine whether the comparison deviation value is within the set error threshold. If it is, the workpiece is judged to be qualified; if not, it is judged to be unqualified and prompts that manual re-inspection is required.

[0016] Thirdly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the workpiece automatic detection method based on 2D and 3D vision of the first aspect.

[0017] Fourthly, embodiments of the present invention provide a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the workpiece automatic detection method based on 2D and 3D vision of the first aspect.

[0018] This invention provides an automatic workpiece inspection method based on 2D and 3D vision, comprising: acquiring 2D and 3D information of the workpiece according to control instructions; preprocessing the 2D and 3D information to obtain information processing results; wherein the information processing results include standard workpiece data and workpiece data to be inspected; extracting features from the workpiece data to be inspected using a feature detection algorithm to obtain feature information data; comparing the feature information data with the standard workpiece data to obtain a comparison deviation value; determining whether the comparison deviation value is within a set error threshold; if yes, the workpiece is judged as qualified; if no, it is judged as unqualified and a manual re-inspection is required. This invention, by processing the 2D and 3D information and then extracting features using a feature detection algorithm, enables the vision inspection system to simultaneously describe the geometric shape and surface features of the workpiece.

[0019] The present invention also provides an automatic workpiece inspection device, computer equipment, and storage medium based on 2D and 3D vision, which have the same beneficial effects as described above. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating an automatic workpiece detection method based on 2D and 3D vision provided in an embodiment of the present invention;

[0022] Figure 2 This is another flowchart illustrating an automatic workpiece detection method based on 2D and 3D vision provided in an embodiment of the present invention.

[0023] Figure 3 This is a schematic block diagram of an automatic workpiece inspection device based on 2D and 3D vision, provided for an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0026] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0027] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0028] Please see below. Figure 1 , Figure 1The present invention provides a schematic diagram of an automatic workpiece detection process based on 2D and 3D vision, which specifically includes steps S101 to S105.

[0029] S101. Obtain the 2D and 3D information of the workpiece according to the control instructions;

[0030] S102. The 2D information and 3D information are preprocessed to obtain information processing results; wherein, the information processing results include standard workpiece data and workpiece data to be inspected;

[0031] S103. Use a feature detection algorithm to extract features from the workpiece data to be detected to obtain feature information data;

[0032] S104. Compare the feature information data with the standard workpiece data to obtain the comparison deviation value;

[0033] S105. Determine whether the comparison deviation value is within the set error threshold. If yes, it is determined to be a qualified workpiece; if no, it is determined to be an unqualified workpiece, and a manual re-inspection is required.

[0034] Combination Figure 2 As shown, in step S101, 2D and 3D information of the workpiece are acquired according to the control commands input by the user. 2D vision devices, such as cameras or image acquisition devices, are used to acquire the 2D information of the workpiece, which can be surface images or image sequences of the workpiece. 3D vision devices, such as laser scanners or structured light scanners, are used to acquire the 3D information of the workpiece, which can be point cloud data of the workpiece. The 2D and 3D information is used for subsequent feature extraction and comparative analysis. The combination of 2D and 3D information can improve the accuracy and efficiency of automatic workpiece detection.

[0035] In one embodiment, prior to step S101, the following steps are included:

[0036] Determine whether this is the first time the automatic detection system is used. If so, obtain the IP address and port of the camera that is capturing the workpiece, and establish a TCP connection between the automatic detection system and the camera using the IP address and port. If not, automatically establish a TCP connection between the automatic detection system and the camera.

[0037] In this embodiment, when the automatic detection system is started, the system checks the system's stored settings and configuration information to determine if it is the user's first time using the system. If it is the first time, the automatic detection system will request the user to provide the IP address and port information of the camera used to photograph the workpiece. The user can input the camera's IP address and port through the configuration interface or other interactive methods. The system will then use network communication protocols and library functions, such as Socket programming, to establish a TCP connection based on the IP address and port information provided by the user. In this way, the automatic detection system can communicate with the camera. If it is not the first time, the automatic detection system will automatically use the previously saved camera IP address and port information without requiring the user to re-enter them. The system will automatically establish a TCP connection based on the saved configuration information, simplifying the operation process and improving ease of use.

[0038] In one embodiment, step S101 includes:

[0039] The camera is activated using the control command to obtain 2D and 3D images respectively; texture information from the 2D image is mapped onto the 3D image, and pixel-level synchronization is performed to obtain image synchronization information; the brightness, contrast, hue, and saturation of the image synchronization information are adjusted to obtain color correction parameters; corner and edge features are extracted from the 2D and 3D images respectively, and feature alignment is performed using a local feature description matching algorithm to obtain feature alignment results; transformation matrices for rotation, translation, and affine transformations are calculated based on the feature alignment results to obtain feature alignment parameters; the color correction parameters and the feature alignment parameters are compensated and superimposed to obtain the image processing result; wherein, the image processing result includes the 2D and 3D information.

[0040] In this embodiment, based on the received control command, an automatic detection system is used to activate the camera, causing it to start operating and thereby acquiring 2D and 3D images of the workpiece. The 2D image can be captured by the camera's image sensor, while the 3D image can be acquired through methods such as laser scanning or structured light scanning. The texture information on the 2D image is mapped onto the 3D image, and pixel-level correlation synchronization is performed to obtain image synchronization information, providing a basis for subsequent feature alignment and image processing. The image synchronization information is then adjusted to improve image brightness, contrast, hue, and saturation to obtain color correction parameters. By adjusting these parameters, image quality and consistency can be improved, ensuring color information consistency between images for subsequent processing. Feature extraction and analysis: Corner and edge features are extracted from 2D and 3D images respectively, and feature alignment is performed using a local feature description matching algorithm to obtain feature alignment results. By extracting and matching corner and edge features in the workpiece image, alignment between 2D and 3D images can be achieved. The transformation matrices of rotation, translation, and affine transformation of the feature alignment results are calculated to obtain feature alignment parameters. The feature alignment parameters obtained by calculating the feature alignment results can describe the rotation, translation, and affine transformation relationships between 2D and 3D images. Finally, by superimposing and compensating the color correction parameters and feature alignment parameters, the image-processed 2D and 3D information can be obtained, making it more accurate and consistent, and providing a reliable data foundation for subsequent workpiece judgment.

[0041] In step S102, the acquired 2D and 3D information is preprocessed. The purpose of preprocessing is to eliminate noise, enhance image features, and adjust the image size and resolution to better extract and analyze the features of the workpiece. After preprocessing, the information processing results are obtained, including standard workpiece data and workpiece data to be inspected. Standard workpiece data consists of feature descriptions and reference data of known qualified workpieces, used for comparison and judgment with the workpiece to be inspected, in order to achieve the accuracy and qualification judgment of the workpiece.

[0042] In one embodiment, step S102 includes:

[0043] The 2D information is subjected to image denoising and image enhancement processing to obtain 2D preprocessed data; the 2D preprocessed data is subjected to color space conversion processing and 2D feature extraction to obtain first detection data; the 3D information is subjected to filtering processing and downsampling to obtain 3D preprocessed data; the 3D preprocessed data is subjected to data registration processing and 3D feature extraction to obtain second detection data; the first detection data and the second detection data are fused to obtain the workpiece data to be detected.

[0044] In this implementation, image denoising can employ filtering or denoising algorithms to reduce noise interference in the image; image enhancement can adjust parameters such as contrast, brightness, and sharpness to enhance image features and details. After image denoising and enhancement, 2D preprocessed data is obtained. This 2D preprocessed data is optimized image data with better quality and usability. Color space conversion is performed on the 2D preprocessed data to a suitable color space, followed by 2D feature extraction to extract corner points, edges, textures, and other feature information to obtain the first detection data. The acquired 3D information is then filtered to remove noise and smooth surfaces. Filtering algorithms such as average filtering and Gaussian filtering can be used. Downsampling is then performed to reduce the density and number of points in the 3D data, thus reducing computational complexity. Data registration is performed on the filtered and downsampled 3D information to align multiple 3D data sets, eliminating deviations caused by different viewpoints and deformations. Finally, 3D feature extraction is performed to extract the shape, curvature, normals, and other feature information of the workpiece surface to obtain the second detection data. The first and second detection data are fused together to combine 2D and 3D feature information to obtain the final workpiece data to be inspected. Feature fusion can be performed using methods such as weighted fusion and feature fusion algorithms to make full use of the advantages of 2D and 3D information and improve the accuracy and reliability of automatic workpiece detection.

[0045] In step S103, the workpiece data to be inspected includes 2D and 3D information after image processing and feature fusion, resulting in better consistency. Feature detection algorithms are used to extract features from the workpiece data. These algorithms can be traditional computer vision algorithms, such as Harris corner detection, SIFT, and SURF, or deep learning-based algorithms, such as convolutional neural networks (CNN). Feature detection algorithms can identify key feature points in the image and extract feature information from the workpiece data for subsequent feature comparison and workpiece determination. Of course, by selecting appropriate feature detection algorithms, different types of workpieces and inspection requirements can be adapted, improving the system's applicability and flexibility.

[0046] In one embodiment, step S103 includes:

[0047] A 3D point cloud recognition algorithm is used to construct features of corner points, edges, textures, and color histograms for the workpiece data to be inspected, respectively, to obtain a first numerical descriptor. Based on the first numerical descriptor, local pixel intensity analysis, gradient direction detection, and texture feature extraction are performed to obtain a second numerical descriptor. Feature matching is performed using the second numerical descriptor to search for similar feature descriptors in the second numerical descriptor. After a successful match, the final feature matching result is obtained. The perspective transformation pose is solved based on the feature matching result to evaluate the camera's rotation and translation, thereby obtaining the feature information data.

[0048] In this embodiment, a 3D point cloud recognition algorithm is used to extract features from the workpiece data to be inspected. This algorithm is a feature extraction algorithm for point cloud data, capable of identifying features such as corners, edges, textures, and color histograms within the point cloud. Using this algorithm, corner, edge, texture, and color histogram features are constructed from the workpiece data to obtain a first numerical descriptor. This first numerical descriptor is a series of numerical descriptors derived from the feature construction and is used to describe the key features of the workpiece. Based on the first numerical descriptor, local pixel intensity analysis, gradient direction detection, and texture feature extraction are performed on the workpiece data to obtain a second numerical descriptor. This second numerical descriptor further extracts local feature information of the workpiece, allowing for a more detailed description. The characteristics of the workpiece are described, and feature matching is performed using the second numerical descriptor. Similar feature descriptors are searched within the second numerical descriptor to obtain feature matching results. Feature matching can use matching algorithms such as nearest neighbor matching, RANSAC algorithm, PPF algorithm, etc., to find feature points in the workpiece data that are similar to known features. Based on the feature matching results, the perspective transformation pose is solved to evaluate the camera's rotation and translation. The perspective transformation pose solution can use pose estimation algorithms such as PnP algorithm to determine the workpiece's pose in the camera coordinate system. Based on the perspective transformation pose solution results, the final feature information data is obtained. The feature information data can help the automatic detection system more accurately evaluate the features and pose of the workpiece, improving the accuracy and efficiency of automatic workpiece detection.

[0049] In step S104, the feature information data includes feature descriptors extracted from the workpiece data to be inspected, which are used to describe the key features of the workpiece. The standard workpiece data is pre-determined workpiece data for comparison and has known feature information. The feature information data is compared with the standard workpiece data. The feature comparison can be based on similarity measurement methods, such as Euclidean distance, correlation coefficient, etc., to evaluate the degree of difference between the features of the workpiece data to be inspected and the features of the standard workpiece data. Through comparison, the feature comparison deviation value can be obtained. The comparison deviation value represents the degree of difference between the workpiece data to be inspected and the standard workpiece data.

[0050] In one embodiment, step S104 includes:

[0051] The first difference is obtained by statistically analyzing the relative differences between the feature information data and the dimensional data of the standard workpiece data; the second difference is obtained by statistically analyzing the relative differences between the feature information data and the angular data of the standard workpiece data; the third difference is obtained by statistically analyzing the relative differences between the feature information data and the shape data of the standard workpiece data; the fourth difference is obtained by statistically analyzing the relative differences between the feature information data and the color data of the standard workpiece data; the first difference, the second difference, the third difference, and the fourth difference are then weighted and averaged to obtain the comparison deviation value.

[0052] In this embodiment, the feature information data includes feature descriptors extracted from the workpiece data to be inspected. The feature information data and the standard workpiece data are compared in sequence using relative difference statistics on size, angle, shape, and color data. The difference statistics can be based on statistical methods such as mean, variance, and correlation to calculate the degree of difference between the feature information data and the standard workpiece data in different aspects. Through the difference statistics, we can obtain a first difference, a second difference, a third difference, and a fourth difference, representing the differences in size, angle, shape, and color, respectively. The first difference, the second difference, the third difference, and the fourth difference are then weighted and averaged to obtain the comparison deviation value. The weighted average can assign corresponding weights to each difference based on the importance of different factors.

[0053] In step S105, the comparison deviation value represents the degree of difference between the data of the workpiece to be inspected and the data of the standard workpiece. During implementation, an error threshold needs to be set in advance, which represents the maximum allowable difference. If the difference exceeds the threshold, the workpiece is considered unqualified. The conformity of the workpiece is judged by comparing the comparison deviation value with the error threshold. If the comparison deviation value is within the set error threshold, the workpiece is judged as qualified. If the comparison deviation value exceeds the set error threshold, the workpiece is judged as unqualified. For cases where the workpiece is judged as unqualified, the system will provide corresponding prompts, indicating that manual re-inspection is required. This can be achieved through interface display, sound reminders, etc., so that operators can perform necessary manual re-inspection operations in a timely manner and guide manual intervention to ensure the reliability and stability of product quality.

[0054] In one embodiment, the 2D information includes grayscale images and color images to provide two-dimensional information of the workpiece; the 3D information includes point cloud data and a three-dimensional model to provide three-dimensional information of the workpiece.

[0055] In this embodiment, the 2D information mainly includes grayscale images and color images. Firstly, grayscale images can acquire brightness information of the workpiece surface. Changes in brightness can reflect the surface's undulations or texture features. The Canny edge detection algorithm can be used to extract the surface's contour edge information, thereby obtaining the surface's shape and geometric features. Secondly, grayscale images can acquire texture information of the workpiece surface. The Gray-Level Co-occurrence Matrix (GLCM) texture analysis method can be used. GLCM is a statistical method used to describe the texture features of grayscale images. It captures the image's texture information by calculating the frequency and distribution of grayscale value pairs between pixels. Typically, GLCM calculates the co-occurrence matrix of pixel pairs in a specific direction, and then represents the image's texture features by calculating statistical features (such as contrast, correlation, energy, and entropy). Thirdly, color images can provide richer surface information. Color changes can reflect different materials, coatings, or surface states of the workpiece surface. Color space conversion (RGB to HSV) can be used to separate color information, and then color distribution analysis, color texture extraction, and other methods can be used to obtain the surface's color distribution and texture features. The second aspect of color images is the acquisition of texture information from the workpiece surface. Color and texture information in color images can be combined to extract richer texture features. Histogram of Oriented Gradients (HOG) can be applied (HOG is a method for image feature extraction, primarily used to describe and identify shape and texture features in images) to analyze texture information in color images and obtain surface texture features. The 3D information mainly includes point cloud data and a 3D model. Processing the acquired point cloud data can extract the geometric shape information of the workpiece; filtering and denoising the point cloud can remove noise and useless points, resulting in cleaner point cloud data; aligning multiple point cloud data sets yields a more complete and consistent workpiece geometry; and surface reconstruction using the point cloud data generates a smooth 3D model, making the workpiece's geometry more visual and easier to analyze.

[0056] In summary, by comprehensively utilizing the 2D and 3D information, this invention achieves high precision and accuracy, improves the efficiency of workpiece inspection, and reduces inspection costs. This invention can be widely applied in industrial production and other fields, and has good practical application value. Furthermore, this invention is not limited by the shape and size of the workpiece and can be applied to workpieces of various shapes and sizes.

[0057] Combination Figure 3 As shown, Figure 3 This is a schematic block diagram of an automatic workpiece inspection device based on 2D and 3D vision, provided by an embodiment of the present invention. The automatic workpiece inspection device 300 based on 2D and 3D vision includes:

[0058] The information acquisition unit 301 is used to acquire the 2D information and 3D information of the workpiece according to the control instructions.

[0059] The information processing unit 302 is used to preprocess the 2D information and 3D information to obtain information processing results; wherein, the information processing results include standard workpiece data and workpiece data to be inspected;

[0060] Information extraction unit 303 is used to extract features from the workpiece data to be detected using a feature detection algorithm to obtain feature information data;

[0061] The information comparison unit 304 is used to compare the feature information data with the standard workpiece data to obtain a comparison deviation value.

[0062] The information judgment unit 305 is used to determine whether the comparison deviation value is within the set error threshold. If it is, the workpiece is judged to be qualified; if not, it is judged to be unqualified and prompts that manual re-inspection is required.

[0063] In this embodiment, the information acquisition unit 301 acquires the 2D and 3D information of the workpiece according to the control instructions; the information processing unit 302 preprocesses the 2D and 3D information to obtain the information processing result; wherein, the information processing result includes standard workpiece data and workpiece data to be inspected; the information extraction unit 303 uses a feature detection algorithm to extract features from the workpiece data to be inspected to obtain feature information data; the information comparison unit 304 compares the feature information data with the standard workpiece data to obtain a comparison deviation value; the information judgment unit 305 judges whether the comparison deviation value is within the set error threshold. If it is, the workpiece is judged to be qualified; if not, it is judged to be unqualified and prompts that manual re-inspection is required.

[0064] In one embodiment, before the information acquisition unit 301, there is:

[0065] The establishment unit is used to determine whether the automatic detection system is being used for the first time. If so, it obtains the IP address and port of the camera that is capturing the workpiece and establishes a TCP connection between the automatic detection system and the camera using the IP address and port. If not, it automatically establishes a TCP connection between the automatic detection system and the camera.

[0066] In one embodiment, the information acquisition unit 301 includes:

[0067] An activation unit is used to activate the camera operation using the control command to obtain 2D and 3D images respectively.

[0068] The mapping unit is used to map the texture information on the 2D image onto the 3D image and perform pixel-level correlation synchronization to obtain image synchronization information.

[0069] The correction unit is used to adjust the image brightness, contrast, hue, and saturation of the image synchronization information respectively to obtain color correction parameters;

[0070] The alignment unit is used to extract corner and edge features from the 2D and 3D images respectively, and perform feature alignment using a local feature description matching algorithm to obtain the feature alignment result.

[0071] The transformation unit is used to calculate the transformation matrix for rotation, translation, and affine transformation based on the feature alignment result, and to obtain the feature alignment parameters.

[0072] The overlay unit is used to compensate and overlay the color correction parameters and the feature alignment parameters to obtain the image processing result; wherein the image processing result includes the 2D information and the 3D information.

[0073] In one embodiment, the information processing unit 302 includes:

[0074] The enhancement unit is used to perform image denoising and image enhancement processing on the 2D information to obtain 2D preprocessed data;

[0075] The color unit is used to perform color space conversion on the 2D preprocessed data and extract 2D features to obtain the first detection data;

[0076] The filtering unit is used to filter and downsample the 3D information to obtain 3D preprocessed data.

[0077] The registration unit is used to perform data registration processing on the 3D preprocessed data and extract 3D features to obtain the second detection data;

[0078] The fusion unit is used to perform feature fusion of the first detection data and the second detection data to obtain the workpiece data to be detected.

[0079] In one embodiment, the information extraction unit 303 includes:

[0080] The construction unit is used to construct the features of the corner point, edge, texture and color histograms of the workpiece data to be detected by using a three-dimensional point cloud recognition algorithm to obtain a first numerical descriptor.

[0081] The parsing unit is used to perform local pixel intensity parsing, gradient direction detection, and texture feature extraction based on the first numerical descriptor to obtain the second numerical descriptor.

[0082] The matching unit is used to perform feature matching using the second numerical descriptor to search for similar feature descriptors in the second numerical descriptor, and finally obtain the feature matching result after a successful match.

[0083] The pose unit is used to solve the perspective transformation pose based on the feature matching results to evaluate the camera's rotation and translation, and obtain the feature information data.

[0084] In one embodiment, the information comparison unit 304 includes:

[0085] A size unit is used to perform relative difference statistics between the feature information data and the size data of the standard workpiece data to obtain a first difference value;

[0086] An angle unit is used to perform relative difference statistics on the angle data of the feature information data and the standard workpiece data to obtain a second difference value.

[0087] A shape unit is used to perform relative difference statistics on the shape data of the feature information data and the standard workpiece data to obtain a third difference value;

[0088] The color unit is used to perform relative difference statistics between the feature information data and the color data of the standard workpiece data to obtain a fourth difference value.

[0089] The weighting unit is used to perform a weighted average of the first difference, the second difference, the third difference, and the fourth difference to obtain the comparison deviation value.

[0090] In one embodiment, the 2D information includes grayscale images and color images to provide two-dimensional information of the workpiece; the 3D information includes point cloud data and a three-dimensional model to provide three-dimensional information of the workpiece.

[0091] Since the embodiments of the apparatus and the embodiments of the method correspond to each other, please refer to the description of the embodiments of the method for the embodiments of the apparatus, which will not be repeated here.

[0092] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed, can perform the steps provided in the above embodiments. The storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0093] This invention also provides a computer device, which may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, it can implement the steps provided in the above embodiments. Of course, the computer device may also include various network interfaces, power supplies, and other components.

[0094] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0095] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. An automatic workpiece inspection method based on 2D and 3D vision, characterized in that, include: The 2D and 3D information of the workpiece are obtained according to the control instructions. The 2D and 3D information are preprocessed to obtain information processing results; wherein, the information processing results include standard workpiece data and workpiece data to be inspected; Feature information data is obtained by extracting features from the workpiece data to be inspected using a feature detection algorithm. The feature information data is compared with the standard workpiece data to obtain the comparison deviation value; Determine whether the comparison deviation value is within the set error threshold. If yes, it is judged as a qualified workpiece; if no, it is judged as a non-qualified workpiece, and a manual re-inspection is required. The step of acquiring 2D and 3D information of the workpiece according to control commands includes: activating camera operation using the control commands to obtain 2D and 3D images respectively; mapping texture information on the 2D image onto the 3D image and performing pixel-level correlation synchronization to obtain image synchronization information; adjusting the image brightness, contrast, hue, and saturation of the image synchronization information to obtain color correction parameters; extracting corner and edge features from the 2D and 3D images respectively, and performing feature alignment using a local feature description matching algorithm to obtain feature alignment results; calculating transformation matrices for rotation, translation, and affine transformations based on the feature alignment results to obtain feature alignment parameters; and compensating and superimposing the color correction parameters and the feature alignment parameters to obtain an image processing result; wherein the image processing result includes the 2D and 3D information.

2. The automatic workpiece inspection method based on 2D and 3D vision according to claim 1, characterized in that, Before acquiring the 2D and 3D information of the workpiece according to the control commands, the process includes: Determine whether this is the first time the automatic detection system is used. If so, obtain the IP address and port of the camera that is capturing the workpiece, and establish a TCP connection between the automatic detection system and the camera using the IP address and port. If not, automatically establish a TCP connection between the automatic detection system and the camera.

3. The automatic workpiece inspection method based on 2D and 3D vision according to claim 1, characterized in that, The preprocessing of the 2D and 3D information to obtain the information processing result includes: The 2D information is subjected to image denoising and image enhancement processes to obtain 2D preprocessed data; The 2D preprocessed data is subjected to color space conversion and 2D feature extraction to obtain the first detection data; The 3D information is filtered and downsampled to obtain 3D preprocessed data; The 3D preprocessed data is registered and 3D features are extracted to obtain the second detection data. The first detection data and the second detection data are fused to obtain the workpiece data to be detected.

4. The automatic workpiece inspection method based on 2D and 3D vision according to claim 1, characterized in that, The feature extraction of the workpiece data to be detected using a feature detection algorithm to obtain feature information data includes: A three-dimensional point cloud recognition algorithm is used to construct the features of the corner point, edge, texture and color histograms of the workpiece data to be detected, so as to obtain the first numerical descriptor. Based on the first numerical descriptor, local pixel intensity parsing, gradient direction detection, and texture feature extraction are performed respectively to obtain the second numerical descriptor; Feature matching is performed using the second numerical descriptor to search for similar feature descriptors in the second numerical descriptor. After a successful match, the final feature matching result is obtained. The perspective transformation pose is solved based on the feature matching results to evaluate the camera's rotation and translation, thereby obtaining the feature information data.

5. The automatic workpiece inspection method based on 2D and 3D vision according to claim 1, characterized in that, The step of comparing the feature information data with the standard workpiece data to obtain a comparison deviation value includes: The relative differences between the feature information data and the dimensional data of the standard workpiece data are statistically analyzed to obtain the first difference; The relative differences between the feature information data and the angle data of the standard workpiece data are statistically analyzed to obtain the second difference value; The relative differences between the feature information data and the shape data of the standard workpiece data are statistically analyzed to obtain a third difference value; The fourth difference value is obtained by statistically analyzing the relative differences between the feature information data and the color data of the standard workpiece data. The first difference, the second difference, the third difference, and the fourth difference are weighted and averaged to obtain the comparison deviation value.

6. The automatic workpiece inspection method based on 2D and 3D vision according to claim 1, characterized in that, The 2D information includes grayscale images and color images, used to provide two-dimensional information about the workpiece; the 3D information includes point cloud data and a three-dimensional model, used to provide three-dimensional information about the workpiece.

7. An automatic workpiece inspection device based on 2D and 3D vision, characterized in that, include: The information acquisition unit is used to acquire the 2D and 3D information of the workpiece according to the control instructions. An information processing unit is used to preprocess the 2D and 3D information to obtain information processing results; wherein, the information processing results include standard workpiece data and workpiece data to be inspected; The information extraction unit is used to extract features from the workpiece data to be detected using a feature detection algorithm to obtain feature information data; An information comparison unit is used to compare the feature information data with the standard workpiece data to obtain a comparison deviation value. The information judgment unit is used to determine whether the comparison deviation value is within the set error threshold. If it is, the workpiece is judged to be qualified; if not, it is judged to be unqualified and prompts that manual re-inspection is required. The information acquisition unit is specifically configured to: activate the camera operation using the control command to obtain 2D and 3D images respectively; map the texture information on the 2D image onto the 3D image and perform pixel-level correlation synchronization to obtain image synchronization information; adjust the image brightness, contrast, hue, and saturation of the image synchronization information respectively to obtain color correction parameters; extract corner and edge features from the 2D and 3D images respectively, and perform feature alignment using a local feature description matching algorithm to obtain feature alignment results; calculate the transformation matrix of rotation, translation, and affine transformation based on the feature alignment results to obtain feature alignment parameters; and compensate and superimpose the color correction parameters and the feature alignment parameters to obtain an image processing result; wherein the image processing result includes the 2D and 3D information.

8. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the automatic workpiece detection method based on 2D and 3D vision as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the automatic workpiece detection method based on 2D and 3D vision as described in any one of claims 1 to 6.

Images