Workpiece detection method, apparatus, and medium

Abstract

This application discloses a workpiece inspection method, apparatus, and medium, which are applied in the field of image processing technology. After preprocessing the image of the target workpiece to be inspected, this application performs two consecutive image alignments with different granularities on the preprocessed target image, thereby improving the difference accuracy between the image to be inspected and the reference image of the target workpiece, as well as the accuracy of the difference region between the two images. Based on the difference region, the abnormal features of the target workpiece are determined, which can improve the detection efficiency of whether the workpiece is abnormal and reduce the false detection rate.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a workpiece inspection method, workpiece inspection device, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] Currently, various electronic and mechanical devices are becoming increasingly sophisticated, requiring greater accuracy in the dimensions and shapes of their workpieces. Typically, after production, images of the workpiece are captured, and the integrity of the workpiece in the image is checked to determine if deformation exists. This method places high demands on the image capture angle, image clarity, and detection algorithm, resulting in low detection efficiency and a high false detection rate. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a workpiece inspection method, a workpiece inspection device, an electronic device, and a computer-readable storage medium to improve inspection speed and efficiency and avoid wear on the workpiece.

[0004] To address the aforementioned technical problems, this application provides a workpiece inspection method, comprising:

[0005] Acquire the image of the target workpiece to be inspected;

[0006] The image to be detected is preprocessed to obtain the target image;

[0007] The target image is compared with the reference image of the target workpiece to obtain a first comparison result; based on the first comparison result, at least one difference region in the target image that is inconsistent with the reference image is determined.

[0008] The image data of the difference region is converted into first point cloud data, and the image data of the region corresponding to the difference region in the reference image is converted into second point cloud data. The first point cloud data and the second point cloud data are compared to obtain a second comparison result. The difference region is optimized based on the second comparison result.

[0009] The abnormal characteristics of the target workpiece are determined based on the optimized difference region.

[0010] Optionally, acquiring the image to be inspected of the target workpiece includes:

[0011] Acquire a depth image of the target workpiece as the image to be inspected.

[0012] Optionally, the image to be detected is preprocessed to obtain the target image, including:

[0013] The image to be detected is filtered to obtain the filtered image;

[0014] The filtered image is then subjected to planar correction to obtain the target image.

[0015] Optionally, if the depth image of the target workpiece is used as the image to be detected, then the image to be detected is filtered to obtain a filtered image, including:

[0016] The pixel data of the image to be detected is split into low 8 bits and high 8 bits.

[0017] Use a histogram of size 255 to determine the first median of the lower 8 bits and the second median of the higher 8 bits;

[0018] The first median is used to perform median filtering on the corresponding lower 8 bits of data, and the second median is used to perform median filtering on the corresponding higher 8 bits of data to obtain the filtered image.

[0019] Optionally, planar correction is performed on the filtered image to obtain the target image, including:

[0020] A correction matrix is ​​constructed based on the carrier plane where the target workpiece is placed;

[0021] By using a correction matrix, the normal of the filtered image is made parallel to the normal of the vehicle plane, thus obtaining the target image.

[0022] Optionally, the target image is compared with a reference image of the target workpiece to obtain a first comparison result, including:

[0023] The target image is converted into an aligned image using a transformation matrix determined based on point-line matching relationships;

[0024] The difference data between the aligned image and the reference image is calculated to obtain the first comparison result.

[0025] Optionally, the difference data between the aligned image and the reference image is calculated, including:

[0026] The difference data between the aligned image and the reference image is calculated using the Gaussian weighted difference algorithm.

[0027] Optionally, the comparison of the first point cloud data and the second point cloud data includes:

[0028] The iterative nearest point algorithm is used to compare the cloud data of the first point and the cloud data of the second point.

[0029] Optionally, the abnormal features of the target workpiece are determined based on the optimized difference region, including:

[0030] Calculate the regional characteristics of the optimized difference region; the regional characteristics include at least one of the following: area, length, skeleton length, skeleton width, and roundness.

[0031] The abnormal characteristics of the target workpiece are determined based on the regional features.

[0032] Optionally, the abnormal features of the target workpiece are determined based on regional features, including:

[0033] If the region features include skeleton length and skeleton width, then the abnormal feature is determined to be a scratch based on the region features.

[0034] This application also provides a workpiece inspection device, including:

[0035] The acquisition module is used to acquire the image of the target workpiece to be inspected.

[0036] The preprocessing module is used to preprocess the image to be detected to obtain the target image;

[0037] The first detection module is used to compare the target image with the reference image of the target workpiece to obtain a first comparison result; and to determine at least one difference region in the target image that is inconsistent with the reference image based on the first comparison result.

[0038] The second detection module is used to convert the image data of the difference region into first point cloud data, convert the image data of the region corresponding to the difference region in the reference image into second point cloud data, compare the first point cloud data and the second point cloud data to obtain a second comparison result, and optimize the difference region based on the second comparison result.

[0039] The determination module is used to identify the abnormal features of the target workpiece based on the optimized difference region.

[0040] Optionally, the acquisition module is specifically used for:

[0041] Acquire a depth image of the target workpiece as the image to be inspected.

[0042] Optionally, the preprocessing module is specifically used for:

[0043] The image to be detected is filtered to obtain the filtered image;

[0044] The filtered image is then subjected to planar correction to obtain the target image.

[0045] Optionally, if the depth image of the target workpiece is used as the image to be detected, the preprocessing module is specifically used for:

[0046] The pixel data of the image to be detected is split into low 8 bits and high 8 bits.

[0047] Use a histogram of size 255 to determine the first median of the lower 8 bits and the second median of the higher 8 bits;

[0048] The first median is used to perform median filtering on the corresponding lower 8 bits of data, and the second median is used to perform median filtering on the corresponding higher 8 bits of data to obtain the filtered image.

[0049] Optionally, the preprocessing module is specifically used for:

[0050] A correction matrix is ​​constructed based on the carrier plane where the target workpiece is placed;

[0051] By using a correction matrix, the normal of the filtered image is made parallel to the normal of the vehicle plane, thus obtaining the target image.

[0052] Optionally, the first detection module is specifically used for:

[0053] The target image is converted into an aligned image using a transformation matrix determined based on point-line matching relationships;

[0054] The difference data between the aligned image and the reference image is calculated to obtain the first comparison result.

[0055] Optionally, the first detection module is specifically used for:

[0056] The difference data between the aligned image and the reference image is calculated using the Gaussian weighted difference algorithm.

[0057] Optionally, the second detection module is specifically used for:

[0058] The iterative nearest point algorithm is used to compare the cloud data of the first point and the cloud data of the second point.

[0059] Optionally, the module is specifically used for:

[0060] Calculate the regional characteristics of the optimized difference region; the regional characteristics include at least one of the following: area, length, skeleton length, skeleton width, and roundness.

[0061] The abnormal characteristics of the target workpiece are determined based on the regional features.

[0062] Optionally, the module is specifically used for:

[0063] If the region features include skeleton length and skeleton width, then the abnormal feature is determined to be a scratch based on the region features.

[0064] This application also provides an electronic device, including a memory and a processor, wherein:

[0065] The memory is used to store computer programs;

[0066] The processor is used to execute the computer program to implement the above-described workpiece inspection method.

[0067] This application also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described workpiece inspection method.

[0068] This application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned disclosed workpiece inspection method.

[0069] The workpiece inspection method provided in this application, after preprocessing the image of the target workpiece to be inspected, performs two consecutive image alignments at different granularities on the preprocessed target image, thereby improving the difference accuracy between the image to be inspected and the reference image of the target workpiece, that is, improving the accuracy of the difference region. Specifically, the two consecutive image alignments at different granularities include: comparing the target image with the reference image of the target workpiece to obtain a first comparison result; determining at least one difference region in the target image that is inconsistent with the reference image based on the first comparison result; then, based on this, converting the image data of the difference region into first point cloud data, converting the image data of the region corresponding to the difference region in the reference image into second point cloud data, comparing the first point cloud data and the second point cloud data, thereby obtaining a second comparison result based on the comparison of point cloud data of a smaller, finer-grained comparison region; then optimizing the difference region based on the second comparison result, thus improving the detection accuracy of the difference region, and determining the abnormal features of the target workpiece based on the optimized difference region, which not only improves the detection efficiency but also reduces the false detection rate.

[0070] In addition, this application also provides a workpiece inspection device, electronic device, computer-readable storage medium, and computer program product, which also have the above-mentioned beneficial effects. Attached Figure Description

[0071] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0072] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0073] Figure 2 This is a flowchart of a workpiece inspection method provided in an embodiment of this application;

[0074] Figure 3 This is a flowchart of another workpiece inspection method provided in an embodiment of this application;

[0075] Figure 4 This is a schematic diagram of the structure of a workpiece inspection device provided in an embodiment of this application. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0077] The steps in this application can be performed by a specified electronic device, the form of which is not limited; for example, it can be a general-purpose computing device such as a computer or server. Please refer to... Figure 1 , Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 100 may include a processor 101 and a memory 102, and may further include one or more of the following: a multimedia component 103, an information input / output (I / O) interface 104, and a communication component 105.

[0078] The processor 101 controls the overall operation of the electronic device 100 to complete all or part of the steps in the workpiece inspection method described above. The memory 102 stores various types of data to support the operation of the electronic device 100. This data may include, for example, instructions for any application or method operating on the electronic device 100, as well as application-related data. The memory 102 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as one or more of Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0079] Multimedia component 103 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 102 or transmitted via communication component 105. The audio component also includes at least one speaker for outputting audio signals. I / O interface 104 provides an interface between processor 101 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 105 is used for wired or wireless communication between electronic device 100 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of these. Therefore, the corresponding communication component 105 may include a Wi-Fi component, a Bluetooth component, and an NFC component.

[0080] The electronic device 100 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the workpiece inspection method proposed in this application.

[0081] In one specific embodiment, the aforementioned electronic device may include a workpiece loading component, a detection component, and a control component. The control component may be the aforementioned processor 101, and the detection component may be connected to other components, such as the control component, via the aforementioned I / O interface 104. The workpiece loading component is used to load workpieces awaiting detection and may specifically include multiple fixtures. The control component may specifically consist of a motion control card, an industrial computer, and a display. The industrial computer, through the motion control card, can control the detection component to move to the detection position of the workpiece, and can also control the workpieces corresponding to the multiple fixtures of the workpiece loading component to receive detection sequentially. Specifically, the workpiece is placed on the carrier plane of the workpiece loading component, and then the control component moves the workpiece to the detection position. The detection component uses image sensors to capture images online, and then performs calculations and analyses based on the workpiece detection method of this application to obtain a detection result indicating whether the workpiece has abnormal features. Finally, the workpiece detection result is sent back to the control component. The control component can further visualize the workpiece detection result, and when necessary, for example, if the detection result is not within the expected range (e.g., no abnormal features are detected), the control component can also issue an alert.

[0082] Please refer to Figure 2 , Figure 2 A flowchart illustrating a workpiece inspection method provided in this application embodiment. The method includes:

[0083] S201. Obtain the image of the target workpiece to be inspected.

[0084] It should be noted that the target workpiece can be placed on the carrier plane of the workpiece loading component. After the target workpiece is moved to the detection position by the control component connected to the workpiece loading component, the detection component controls the image sensor therein to perform online image acquisition, thereby capturing an image of the target workpiece to be detected. Then, the workpiece detection method of this embodiment is run to perform calculations and analysis to obtain the detection result of whether the target workpiece has abnormal features.

[0085] Generally, the image to be inspected for the target workpiece can be a traditional RGB image or a depth image. Each pixel value in the depth image represents the distance of that point from the camera (usually in millimeters or meters), focusing on the expression of spatial depth information. The distance information of each pixel value can be stored using a 16-bit short, which can be used to reconstruct the true three-dimensional point data of the image in space, thereby improving the realism of the image. Therefore, in one embodiment, acquiring the image to be inspected for the target workpiece includes: acquiring the depth image of the target workpiece as the image to be inspected, so as to provide a more realistic and effective data basis for subsequent inspection steps.

[0086] S202. Preprocess the image to be detected to obtain the target image.

[0087] To improve detection accuracy, images can be filtered for noise reduction and subjected to planar correction. Therefore, in one embodiment, preprocessing the image to be detected to obtain a target image includes: filtering the image to be detected to obtain a filtered image; and performing planar correction on the filtered image to obtain the target image.

[0088] In this embodiment, the filtering and denoising method can be selected from at least one of box filtering, mean filtering, Gaussian filtering, median filtering, and bilateral filtering as needed. Specifically, this embodiment provides an adaptive, fast median filtering method for depth images, which includes: determining the first median of the lower 8 bits and the second median of the higher 8 bits of each pixel data in the image to be detected using a histogram of size 255, thereby improving the calculation speed of the second median of the higher 8 bits of each pixel data, and thus improving the filtering speed. Therefore, in one implementation, if the depth image of the target workpiece is used as the image to be detected, the image to be detected is filtered to obtain a filtered image, including: splitting each pixel data of the image to be detected into lower 8 bits and higher 8 bits; determining the first median of the lower 8 bits and the second median of the higher 8 bits using a histogram of size 255; performing median filtering on the corresponding lower 8 bits using the first median, and performing median filtering on the corresponding higher 8 bits using the second median, to obtain the filtered image.

[0089] Since the carrier or camera may be tilted during image acquisition, causing the image plane to be unable to be perfectly parallel to the carrier plane, plane correction can be performed on the image. Therefore, in one embodiment, plane correction is performed on the filtered image to obtain the target image, including: constructing a correction matrix based on the carrier plane where the target workpiece is placed; using the correction matrix, making the normal of the filtered image parallel to the normal of the carrier plane, thereby making the image plane parallel to the carrier plane, thus obtaining the target image for subsequent detection steps.

[0090] S203. Compare the target image with the reference image of the target workpiece to obtain a first comparison result; determine at least one difference region in the target image that is inconsistent with the reference image based on the first comparison result.

[0091] S204. Convert the image data of the difference region into first point cloud data, convert the image data of the region corresponding to the difference region in the reference image into second point cloud data, compare the first point cloud data and the second point cloud data to obtain a second comparison result; optimize the difference region based on the second comparison result.

[0092] It should be noted that this embodiment provides two image alignment steps with different granularities, namely S203 and S204. In step S203, the target image is directly compared with the reference image of the target workpiece to determine their differences, namely, at least one difference region. In step S204, each difference region is further compared with the corresponding region of the reference image of the target workpiece to further refine the boundaries of each difference region and achieve optimization processing of each difference region. To achieve a finer granular comparison between each difference region and the corresponding region of the reference image, this embodiment implements this comparison process based on point cloud data. This embodiment assumes that the size, edges, etc., of the workpiece in the reference image are acceptable; that is, the reference image is an image obtained under ideal conditions of a qualified workpiece (i.e., a standard part), and this image is a traditional RGB image or a depth image, the image format of which needs to be consistent with the image format of the image to be detected.

[0093] In step S203, the target image is compared with a reference image of the target workpiece to obtain a first comparison result. This includes: converting the target image into an aligned image using a transformation matrix determined based on point-line matching relationships; and calculating the difference data between the aligned image and the reference image to obtain the first comparison result. Calculating the difference data between the aligned image and the reference image includes: using a Gaussian weighted difference algorithm to calculate the difference data between the aligned image and the reference image. For example, using a Gaussian weighted difference algorithm, weighting the 3×3 neighboring pixels of a single pixel, and aligning the image and the reference image based on the weighted result.

[0094] In step S204, the comparison of the first point cloud data and the second point cloud data includes: using the iterative nearest point algorithm to find the optimal transformation matrix by continuously iterating to compare the first point cloud data and the second point cloud data.

[0095] S205. Determine the abnormal characteristics of the target workpiece based on the optimized difference region.

[0096] This embodiment, after accurately defining the boundaries of each difference region and obtaining the optimized difference region, can calculate the area, length, skeleton length, skeleton width, and roundness of each optimized difference region. It can also classify each optimized difference region based on its area, length, skeleton length, skeleton width, and roundness. Since scratches on workpieces often appear as elongated strips, once it is confirmed that the regional characteristics of the optimized difference region include skeleton length and skeleton width, the presence of scratches within this optimized difference region can be determined based on the size of these skeleton length and width. In one embodiment, determining the abnormal characteristics of the target workpiece based on the optimized difference region includes: calculating the regional characteristics of the optimized difference region; the regional characteristics include at least one of area, length, skeleton length, skeleton width, and roundness; and determining the abnormal characteristics of the target workpiece based on the regional characteristics. In another embodiment, determining the abnormal characteristics of the target workpiece based on the regional characteristics includes: if the regional characteristics include skeleton length and skeleton width, then determining whether the abnormal characteristic is a scratch based on the regional characteristics. The Zhang-Suen algorithm can be used to calculate the size of features such as skeleton length and skeleton width.

[0097] This embodiment preprocesses the image of the target workpiece to be inspected, and then performs two consecutive image alignments at different granularities on the preprocessed target image. This improves the accuracy of the difference between the image to be inspected and the reference image of the target workpiece, that is, it improves the accuracy of the difference region. Specifically, the two consecutive image alignments at different granularities include: comparing the target image with the reference image of the target workpiece to obtain a first comparison result; determining at least one difference region in the target image that is inconsistent with the reference image based on the first comparison result; then, based on this, converting the image data of the difference region into first point cloud data, converting the image data of the region corresponding to the difference region in the reference image into second point cloud data, comparing the first point cloud data and the second point cloud data, thereby obtaining a second comparison result based on the comparison of point cloud data of a smaller, finer-grained comparison region; then optimizing the difference region based on the second comparison result, thus improving the detection accuracy of the difference region. Based on the optimized difference region, the abnormal features of the target workpiece are determined, which not only improves the detection efficiency but also reduces the false detection rate.

[0098] Please see Figure 3 Another workpiece inspection method includes:

[0099] Step 1: Acquire depth images.

[0100] The depth image acquired in this step is the image to be inspected of the target workpiece. The depth image is also called the distance image, which can record the distance information of each pixel in the image. In this embodiment, the distance information is stored with 16-bit short precision.

[0101] Step 2, high and low bit median filtering.

[0102] It should be noted that median filtering is a preprocessing step for images, effectively removing noise and obtaining the target image for subsequent steps. However, traditional median filtering algorithms suffer from low efficiency due to the excessive time required to calculate the median of 16-bit data. This is because for a 16-bit depth map, calculating a 16-bit histogram involves counting values ​​from 0 to 65535, which is time-consuming due to the large number of values. Therefore, this embodiment splits the 16-bit data of each pixel into two 8-bit data sets, one for high bits and one for low bits, so that the histograms of the two 8-bit data sets are both between 0 and 255. That is, the high 8 bits are calculated into a histogram of size 255, and the median of the high 8 bits is found based on this histogram. Similarly, the low 8 bits are also calculated into a histogram of size 255, and the median of the low 8 bits is found based on this histogram. This reduces the calculation time for the median of the high 8 bits.

[0103] Step 3, plane correction.

[0104] When acquiring depth images, considering the potential tilt of the vehicle or camera installation, direct image alignment can easily lead to significant misjudgments of differing areas due to shooting errors. This embodiment addresses this issue through a plane correction method. Plane correction is also a preprocessing step in the image. Its specific steps are as follows: a correction matrix is ​​constructed based on the plane normal vector parallel to the vehicle, correcting the plane normal vector of the vehicle plane so that the normal vector of the corrected image plane is parallel to the Z-axis direction vector of the vehicle plane's coordinate system. In other words, the depth image is corrected using the correction matrix to obtain a depth map parallel to the XOY plane of the vehicle plane's coordinate system.

[0105] Steps 2 and 3 in this embodiment correspond to S202 in the above embodiment, and the relevant content can be referred to each other.

[0106] Step 4: Create a template image (i.e., a baseline image).

[0107] The template image is an image of a standard part without deformation, which serves as a reference for subsequent detection of deformed areas in the workpiece being tested. The template image can be created by photographing the standard part using relevant techniques.

[0108] Step 5, quick alignment.

[0109] In this embodiment, the fast alignment is used to efficiently and accurately align the acquired depth image with the template image, i.e., the comparison process between the target image and the template image. Specifically, the comparison includes determining the transformation matrix based on the matching relationship between points and lines. This transformation matrix achieves fast image alignment, providing an accurate foundation for subsequent operations such as difference detection. To improve the accuracy and robustness of the matching, two edge lines can be found using calipers. The intersection point of the lines and the transformation matrix are calculated. The intersection point is denoted as P(x,y), and the angle between the line and the coordinate system is denoted as α. Then, the pose matrix is ​​obtained. Then the transformation matrix is ​​obtained: .

[0110] Step 6: Use the Gaussian weighted difference algorithm to initially detect the difference regions.

[0111] The difference region calculation method is implemented by subtracting the aligned depth image from the template image to determine at least one difference region where the aligned depth image and the template image are inconsistent. Considering the susceptibility to noise and alignment accuracy, this embodiment uses a weighted difference algorithm, subtracting the template image from its 3×3 neighboring pixels, and then using Gaussian weights to perform a weighted summation as the difference value. The Gaussian weight matrix is ​​as follows: Next, a threshold provided by the client is set to judge the difference value: if the difference value is within the threshold range, the area is determined to be a normal area; otherwise, it is determined to be a difference area. To efficiently represent the difference location, the difference area is stored using run-length encoding. This step is used for preliminary coarse alignment of the depth image and the template image.

[0112] Steps 5 and 6 in this embodiment correspond to S203 in the above embodiment, and the relevant content can be referred to each other.

[0113] Step 7: Local fine alignment and re-inspection of the difference areas.

[0114] To improve the accuracy of difference region detection, a difference region re-detection algorithm was designed. Specifically, it includes: performing connectivity processing on the difference regions, connecting adjacent difference pixels into connected regions, and filling these connected regions to eliminate holes; then, sequentially extracting each connected region, simultaneously converting the image data of that region in the template image and the graphical data in the image to be detected into point cloud data; employing the ICP (Iterative Closest Point) algorithm to achieve fine alignment between the template point cloud data and the point cloud data to be detected; and recalculating the difference regions to optimize their boundaries. This step aims to reduce errors that may arise during coarse alignment, further improving the accuracy of difference region detection.

[0115] This step compares the point cloud data of the image data for each difference region. It meticulously compares the point cloud data of the corresponding region in the template image with the point cloud data of the difference region determined in step 6 to optimize the borders of the difference regions. Step 7 in this embodiment corresponds to S204 in the above embodiment, and the relevant content can be referred to each other.

[0116] Step 8: Blob analysis detects the deformed areas.

[0117] Blob analysis can calculate features such as area, length, skeleton length, skeleton width, and roundness of the discrepancy region. Based on these features, different discrepancy regions can be classified and filtered. Specifically, the Zhang-Suen algorithm can extract the skeleton from any discrepancy region and calculate its length and width, thereby determining whether scratches exist within the region. The longest side of the skeleton is the skeleton length, and the average width of the skeleton's normal direction is the skeleton width. Based on these skeleton length and width, elongated deformations can be further filtered to determine whether they are scratches.

[0118] This step uses blob analysis to calculate features such as area, length, skeleton length, skeleton width, and roundness of each optimized difference region. Based on these features, region classification can be achieved, and abnormal features of the target workpiece can be determined. For example, after confirming that the regional features of the optimized difference region include skeleton length and skeleton width, the presence of scratches within this optimized difference region can be determined based on the size of these skeleton length and width, i.e., whether the target workpiece has scratches. Step 8 in this embodiment corresponds to S204 in the above embodiment, and the relevant content can be referred to accordingly.

[0119] As can be seen, the workpiece deformation defect detection method provided in this embodiment improves the detection speed by using high and low bit median filtering, as well as coarse and fine difference detection strategies; it improves the detection accuracy and robustness by using Gaussian weighted difference and re-inspection methods, enhances the ability to resist false detection, and can effectively handle deformed areas, providing a more accurate and efficient image difference detection solution for industrial production.

[0120] The workpiece inspection device provided in the embodiments of this application is described below. The workpiece inspection device described below and the workpiece inspection method described above can be referred to each other.

[0121] Please refer to Figure 4 , Figure 4 A schematic diagram of a workpiece inspection device provided in this application embodiment includes:

[0122] The acquisition module 401 is used to acquire the image to be inspected of the target workpiece;

[0123] Preprocessing module 402 is used to preprocess the image to be detected to obtain the target image;

[0124] The first detection module 403 is used to compare the target image with the reference image of the target workpiece to obtain a first comparison result; and to determine at least one difference region in the target image that is inconsistent with the reference image based on the first comparison result.

[0125] The second detection module 404 is used to convert the image data of the difference region into first point cloud data, convert the image data of the region corresponding to the difference region in the reference image into second point cloud data, compare the first point cloud data and the second point cloud data to obtain a second comparison result, and optimize the difference region based on the second comparison result.

[0126] The determination module 405 is used to determine the abnormal features of the target workpiece based on the optimized difference region.

[0127] In one implementation, the acquisition module is specifically used for:

[0128] Acquire a depth image of the target workpiece as the image to be inspected.

[0129] In one implementation, the preprocessing module is specifically used for:

[0130] The image to be detected is filtered to obtain the filtered image;

[0131] The filtered image is then subjected to planar correction to obtain the target image.

[0132] In one implementation, if the depth image of the target workpiece is used as the image to be detected, the preprocessing module is specifically used for:

[0133] The pixel data of the image to be detected is split into low 8 bits and high 8 bits.

[0134] Use a histogram of size 255 to determine the first median of the lower 8 bits and the second median of the higher 8 bits;

[0135] The first median is used to perform median filtering on the corresponding lower 8 bits of data, and the second median is used to perform median filtering on the corresponding higher 8 bits of data to obtain the filtered image.

[0136] In one implementation, the preprocessing module is specifically used for:

[0137] A correction matrix is ​​constructed based on the carrier plane where the target workpiece is placed;

[0138] By using a correction matrix, the normal of the filtered image is made parallel to the normal of the vehicle plane, thus obtaining the target image.

[0139] In one implementation, the first detection module is specifically used for:

[0140] The target image is converted into an aligned image using a transformation matrix determined based on point-line matching relationships;

[0141] The difference data between the aligned image and the reference image is calculated to obtain the first comparison result.

[0142] In one implementation, the first detection module is specifically used for:

[0143] The difference data between the aligned image and the reference image is calculated using the Gaussian weighted difference algorithm.

[0144] In one implementation, the second detection module is specifically used for:

[0145] The iterative nearest point algorithm is used to compare the cloud data of the first point and the cloud data of the second point.

[0146] In one implementation, the determining module is specifically used for:

[0147] Calculate the regional characteristics of the optimized difference region; the regional characteristics include at least one of the following: area, length, skeleton length, skeleton width, and roundness.

[0148] The abnormal characteristics of the target workpiece are determined based on the regional features.

[0149] In one implementation, the determining module is specifically used for:

[0150] If the region features include skeleton length and skeleton width, then the abnormal feature is determined to be a scratch based on the region features.

[0151] The computer-readable storage medium provided in the embodiments of this application is described below. The computer-readable storage medium described below can be referred to in correspondence with the workpiece inspection method described above.

[0152] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described workpiece inspection method.

[0153] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0154] The following describes a computer program product provided by an embodiment of this application. The computer program product described below can be referred to in conjunction with other embodiments described herein.

[0155] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned disclosed workpiece inspection method.

[0156] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the steps in any of the above embodiments.

[0157] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0158] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0159] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0160] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "include," "contain," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0161] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A workpiece inspection method, characterized in that, include: Acquire the image of the target workpiece to be inspected; The image to be detected is preprocessed to obtain the target image; The target image is compared with a reference image of the target workpiece to obtain a first comparison result; at least one difference region in the target image that is inconsistent with the reference image is determined based on the first comparison result; wherein, the target image is converted into an aligned image using a transformation matrix determined based on point-line matching relationship; the difference data between the aligned image and the reference image is calculated to obtain the first comparison result; The image data of the difference region is converted into first point cloud data, and the image data of the region corresponding to the difference region in the reference image is converted into second point cloud data. The first point cloud data and the second point cloud data are compared to obtain a second comparison result. The difference region is optimized based on the second comparison result. The iterative nearest point algorithm is used to find the optimal transformation matrix by iteratively comparing the first point cloud data and the second point cloud data. The abnormal features of the target workpiece are determined based on the optimized difference region.

2. The method according to claim 1, characterized in that, The image to be detected is preprocessed to obtain the target image, including: The image to be detected is filtered to obtain a filtered image; The filtered image is then subjected to planar correction to obtain the target image.

3. The method according to claim 2, characterized in that, Acquire a depth image of the target workpiece as the image to be detected; Accordingly, filtering the image to be detected to obtain a filtered image includes: The pixel data of the image to be detected is split into low 8 bits and high 8 bits. The first median of the lower 8 bits of data and the second median of the higher 8 bits of data are determined using a histogram of size 255; wherein, the lower 8 bits of data are plotted into a histogram of size 255 to find the first median of the lower 8 bits of data, and the higher 8 bits of data are plotted into a histogram of size 255 to find the second median of the higher 8 bits of data. The first median is used to perform median filtering on the corresponding lower 8 bits of data, and the second median is used to perform median filtering on the corresponding higher 8 bits of data to obtain the filtered image.

4. The method according to claim 3, characterized in that, The step of performing planar correction on the filtered image to obtain the target image includes: A correction matrix is ​​constructed based on the carrier plane on which the target workpiece is placed; Using the correction matrix, the normal of the filtered image is made parallel to the normal of the vehicle plane to obtain the target image.

5. The method according to claim 1, characterized in that, The calculation of the difference data between the aligned image and the reference image includes: The difference data between the aligned image and the reference image is calculated using a Gaussian weighted difference algorithm.

6. The method according to any one of claims 1 to 5, characterized in that, The step of determining the abnormal features of the target workpiece based on the optimized difference region includes: Calculate the regional features of the optimized difference region; the regional features include at least one of the following: area, length, skeleton length, skeleton width, and roundness. The abnormal characteristics of the target workpiece are determined based on the regional characteristics.

7. A workpiece inspection device, characterized in that, include: The acquisition module is used to acquire the image of the target workpiece to be inspected. The preprocessing module is used to preprocess the image to be detected to obtain the target image; A first detection module is used to compare the target image with a reference image of the target workpiece to obtain a first comparison result; determine at least one difference region in the target image that is inconsistent with the reference image based on the first comparison result; wherein, the target image is converted into an aligned image using a transformation matrix determined based on point-line matching relationship; and the difference data between the aligned image and the reference image is calculated to obtain the first comparison result. The second detection module is used to convert the image data of the difference region into first point cloud data, convert the image data of the region corresponding to the difference region in the reference image into second point cloud data, compare the first point cloud data and the second point cloud data to obtain a second comparison result; optimize the difference region according to the second comparison result; wherein, the iterative nearest point algorithm is used to continuously iterate to find the optimal transformation matrix to compare the first point cloud data and the second point cloud data; The determination module is used to determine the abnormal features of the target workpiece based on the optimized difference region.

8. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the method as described in any one of claims 1 to 6.

Images