A workpiece detection method, device, apparatus and medium

Abstract

This application discloses a workpiece inspection method, apparatus, equipment, and medium, applied in the field of image processing technology. This application can accurately capture the three-dimensional spatial morphology information of a workpiece using a depth map, and then generate a three-dimensional test image through denoising processing, effectively filtering out noise during the depth acquisition process; by extracting row and column pixel change information from the three-dimensional test image, a three-dimensional texture map is generated, more comprehensively capturing three-dimensional spatial defects of the workpiece; subsequently, a defect detection model is used to detect the three-dimensional texture map, improving detection efficiency and accuracy; after obtaining the detection results, defect quantification is further achieved, such as accurately outputting key parameters like the location, size, and depth of defects. This not only enables the detection of minute defects on the workpiece, improving detection accuracy and reducing the missed detection rate and false detection rate, but also provides precise data support for workpiece quality grading, defect cause analysis, and repair plan formulation.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a workpiece inspection method, workpiece inspection device, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] Currently, various electronic and mechanical devices are becoming increasingly sophisticated, and the workpieces on them also require corresponding precision machining. Typically, after production, images of the workpiece are taken, and the workpiece's quality is determined by detecting scratches, deformations, etc., in the images. This method places high demands on the shooting angle, image clarity, and detection algorithms. Minor deformations, micro-dents, and scratches on the workpiece may be difficult to detect due to unclear images, leading to missed or false detections. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a workpiece inspection method, workpiece inspection device, electronic device, computer-readable storage medium and computer program product, so as to detect minute defects such as minute deformation, minute indentation, and minute scratches on workpieces, improve inspection accuracy and reduce the missed detection rate and false detection rate.

[0004] To solve the above-mentioned technical problems, this application provides a workpiece inspection method, including:

[0005] Obtain the depth map of the target workpiece;

[0006] The depth map is denoised to obtain the 3D map to be measured.

[0007] Determine the row pixel change information and column pixel change information in the 3D image to be tested, and generate the corresponding 3D texture map based on the row pixel change information and column pixel change information;

[0008] Defect detection results of the target workpiece are obtained by using a defect detection model to perform defect detection on a 3D texture map.

[0009] The defects of the target workpiece are quantified based on the defect detection results.

[0010] Optionally, the depth map is denoised to obtain a 3D image to be measured, including:

[0011] After setting the height of invalid pixels in the depth map to zero, a filtering window is set to filter the row and column pixels in the depth map in parallel to obtain the 3D test image.

[0012] Optionally, determine the row pixel change information and column pixel change information in the 3D image to be measured, including:

[0013] By completing the row and column invalid pixels in the 3D image to be tested, a 3D completed image is obtained.

[0014] Calculate the gradients of row and column pixels in the 3D completed image to obtain row gradient values ​​and column gradient values.

[0015] Row pixel change information is determined based on row gradient values, and column pixel change information is determined based on column gradient values.

[0016] Optionally, a corresponding 3D texture map is generated based on row pixel change information and column pixel change information, including:

[0017] The row pixel change information and column pixel change information are normalized respectively, and a three-dimensional texture map is generated based on the normalization results.

[0018] Optionally, the defects in the target workpiece are quantified based on the defect detection results, including:

[0019] The number of pixels in any defect region included in the defect detection results;

[0020] The area of ​​the current defect region is determined based on the number of pixels;

[0021] Mark the area of ​​the current defect region and the corresponding defect type in the defect detection results in the current defect region.

[0022] Optionally, the defects in the target workpiece are quantified based on the defect detection results, including:

[0023] Morphological expansion is performed on any defect region included in the defect detection results to obtain the coverage area that covers the current defect region;

[0024] Within the coverage area, identify the expanded area excluding the current defective area;

[0025] Plane fitting is performed based on the point cloud data within the expansion region to obtain the fitted plane;

[0026] Calculate the distance between the point cloud data within the current defect area and the fitted plane;

[0027] The defect height of the current defect area is determined based on distance.

[0028] Optionally, the distance between the point cloud data within the current defect region and the fitted plane is calculated, including:

[0029] Calculate the distance between each point cloud data point within the current defect area and the fitted plane;

[0030] Accordingly, the defect height of the current defect region is determined based on distance, including:

[0031] The distances between each point cloud data point and the fitted plane are sorted in descending order to obtain the target sequence;

[0032] The defect height of the current defect region is determined based on the mean of the first N elements in the target sequence.

[0033] Optionally, the defects in the target workpiece are quantified based on the defect detection results, including:

[0034] If the defect type corresponding to any defect area in the defect detection result is a scratch, then the scratch is skeletonized to obtain the skeleton diagram of the scratch.

[0035] Perform a depth-first search on the skeleton graph of the scratch to obtain the longest connected skeleton in the skeleton graph of the scratch.

[0036] Measure the length of the longest connected skeleton as the length of the scratch.

[0037] This application also provides a workpiece inspection device, including:

[0038] The acquisition module is used to acquire the depth map of the target workpiece;

[0039] The denoising module is used to denoise the depth map to obtain a 3D image to be measured.

[0040] The mapping module is used to determine the row pixel change information and column pixel change information in the 3D image to be tested, and to generate the corresponding 3D texture map based on the row pixel change information and column pixel change information.

[0041] The detection module is used to perform defect detection on the 3D texture map using a defect detection model to obtain the defect detection results of the target workpiece.

[0042] The quantization module is used to quantify the defects of the target workpiece based on the defect detection results.

[0043] Optionally, the noise reduction module is specifically used for:

[0044] After setting the height of invalid pixels in the depth map to zero, a filtering window is set to filter the row and column pixels in the depth map in parallel to obtain the 3D test image.

[0045] Optionally, the mapping module is specifically used for:

[0046] By completing the row and column invalid pixels in the 3D image to be tested, a 3D completed image is obtained.

[0047] Calculate the gradients of row and column pixels in the 3D completed image to obtain row gradient values ​​and column gradient values.

[0048] Row pixel change information is determined based on row gradient values, and column pixel change information is determined based on column gradient values.

[0049] Optionally, the mapping module is specifically used for:

[0050] The row pixel change information and column pixel change information are normalized respectively, and a three-dimensional texture map is generated based on the normalization results.

[0051] Optionally, the quantization module is specifically used for:

[0052] The number of pixels in any defect region included in the defect detection results;

[0053] The area of ​​the current defect region is determined based on the number of pixels;

[0054] Mark the area of ​​the current defect region and the corresponding defect type in the defect detection results in the current defect region.

[0055] Optionally, the quantization module is specifically used for:

[0056] Morphological expansion is performed on any defect region included in the defect detection results to obtain the coverage area that covers the current defect region;

[0057] Within the coverage area, identify the expanded area excluding the current defective area;

[0058] Plane fitting is performed based on the point cloud data within the expansion region to obtain the fitted plane;

[0059] Calculate the distance between the point cloud data within the current defect area and the fitted plane;

[0060] The defect height of the current defect area is determined based on distance.

[0061] Optionally, the quantization module is specifically used for:

[0062] Calculate the distance between each point cloud data point within the current defect area and the fitted plane;

[0063] The distances between each point cloud data point and the fitted plane are sorted in descending order to obtain the target sequence;

[0064] The defect height of the current defect region is determined based on the mean of the first N elements in the target sequence.

[0065] Optionally, the quantization module is specifically used for:

[0066] If the defect type corresponding to any defect area in the defect detection result is a scratch, then the scratch is skeletonized to obtain the skeleton diagram of the scratch.

[0067] Perform a depth-first search on the skeleton graph of the scratch to obtain the longest connected skeleton in the skeleton graph of the scratch.

[0068] Measure the length of the longest connected skeleton as the length of the scratch.

[0069] This application also provides an electronic device, including a memory and a processor, wherein:

[0070] The memory is used to store computer programs;

[0071] The processor is used to execute the computer program to implement the above-described workpiece inspection method.

[0072] This application also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described workpiece inspection method.

[0073] This application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned disclosed workpiece inspection method.

[0074] As can be seen, this application can accurately capture the three-dimensional spatial morphology information of a workpiece using a depth map, and then generate a three-dimensional test image through noise reduction processing, effectively filtering out environmental interference and other noise during the depth acquisition process. Subsequently, by extracting the row and column pixel change information of the three-dimensional test image, a three-dimensional texture map is generated, transforming the subtle surface irregularities (such as minor deformations, minor indentations, minor scratches, and other minor defects) and structural anomalies of the workpiece into texture features that can be recognized by the model. Compared with traditional two-dimensional detection, it can more comprehensively capture three-dimensional spatial defects of the workpiece, especially suitable for detection scenarios involving subtle surface defects and internal hidden defects. Further, a defect detection model is used to detect the three-dimensional texture map, relying on the model's autonomous learning and accurate recognition capabilities to significantly improve detection efficiency and accuracy. After obtaining the detection results, defect quantification is further achieved, such as accurately outputting key parameters like the location, size, and depth of the defect, rather than simply determining whether a defect exists. This not only allows for the detection of minor defects such as minor deformations, minor indentations, and minor scratches on the workpiece, improving detection accuracy and reducing the missed detection rate and false detection rate, but also provides accurate data support for workpiece quality grading, defect cause analysis, and repair plan formulation.

[0075] In addition, this application also provides a workpiece inspection device, electronic device, computer-readable storage medium, and computer program product, which also have the above-mentioned beneficial effects. Attached Figure Description

[0076] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0077] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0078] Figure 2 This is a flowchart of a workpiece inspection method provided in an embodiment of this application;

[0079] Figure 3 A comparison diagram of a three-dimensional texture map and the original three-dimensional depth map provided for embodiments of this application;

[0080] Figure 4 This is a flowchart of another workpiece inspection method provided in an embodiment of this application;

[0081] Figure 5 This is a schematic diagram of the structure of a workpiece inspection device provided in an embodiment of this application. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0083] The steps in this application can be performed by a specified electronic device, the form of which is not limited; for example, it can be a general-purpose computing device such as a computer or server. Please refer to... Figure 1 , Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 100 may include a processor 101 and a memory 102, and may further include one or more of the following: a multimedia component 103, an information input / output (I / O) interface 104, and a communication component 105.

[0084] The processor 101 controls the overall operation of the electronic device 100 to complete all or part of the steps in the workpiece inspection method described above. The memory 102 stores various types of data to support the operation of the electronic device 100. This data may include, for example, instructions for any application or method operating on the electronic device 100, as well as application-related data. The memory 102 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as one or more of Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0085] Multimedia component 103 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 102 or transmitted via communication component 105. The audio component also includes at least one speaker for outputting audio signals. I / O interface 104 provides an interface between processor 101 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 105 is used for wired or wireless communication between electronic device 100 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of these. Therefore, the corresponding communication component 105 may include a Wi-Fi component, a Bluetooth component, and an NFC component.

[0086] The electronic device 100 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the workpiece inspection method proposed in this application.

[0087] In one specific embodiment, the aforementioned electronic device may include a material loading component, a detection component, and a control component. The control component may be the aforementioned processor 101, and the detection component may be connected to other components, such as the control component, via the aforementioned I / O interface 104. The material loading component is used to load materials (i.e., target workpieces) awaiting workpiece inspection, and may specifically include multiple fixtures. The control component may specifically consist of a motion control card, an industrial computer, and a display. The industrial computer, through the motion control card, can control the detection component to move to the material detection position, and can also control the materials corresponding to the multiple fixtures of the material loading component to be sequentially inspected. Specifically, the material is placed on the material loading component, and then the control component moves the material to the detection position. The detection component captures images by controlling a camera, etc., and then performs calculations and analyses based on the workpiece inspection method of this application to obtain workpiece defect detection results and related quantitative data. Finally, the workpiece defect detection results and related quantitative data are sent back to the control component. The control component can further visualize the workpiece inspection results, and when necessary, for example, if the defect quantification size is outside the tolerance range of the material, the control component can also issue an alert.

[0088] Please refer to Figure 2 , Figure 2 A flowchart illustrating a workpiece inspection method provided in this application embodiment. The method includes:

[0089] S201. Obtain the depth map of the target workpiece.

[0090] It should be noted that the target workpiece can be placed on the carrier plane of the workpiece loading component. After the target workpiece is moved to the detection position by the control component connected to the workpiece loading component, the detection component controls the 3D camera to perform online image acquisition, thereby obtaining a depth map of the target workpiece. Then, the workpiece detection method of this embodiment is run for calculation and analysis to obtain the defect detection results and related quantitative data of the workpiece. For example, various specific values ​​such as defect area, defect height, and defect length can be calculated. Using these specific values, the size and degree of defect can be described more accurately, which is beneficial for measuring workpiece quality.

[0091] Generally, each pixel value in a depth image represents the distance of that point from the camera (usually in millimeters or meters), focusing on the representation of spatial depth information. The distance information for each pixel value can be stored as a 16-bit short, from which the true 3D point data in space can be reconstructed, enhancing the realism of the image. More specifically, the depth map data format consists of two channels: a height channel and a marker channel. The height channel is stored as a 16-bit integer; the marker channel is stored as a Boolean type, with true indicating validity and false indicating invalidity.

[0092] S202. Denoise the depth map to obtain the three-dimensional map to be tested.

[0093] A depth image stores height information. Due to the imaging principle of a camera, invalid values ​​exist in the height values. This embodiment can filter out these invalid values ​​to prevent them from interfering with subsequent image processing. In one implementation, denoising the depth image to obtain a 3D test image includes: setting the height of invalid pixels in the depth image to zero, setting a filtering window, and filtering row and column pixels in the depth image in parallel to obtain the 3D test image. Parallel filtering of row and column pixels can improve filtering efficiency. Specifically, the process of sliding window summation for row pixels includes: first calculating the row neighborhood sum (pixel sum within the sliding window) of each pixel in the first row of the image; then, the neighborhood sum of each pixel in the subsequent row is superimposed with the pixel value of the newly added column after sliding windowing; other row pixels are integrated accordingly, and the neighborhood sum of pixels in other rows is superimposed with the neighborhood sum of the previous row. Column pixels are also summed accordingly using sliding windowing. Next, the row and column means are calculated separately: the total number of valid (non-zero) values ​​within the sliding window is used as the denominator, and the sum of the neighborhood values ​​is used as the numerator. The average value of the valid data within each window is calculated, so that each window corresponds to a mean, which is denoted as the filtering result. The number of valid values ​​is also calculated separately for rows and columns.

[0094] S203. Determine the row pixel change information and column pixel change information in the three-dimensional image to be tested, and generate the corresponding three-dimensional texture map based on the row pixel change information and column pixel change information.

[0095] In this embodiment, to more clearly display minute defects such as minor deformations, minor indentations, and minor scratches on the target workpiece in a 3D depth map, the row pixel change information and column pixel change information in the 3D test image are first determined, and a corresponding 3D texture map is generated based on this change information. In one implementation, determining the row pixel change information and column pixel change information in the 3D test image includes: completing invalid row and column pixels in the 3D test image to obtain a 3D completed image; calculating gradients for the row and column pixels in the 3D completed image to obtain row gradient values ​​and column gradient values; determining row pixel change information based on the row gradient values, and determining column pixel change information based on the column gradient values. Specifically, calculating gradients for the row and column pixels in the 3D completed image to obtain row gradient values ​​and column gradient values ​​includes: performing convolution on the row and column pixels in the 3D completed image respectively to calculate the row gradient values ​​and column gradient values. When calculating the gradient in the X direction (row direction), linear differences in the X direction are preferentially used. When calculating the gradient in the Y direction (column direction), linear differences in the Y direction are preferentially used.

[0096] In one implementation, generating a corresponding 3D texture map based on row pixel change information and column pixel change information includes: normalizing the row pixel change information and column pixel change information respectively, and generating a 3D texture map based on the normalization results, so that the 3D texture map effectively highlights surface texture and subtle changes, and clearly displays the 3D spatial changes in the depth map in grayscale. For a comparison between the 3D texture map and the original 3D depth map, please refer to [link to relevant documentation]. Figure 3 , Figure 3 Two sets of comparison diagrams are shown. For example... Figure 3 The two sets of comparison images shown demonstrate that the texture map reveals more details about the minute deformations such as protrusions and depressions on the surface of the target workpiece.

[0097] S204. Use the defect detection model to perform defect detection on the three-dimensional texture map to obtain the defect detection results of the target workpiece.

[0098] In this embodiment, the defect detection model outputs the following defect detection results: the region where the defect is located and the corresponding defect type, achieving accurate defect localization and category analysis. The defect detection model can employ a structure such as UNET. Before training the model, a training dataset is constructed following the steps described above. The training dataset includes multiple 3D texture maps; the region where the defect is located and the corresponding defect type in each 3D texture map are known and used as training ground values. The model is trained using the training dataset to obtain the defect detection model. After training, the defect detection model can be tested. Given a new texture map, the model is tested to see if it can correctly detect the region where the defect is located and the corresponding defect type. If it can, the model passes the test; otherwise, the training dataset is changed, and the model is trained again until the model passes the test.

[0099] S205. Quantify the defects of the target workpiece based on the defect detection results.

[0100] It should be noted that defect types can include: dents, minor dents, deformation, minor deformation, scratches, minor scratches, etc. For these types of defects, the length, area, depth of indentation, or height of protrusion can be accurately measured.

[0101] In one implementation, defects in the target workpiece are quantified based on defect detection results, including: counting the number of pixels in any defect region included in the defect detection results; determining the area of ​​the current defect region based on the number of pixels; and labeling the area of ​​the current defect region and the corresponding defect type in the defect detection results onto the current defect region; wherein the labeled defect type is the defect type corresponding to the current defect region recorded in the defect detection results. Thus, by counting the number of pixels in the defect region, calculating the area of ​​the defect region, and simultaneously labeling the area and defect type, quantitative representation and visual association of defect information are achieved. This enables accurate acquisition of quantitative dimensional information of defects, intuitive presentation of defect regions, and improvement of the completeness and readability of defect detection results. It provides quantitative data support for subsequent defect analysis, classification, and quality assessment, while also enhancing the intuitiveness and traceability of defect detection results.

[0102] In one implementation, quantifying defects in a target workpiece based on defect detection results includes: morphologically dilating any defect region included in the defect detection results to obtain a coverage area covering the current defect region; determining a dilated region other than the current defect region within the coverage area; performing plane fitting based on point cloud data within the dilated region to obtain a fitted plane; calculating the distance between the point cloud data within the current defect region and the fitted plane; and determining the defect height of the current defect region based on the distance. Thus, by obtaining a coverage area through morphological dilution of the defect region, constructing a reference plane using the non-defect point cloud of the dilated region for plane fitting, and accurately quantifying the defect height by calculating the distance between the defect region's point cloud and the fitted plane, the interference of the defect's own point cloud on the reference plane can be effectively eliminated, improving the accuracy and reliability of defect height measurement. In one example, the distances between all or part of the point cloud data within the current defect region and the fitted plane can be calculated, and the average of the first 20% of these distances is calculated as the defect height of the current defect region. Accordingly, the distance between the point cloud data within the current defect area and the fitted plane is calculated, including: calculating the distance between each point cloud data point within the current defect area and the fitted plane respectively; correspondingly, the defect height of the current defect area is determined based on the distance, including: sorting the distances between each point cloud data point and the fitted plane in descending order to obtain a target sequence; determining the defect height of the current defect area based on the mean of the first N elements in the target sequence. For example, the mean of the first N elements in the target sequence is used as the defect height of the current defect area. N can be flexibly set based on the quality inspection standards of the current target workpiece, the workpiece application environment, etc. This method selects the first N representative distances by sorting and calculating their mean, so that the final height value can objectively reflect the actual degree of protrusion or depression in the defect area, which can avoid errors caused by single-point data in height measurement to a certain extent, and is conducive to achieving accurate quantification of defect height.

[0103] In one implementation, the defects of the target workpiece are quantified based on the defect detection results, including: if the defect type corresponding to any defect region in the defect detection results is a scratch, then the scratch is skeletonized to obtain a skeleton diagram of the scratch; a depth-first search is performed on the skeleton diagram of the scratch to obtain the longest connected skeleton in the skeleton diagram of the scratch; the length of the longest connected skeleton is measured as the length of the scratch. It can be seen that for scratch-type defects, extracting the scratch skeleton through skeletonization and using a depth-first search to determine the longest connected skeleton and using its length as the scratch length can accurately characterize the true extension length of the scratch, effectively eliminate redundant branch interference, and improve the accuracy and stability of scratch length measurement. Furthermore, multiple depth-first searches can be used to determine the longest connected skeleton, thereby improving the comprehensiveness of the traversal and effectively eliminating the influence of short branches and isolated noise points, avoiding the problem of a single search easily getting trapped in local optima and missing the true length.

[0104] This embodiment can accurately capture the three-dimensional spatial morphology information of a workpiece using a depth map, and then generate a three-dimensional test image through noise reduction processing, effectively filtering out environmental interference and other noise during the depth acquisition process. Then, by extracting the row and column pixel change information of the three-dimensional test image, a three-dimensional texture map is generated, transforming subtle surface irregularities (such as minor deformations, minor indentations, minor scratches, and other minor defects) and structural anomalies into texture features that can be recognized by the model. Compared with traditional two-dimensional inspection, it can more comprehensively capture three-dimensional spatial defects of the workpiece, especially suitable for detecting subtle surface defects and internal hidden defects. Subsequently, a defect detection model is used to detect the three-dimensional texture map, relying on the model's autonomous learning and accurate recognition capabilities to significantly improve detection efficiency and accuracy. After obtaining the detection results, defect quantification is further achieved, such as accurately outputting key parameters like the location, size, and depth of defects, rather than simply determining whether a defect exists. This not only allows for the detection of minor defects such as minor deformations, minor indentations, and minor scratches on the workpiece, improving detection accuracy and reducing the missed detection rate and false detection rate, but also provides accurate data support for workpiece quality grading, defect cause analysis, and repair plan formulation.

[0105] Please see Figure 4 Another workpiece inspection method includes:

[0106] Step 1: Acquire depth images.

[0107] A depth image is an image that stores height information. Due to the imaging principle of cameras, invalid height values ​​may exist. The data format of a depth map consists of two channels: a height channel and a label channel. The height channel is stored as a 16-bit integer, and the label channel is stored as a Boolean type, where true indicates valid and false indicates invalid.

[0108] Step 2, Fast Gaussian Filter.

[0109] Mean filtering is performed by separating rows and columns and using a sliding window method, thereby improving the filtering speed.

[0110] Step 3: Map the filtered depth image to a surface texture image. The specific process includes:

[0111] 1. Invalid value handling and interpolation: Since some pixels have invalid values, the gradient cannot be calculated directly. Therefore, linear interpolation is performed first to fill in these invalid pixels. When calculating the gradient in the X direction, the linear interpolation in the X direction is used first, and when calculating the gradient in the Y direction, the linear interpolation in the Y direction is used first.

[0112] 2. Gradient calculation: Perform convolution on the completed depth image in the X and Y directions to calculate the gradient.

[0113] In one example, the convolution kernel in the X direction is: The convolution kernel in the Y direction is: The X-direction convolution kernel is multiplied by the padded depth image to obtain the X-direction gradient value Gs. x The gradient value Gs in the Y-direction is obtained by multiplying the convolution kernel in the Y-direction by the padded depth image. y .

[0114] 3. Gradient Normalization and Gray-Scale Mapping: The gradient values ​​in the X and Y directions are normalized to the interval [0, 255]. This can be expressed by the formula: Normalized X-direction gradient value Gray x =(Gs x +1) / 2×255; Normalized gradient value in the Y direction Gray y =(Gs y +1) / 2×255.

[0115] The resulting texture map effectively highlights the surface texture and subtle changes in the depth map, clearly displaying spatial variations in grayscale.

[0116] Step 4: Train the defect segmentation and detection model, and enable the model to automatically identify and segment defective regions in the texture map during inference. The output image of the model is a segmentation mask, which clearly marks the defective regions and their defect types.

[0117] Step 5, Measure defects: Quantify defects precisely.

[0118] Common defect types include: dents, minor dents, deformation, minor deformation, scratches, and minor scratches. When assessing the severity of dents and deformations, the area and height are the primary indicators, while scratches are mainly assessed based on length.

[0119] The defect area is obtained directly from pixel statistics using the output of the defect segmentation and detection model. The number of pixels within the defect area is the area, which can be further converted into a practical unit of measurement by combining the resolution.

[0120] In one example, height measurement is applicable to defects such as dents and deformations. The calculation steps are as follows:

[0121] ①Region expansion: Morphological expansion and magnification are performed on the segmented defect region to obtain a new region slightly larger than the defect.

[0122] ② Fitting Plane Region Acquisition: Subtract the point cloud of the original defect region from the point cloud of the expanded region to obtain the point cloud data of the fitting plane region (i.e., the normal surface surrounding the defect).

[0123] ③ Plane fitting: In the filtered depth map, the point cloud data of this fitting plane region is fitted with RANSAC (random consensus algorithm) to obtain the best reference plane.

[0124] ④ Height Statistics: Calculate the distance from the point cloud data of the defect area to the fitted plane, and take the average of the top 20% of all these distances as the final height of the defect (damage, deformation). By selecting the top 20% of distance values ​​and calculating the average, the final height value can more objectively reflect the actual degree of protrusion or depression in the defect area, avoiding single-point calculation errors.

[0125] In one example, the scratch length measurement steps include:

[0126] ①Skeleton extraction: The segmented scratch area is skeletonized to obtain one (or more) pixel lines.

[0127] ② Longest path search: Two depth-first search (DFS) steps are used to find the longest connected skeleton in the skeleton graph. The length of this path is the scratch length. The two DFS steps improve the comprehensiveness of the traversal and can also eliminate interference from breaks, burrs, isolated noise points, etc., thus improving the accuracy of length measurement.

[0128] As can be seen, this embodiment effectively achieves automatic defect detection and accurate quantification through steps such as depth image acquisition, fast filtering, texture mapping, defect segmentation model training and inference, and automatic defect measurement. This allows for the detection of minute defects such as minor deformations, minor pressure marks, and minor scratches on workpieces, improving detection accuracy and reducing the false negative and false positive rates.

[0129] The workpiece inspection device provided in the embodiments of this application is described below. The workpiece inspection device described below and the workpiece inspection method described above can be referred to each other.

[0130] Please refer to Figure 5 , Figure 5 A schematic diagram of a workpiece inspection device provided in this application embodiment includes:

[0131] The acquisition module 501 is used to acquire the depth map of the target workpiece;

[0132] The denoising module 502 is used to denoise the depth map to obtain a three-dimensional image to be measured.

[0133] The mapping module 503 is used to determine the row pixel change information and column pixel change information in the three-dimensional image to be tested, and to generate the corresponding three-dimensional texture map based on the row pixel change information and column pixel change information.

[0134] The detection module 504 is used to perform defect detection on the three-dimensional texture map using a defect detection model to obtain the defect detection results of the target workpiece.

[0135] The quantization module 505 is used to quantify the defects of the target workpiece based on the defect detection results.

[0136] In one implementation, the noise reduction module is specifically used for:

[0137] After setting the height of invalid pixels in the depth map to zero, a filtering window is set to filter the row and column pixels in the depth map in parallel to obtain the 3D test image.

[0138] In one implementation, the mapping module is specifically used for:

[0139] By completing the row and column invalid pixels in the 3D image to be tested, a 3D completed image is obtained.

[0140] Calculate the gradients of row and column pixels in the 3D completed image to obtain row gradient values ​​and column gradient values.

[0141] Row pixel change information is determined based on row gradient values, and column pixel change information is determined based on column gradient values.

[0142] In one implementation, the mapping module is specifically used for:

[0143] The row pixel change information and column pixel change information are normalized respectively, and a three-dimensional texture map is generated based on the normalization results.

[0144] In one implementation, the quantization module is specifically used for:

[0145] The number of pixels in any defect region included in the defect detection results;

[0146] The area of ​​the current defect region is determined based on the number of pixels;

[0147] Mark the area of ​​the current defect region and the corresponding defect type in the defect detection results in the current defect region.

[0148] Optionally, the quantization module is specifically used for:

[0149] Morphological expansion is performed on any defect region included in the defect detection results to obtain the coverage area that covers the current defect region;

[0150] Within the coverage area, identify the expanded area excluding the current defective area;

[0151] Plane fitting is performed based on the point cloud data within the expansion region to obtain the fitted plane;

[0152] Calculate the distance between the point cloud data within the current defect area and the fitted plane;

[0153] The defect height of the current defect area is determined based on distance.

[0154] Optionally, the quantization module is specifically used for:

[0155] Calculate the distance between each point cloud data point within the current defect area and the fitted plane;

[0156] The distances between each point cloud data point and the fitted plane are sorted in descending order to obtain the target sequence;

[0157] The defect height of the current defect region is determined based on the mean of the first N elements in the target sequence.

[0158] Optionally, the quantization module is specifically used for:

[0159] If the defect type corresponding to any defect area in the defect detection result is a scratch, then the scratch is skeletonized to obtain the skeleton diagram of the scratch.

[0160] Perform a depth-first search on the skeleton graph of the scratch to obtain the longest connected skeleton in the skeleton graph of the scratch.

[0161] Measure the length of the longest connected skeleton as the length of the scratch.

[0162] As can be seen, the workpiece inspection device provided in this embodiment effectively realizes automatic detection and accurate quantification of defects; it can detect minute defects such as minor deformations, minor pressure marks, and minor scratches on workpieces, thereby improving detection accuracy and reducing the rate of missed detections and false detections.

[0163] The computer-readable storage medium provided in the embodiments of this application is described below. The computer-readable storage medium described below can be referred to in correspondence with the workpiece inspection method described above.

[0164] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described workpiece inspection method.

[0165] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0166] Furthermore, the computer program stored on the computer-readable storage medium in this embodiment, when executed by the processor, can perform the following steps: acquiring a depth map of the target workpiece; performing noise reduction processing on the depth map to obtain a three-dimensional test map; determining the row pixel change information and column pixel change information in the three-dimensional test map, and generating a corresponding three-dimensional texture map based on the row pixel change information and column pixel change information; performing defect detection on the three-dimensional texture map using a defect detection model to obtain the defect detection result of the target workpiece; and quantifying the defects of the target workpiece based on the defect detection result.

[0167] Furthermore, in this embodiment, the computer program stored on the computer-readable storage medium, when executed by the processor, can perform the following steps: after setting the height of invalid pixels in the depth map to zero, a filtering window is set to filter the row pixels and column pixels in the depth map in parallel to obtain a three-dimensional image to be tested.

[0168] Furthermore, the computer program stored on the computer-readable storage medium in this embodiment, when executed by the processor, can perform the following steps: completing invalid row pixels and invalid column pixels in the three-dimensional image to be tested to obtain a three-dimensional completed image; calculating gradients for row pixels and column pixels in the three-dimensional completed image to obtain row gradient values ​​and column gradient values; determining row pixel change information based on row gradient values, and determining column pixel change information based on column gradient values.

[0169] Furthermore, the computer program stored on the computer-readable storage medium in this embodiment, when executed by the processor, can perform the following steps: normalize the row pixel change information and column pixel change information respectively, and generate a three-dimensional texture map based on the normalization result.

[0170] Furthermore, the computer program stored on the computer-readable storage medium in this embodiment, when executed by the processor, can perform the following steps: count the number of pixels in any defect region included in the defect detection result; determine the area of ​​the current defect region based on the number of pixels; and mark the area of ​​the current defect region and the defect type corresponding to the current defect region in the defect detection result on the current defect region.

[0171] Furthermore, the computer program stored on the computer-readable storage medium in this embodiment, when executed by the processor, can perform the following steps: morphological dilation of any defect region included in the defect detection result to obtain a coverage region covering the current defect region; determining a dilated region other than the current defect region within the coverage region; performing plane fitting based on the point cloud data within the dilated region to obtain a fitted plane; calculating the distance between the point cloud data within the current defect region and the fitted plane; and determining the defect height of the current defect region based on the distance.

[0172] Furthermore, the computer program stored on the computer-readable storage medium in this embodiment, when executed by the processor, can perform the following steps: calculate the distance between each point cloud data in the current defect area and the fitting plane respectively; sort the distances between each point cloud data and the fitting plane in descending order to obtain the target sequence; and determine the defect height of the current defect area based on the mean of the first N elements in the target sequence.

[0173] Furthermore, the computer program stored on the computer-readable storage medium in this embodiment, when executed by the processor, can perform the following steps: if the defect type corresponding to any defect area in the defect detection result is a scratch, then the scratch is skeletonized to obtain a skeleton map of the scratch; a depth-first search is performed on the skeleton map of the scratch to obtain the longest connected skeleton in the skeleton map of the scratch; the length of the longest connected skeleton is measured as the length of the scratch.

[0174] The following describes a computer program product provided by an embodiment of this application. The computer program product described below can be referred to in conjunction with other embodiments described herein.

[0175] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned disclosed workpiece inspection method.

[0176] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the steps in any of the above embodiments.

[0177] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0178] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0179] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0180] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "include," "contain," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0181] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A workpiece inspection method characterized by, include: Obtain the depth map of the target workpiece; The depth map is denoised to obtain a three-dimensional image to be measured. Determine the row pixel change information and column pixel change information in the three-dimensional image to be tested, and generate a corresponding three-dimensional texture map based on the row pixel change information and column pixel change information; The defect detection model is used to perform defect detection on the three-dimensional texture map to obtain the defect detection results of the target workpiece; The defects of the target workpiece are quantified based on the defect detection results.

2. The method of claim 1, wherein, The denoising process of the depth map to obtain the three-dimensional image to be measured includes: After setting the height of invalid pixels in the depth map to zero, a filtering window is set to filter the row and column pixels in the depth map in parallel to obtain the three-dimensional test image.

3. The method of claim 1, wherein, The process of determining the row pixel change information and column pixel change information in the three-dimensional image to be measured includes: By completing the row and column invalid pixels in the three-dimensional image to be tested, a three-dimensional completed image is obtained. Calculate the gradients of row and column pixels in the three-dimensional completed image to obtain row gradient values ​​and column gradient values; The row pixel change information is determined based on the row gradient value, and the column pixel change information is determined based on the column gradient value.

4. The method of claim 3, wherein, The step of generating a corresponding 3D texture map based on the row pixel change information and column pixel change information includes: The row pixel change information and the column pixel change information are normalized respectively, and the three-dimensional texture map is generated based on the normalization results.

5. The method of claim 1, wherein, The quantification of defects in the target workpiece based on the defect detection results includes: The number of pixels in any defect region included in the defect detection results is counted. The area of ​​the current defect region is determined based on the number of pixels; The area of ​​the current defect region and the corresponding defect type in the defect detection results are marked on the current defect region.

6. The method according to any one of claims 1 to 5, characterized in that, The quantification of defects in the target workpiece based on the defect detection results includes: Morphological dilation is performed on any defect region included in the defect detection result to obtain a coverage area that covers the current defect region; Within the coverage area, identify the expansion region other than the current defect region; A plane fitting is performed based on the point cloud data within the expansion region to obtain the fitting plane; Calculate the distance between the point cloud data within the current defect area and the fitted plane; The defect height of the current defect area is determined based on the distance.

7. The method of claim 6, wherein, The calculation of the distance between the point cloud data within the current defect region and the fitted plane includes: Calculate the distance between each point cloud data point within the current defect area and the fitted plane; Accordingly, determining the defect height of the current defect region based on the distance includes: The distances between each point cloud data point and the fitted plane are sorted in descending order to obtain the target sequence; The defect height of the current defect region is determined based on the mean of the first N elements in the target sequence.

8. The method according to any one of claims 1 to 5, characterized in that, The quantification of defects in the target workpiece based on the defect detection results includes: If the defect type corresponding to any defect area in the defect detection result is a scratch, then the scratch is skeletonized to obtain the skeleton diagram of the scratch. A depth-first search is performed on the skeleton graph of the scratch to obtain the longest connected skeleton in the skeleton graph of the scratch. The length of the longest connected skeleton is measured and taken as the length of the scratch.

9. A workpiece inspection apparatus, characterized by comprising: include: The acquisition module is used to acquire the depth map of the target workpiece; A denoising module is used to denoise the depth map to obtain a three-dimensional image to be tested; The mapping module is used to determine the row pixel change information and column pixel change information in the three-dimensional image to be tested, and to generate a corresponding three-dimensional texture map based on the row pixel change information and column pixel change information. The detection module is used to perform defect detection on the three-dimensional texture map using a defect detection model to obtain the defect detection results of the target workpiece; The quantization module is used to quantify the defects of the target workpiece based on the defect detection results.

10. An electronic device, comprising: Includes memory and processor, wherein: The memory is used to store computer programs; The processor is configured to execute the computer program to implement the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the method as described in any one of claims 1 to 8.

Images