Image detection method and system for finishing metal surface defects

Abstract

The application provides a kind of image detection method of finishing metal surface defect, comprising: S1.3D line laser sensor and motion device are used to collect metal surface data;S2. based on metal surface data, the reference height of metal surface is obtained by histogram statistics;S3. the difference between each laser line and reference height is calculated, and the height of each laser line is adjusted based on the difference;S4. data linear interpolation is completed and data is corrected to obtain gray scale image;S5. data noise reduction filtering is carried out on gray scale image;S6. the depth data of metal surface defect is obtained by defect detection on gray scale image, and the absolute position of defect is obtained by defect positioning based on the position data of motion device;S7. global map and / or local map and / or global three-dimensional map display is carried out on gray scale image.The application has lower requirement for shooting environment, can reflect the depth data of metal surface, and clearly show the edge details of defect.

Description

Technical Field

[0001] This invention relates to the field of metal surface defect detection technology, and in particular to an image detection method and system for defects on the surface of precision-machined metals. Background Technology

[0002] The general method for detecting defects on metal surfaces is as follows: 1. Take pictures of the metal surface using a camera; 2. Preprocess the images (denoising, image enhancement, etc.); 3. Use algorithms (traditional algorithms or deep learning algorithms) to detect and identify defects in the images. A more advanced method is: 1. Attach the camera to a motion device that ensures the camera can capture images of all metal surfaces; 2. After obtaining the images, perform defect detection, classifying and locating the defects.

[0003] However, the above method has the following problems:

[0004] 1. Traditional cameras have high requirements for the shooting environment, requiring good lighting and a suitable angle; otherwise, the accuracy of defect identification will be greatly affected.

[0005] 2. The defect location is limited to this image; that is, we can only pinpoint the defect's exact location within the image. However, we often need to clearly define the defect's location within the entire metal workpiece to allow operators to quickly identify the defect and perform relevant actions.

[0006] 3. Due to issues such as pixel count, traditional cameras can only obtain the planar properties of defects, such as the length and width of the defects, but cannot obtain another important property of defects, namely the depth of the defects.

[0007] 4. Traditional cameras often have poor accuracy and are easily affected by external interference, which can cause the edge details of defects to be unclear.

[0008] 5. Traditional defect detection systems only focus on the detection and location of defects, without providing a three-dimensional view of the entire acquisition surface (including defects). Summary of the Invention

[0009] This invention provides an image detection method and system for defects on the surface of precision-machined metals to solve the above-mentioned problems.

[0010] One aspect of this invention discloses an image detection method for defects on the surface of finished metals, comprising:

[0011] Step S1. Use a 3D line laser sensor and motion device to perform laser scanning detection on the metal surface and collect metal surface data;

[0012] Step S2. Based on the metal surface data, obtain the reference height of the metal surface through histogram statistics;

[0013] Step S3. Calculate the difference between each laser line and the reference height, and adjust the height of each laser line based on the difference;

[0014] Step S4. Based on each adjusted laser line, perform linear interpolation and data regularization correction to obtain a grayscale image;

[0015] Step S5. Perform data noise reduction filtering on the grayscale image;

[0016] Step S6. Perform defect detection on the grayscale image after data noise reduction and filtering to obtain the depth data of the defects on the metal surface, and perform defect localization based on the position data of the motion device to obtain the absolute position of the defects;

[0017] Step S7. Display the grayscale image as a global image and / or a local image and / or a global 3D image.

[0018] In some embodiments, a position sensor is used to collect position data for each laser scan detection performed by the 3D line laser sensor, and the defect is located based on the position data to obtain the absolute position of the defect on the metal surface.

[0019] In some embodiments, in step S2, the edge of the metal surface is used as the boundary line of the effective surface to remove data outside the effective surface and only display data within the effective surface.

[0020] In some embodiments, in step S3, after adjusting the height of each laser line, a straight line is fitted to each laser line, and the tilt angle between the fitted straight line and the metal surface is calculated. When the tilt angle exceeds a threshold, the laser line is straightened and corrected.

[0021] In some embodiments, step S4, data linear interpolation completion includes:

[0022] The average point distance of the entire point cloud is obtained by calculating the average point distance.

[0023] Based on the average point distance, traverse the data points, find the points with larger x-axis distances from adjacent data points, mark them as holes, and then insert data points in the middle of the holes to fill them, according to the rate of change of data points at both ends of the holes.

[0024] In some embodiments, in step S4, data alignment correction is performed by comparing the flat regions in each row of data and raising the corresponding regions of each row of data to the same height.

[0025] In some embodiments, step S5, data noise reduction filtering includes median filtering, mean filtering, and image closing operation.

[0026] Another aspect of the present invention discloses an image detection system for defects on the surface of finished metal, used to perform the image detection method for defects on the surface of finished metal as described above;

[0027] The image detection system for defects on the finished metal surface includes:

[0028] 3D line laser sensor is used for laser scanning and detection of metal surfaces to collect metal surface data;

[0029] The motion device is connected to the 3D line laser sensor to ensure that data can be collected from all metal surfaces;

[0030] A position sensor is connected to the motion device to collect position data during each laser scan detection;

[0031] The processor is connected to the 3D line laser sensor, the motion device, and the position sensor respectively to control the operation of the 3D line laser sensor, the motion device, and the position sensor;

[0032] A data preprocessing module, connected to the processor, is used to obtain the reference height of the metal surface through histogram statistics based on the metal surface data, calculate the difference between each laser line and the reference height, adjust the height of each laser line based on the difference, and perform data linear interpolation completion and data regularization correction based on each adjusted laser line to obtain a grayscale image.

[0033] The defect detection module, connected to the processor, is used to perform data noise reduction filtering on the grayscale image, perform defect detection on the grayscale image after data noise reduction filtering, obtain the depth data of the defects on the metal surface, and perform defect localization based on the position data of the motion device to obtain the absolute position of the defect.

[0034] The defect display module, connected to the processor, is used to display the grayscale image as a global image and / or a local image and / or a global 3D image.

[0035] In summary, the present invention has at least the following beneficial effects:

[0036] This invention uses a 3D line laser sensor to collect data from metal surfaces, which has low requirements for the shooting environment and relatively high accuracy in defect identification.

[0037] This invention uses a 3D line laser sensor and a motion device in conjunction to locate defects by using the position data of the motion device. This allows for the determination of the absolute location of the defect, which in turn clarifies the position of the defect on the entire metal surface, making it convenient for operators to quickly view the defect and perform related operations.

[0038] This invention uses a 3D line laser sensor to collect data on the metal surface using line lasers. Multiple line lasers can be combined into a high-resolution grayscale image through mapping, which can reflect the depth data of the metal surface. The depth data can be used to determine defects and their related attributes such as length, width, and depth.

[0039] This invention obtains the reference height of the metal surface through histogram statistics, calculates the difference between each laser line and the reference height, and adjusts the height of each laser line based on the difference. Based on each adjusted laser line, linear interpolation and data regularization correction are performed to obtain a grayscale image. This can reduce or even eliminate the influence of motion devices on the acquisition effect of the 3D line laser sensor, ensuring that the 3D line laser sensor can acquire stable and effective metal surface data in unstable physical environments, thereby improving the accuracy of defect identification. Even under external interference, it can clearly display the edge details of defects.

[0040] This invention uses a 3D line laser sensor to collect data from a metal surface using a line laser, and can subsequently display the grayscale image as a global map and / or a local map and / or a global 3D map. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram illustrating the steps of the image detection method for defects on the finished metal surface involved in this invention.

[0043] Figure 2 This is a schematic diagram of the image detection system for defects on the finished metal surface involved in this invention. Detailed Implementation

[0044] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the embodiments of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0045] The following disclosure provides many different implementations or examples for carrying out different structures of the embodiments of the present invention. To simplify the disclosure of the embodiments of the present invention, specific examples of components and arrangements are described below. Of course, these are merely examples and are not intended to limit the embodiments of the present invention. Furthermore, reference numerals and / or reference letters may be repeated in different examples of the embodiments of the present invention; such repetition is for simplification and clarity and does not in itself indicate a relationship between the various implementations and / or arrangements discussed.

[0046] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0047] like Figure 1 As shown, one aspect of this invention discloses an image detection method for defects on the surface of finished metals, comprising:

[0048] Step S1. Use a 3D line laser sensor and motion device to perform laser scanning detection on the metal surface and collect metal surface data;

[0049] Step S2. Based on the metal surface data, obtain the reference height of the metal surface through histogram statistics;

[0050] Step S3. Calculate the difference between each laser line and the reference height, and adjust the height of each laser line based on the difference;

[0051] Step S4. Based on each adjusted laser line, perform linear interpolation and data regularization correction to obtain a grayscale image;

[0052] Step S5. Perform data noise reduction filtering on the grayscale image;

[0053] Step S6. Perform defect detection on the grayscale image after data noise reduction and filtering to obtain the depth data of the defects on the metal surface, and perform defect localization based on the position data of the motion device to obtain the absolute position of the defects;

[0054] Step S7. Display the grayscale image as a global image and / or a local image and / or a global 3D image.

[0055] It should be understood that due to the instability of the motion device, there will be up-and-down shaking, which will affect the acquisition effect. Up-and-down shaking will cause the displayed image to form a wave-like undulation, which will affect the identification of defects. However, through steps S2-S3, the up-and-down shaking can be overcome, thereby improving the accuracy of defect identification.

[0056] In practical applications, the distance from the laser emitter of a line laser sensor to the metal surface is defined as height. The reference height is the average distance from all laser points to the metal surface, obtained by statistically analyzing a histogram, and is defined as the reference depth (reference height). A depth is also calculated for each laser line. This laser line depth is compared to the reference depth and adjusted accordingly. For example, in step 3, if the laser line depth is lower than the reference depth, each point on the laser line is adjusted upwards. If the calculated laser line depth is 9900mm and the reference depth is 10000mm, the depth of each point on the laser line is increased by 100mm; if the calculated laser line depth is 10100mm and the reference depth is 10000mm, the depth of each point on the laser line is decreased by 100mm.

[0057] It is clear that in step S6, defect detection is performed by a software program. The first step involves filtering and dilation / erosion processing the depth map to remove image noise. The second step uses OpenCV's Cannny operator to find the defect edges. The third step identifies the defects based on the edges and then determines their length, width, and depth. This defect detection process is a flexible application of mature and conventional techniques and will not be described in detail here.

[0058] Furthermore, the defect location in step S6 is explained as follows:

[0059] If the motion device can move approximately 40° per movement, it can form a 360° circle after 9 movements. At the start and end of each movement, a current angle (e.g., starting angle 0°, ending angle 40°) is transmitted to the processor. This 0° to 40° range is composed of data collected by tens of thousands of laser lines. After processing and converting it into a grayscale image, defect identification is performed. The location of the defect can be determined in the grayscale image, and the absolute location of the defect can be calculated based on the starting and ending angles of the collected data.

[0060] In some embodiments, a position sensor is used to collect position data for each laser scan detection performed by the 3D line laser sensor, and the defect is located based on the position data to obtain the absolute position of the defect on the metal surface.

[0061] In some embodiments, in step S2, the edge of the metal surface is used as the boundary line of the effective surface to remove data outside the effective surface and only display data within the effective surface.

[0062] In this scheme, since the motion device cannot guarantee that the 3D line laser sensor can always be centered on the defect surface, the image display will show a left-right offset (snake-like). Taking the annular groove as an example, the inner sidewalls of the two ends of the metal surface (the inner and outer rings) are used as the dividing line of the effective surface. Only the surface data between the inner sidewalls of the two ends is displayed, which can overcome the impact of the left-right offset on defect recognition and thus improve the accuracy of defect recognition.

[0063] In some embodiments, in step S3, after adjusting the height of each laser line, a straight line is fitted to each laser line, and the tilt angle between the fitted straight line and the metal surface is calculated. When the tilt angle exceeds a threshold, the laser line is straightened and corrected.

[0064] In this solution, since the irradiation surface of the 3D line laser sensor cannot always remain horizontal with the metal surface, it will affect the depth accuracy of the defect. However, by leveling and correcting the tilt angle, the influence of the horizontal tilt angle on defect identification can be overcome, thereby improving the defect identification accuracy.

[0065] Leveling and correction method: Fit a straight line to each laser line to obtain the tilt angle α of the line. Taking one end of the laser line as the starting point, the distance from each point to the starting point can be obtained. Based on the tilt angle α, the Z-value (depth value) offset of each point from the non-tilted points can be obtained. Then, the Z-value of each point is corrected to achieve the correction.

[0066] Threshold: The threshold is determined based on the detection results. When the tilt angle is within 5°, it indicates that the acquisition process is normal and can be corrected by leveling. If the tilt angle exceeds 5° during the acquisition process, it indicates that the acquisition process is abnormal and the operator needs to check whether there is a problem with the motion device.

[0067] In some embodiments, step S4, data linear interpolation completion includes:

[0068] The average point distance of the entire point cloud is obtained by calculating the average point distance.

[0069] Based on the average point distance, traverse the data points, find the points with larger x-axis distances from adjacent data points, mark them as holes, and then insert data points in the middle of the holes to fill them, according to the rate of change of data points at both ends of the holes.

[0070] In this scheme, the average point distance is calculated as follows: The average point distance is used to provide a reference for the subsequent radius search and hole filling threshold setting. It can be estimated using a spatial block strategy. By calculating the minimum bounding box that contains the entire point cloud and placing it in the grid, the average point distance of the entire point cloud can be estimated.

[0071] Hole filling: For voids, data preprocessing is performed before analysis through steps such as uniform void boundaries, unified boundary direction, and inner / outer boundary judgment. After preprocessing, filling point calculation can proceed. Finally, illegal judgment points falling outside the void area need to be removed to obtain the coordinates of the filling points. This process is repeated until no more filling points can be calculated, at which point the void repair (hole filling) is complete.

[0072] In some embodiments, in step S4, data alignment correction is performed by comparing the flat regions in each row of data and raising the corresponding regions of each row of data to the same height.

[0073] In this scheme, vibration is inevitable during the acquisition process of the motion device, which will cause unevenness and jitter distortion in the acquired data. This results in undulation in the height direction during acquisition, causing wavy lines with alternating light and dark on the depth map. By using the part of the data with higher flatness as the correction benchmark, the data points that deviate from the plane are smoothed out, thereby eliminating the wavy lines caused by vibration.

[0074] In some embodiments, step S5, data noise reduction filtering includes median filtering, mean filtering, and image closing operation.

[0075] like Figure 2 As shown, another aspect of the present invention discloses an image detection system for defects on the surface of finished metal, used to perform the image detection method for defects on the surface of finished metal as described above;

[0076] The image detection system for defects on the finished metal surface includes:

[0077] 3D line laser sensor is used for laser scanning and detection of metal surfaces to collect metal surface data;

[0078] The motion device is connected to the 3D line laser sensor to ensure that data can be collected from all metal surfaces;

[0079] A position sensor is connected to the motion device to collect position data during each laser scan detection;

[0080] The processor is connected to the 3D line laser sensor, the motion device, and the position sensor respectively to control the operation of the 3D line laser sensor, the motion device, and the position sensor;

[0081] A data preprocessing module, connected to the processor, is used to obtain the reference height of the metal surface based on the metal surface data through histogram statistics, calculate the difference between each laser line and the reference height, adjust the height of each laser line based on the difference, and perform linear interpolation and data alignment correction on each adjusted laser line to obtain a grayscale image; (to optimize time efficiency, the tilt angle can be corrected while adjusting the height of each laser line.)

[0082] The defect detection module, connected to the processor, is used to perform data noise reduction filtering on the grayscale image, perform defect detection on the grayscale image after data noise reduction filtering, obtain the depth data of the defects on the metal surface, and perform defect localization based on the position data of the motion device to obtain the absolute position of the defect.

[0083] A defect display module, connected to the processor, is used to display the grayscale image as a global image and / or a local image and / or a global 3D image. (In practice, the global 3D image, the global image of the grayscale image, and the local image all have dedicated display locations and can be displayed simultaneously or selectively.)

[0084] The embodiments described above are for illustrative purposes only and are not intended to limit the invention. Therefore, any changes in numerical values ​​or substitutions of equivalent elements should still fall within the scope of this invention.

[0085] The above detailed description will enable those skilled in the art to understand that the present invention can indeed achieve the aforementioned objectives and has complied with the provisions of the Patent Law.

[0086] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention. The above descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. It should be noted that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the invention should be included within the scope of protection of the invention.

[0087] It should be noted that the above description of the process is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to the process under the guidance of this specification. However, these modifications and changes remain within the scope of this specification.

[0088] The basic concepts have been described above. Obviously, for those skilled in the art who have read this application, the above disclosure is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore, such modifications, improvements, and corrections still fall within the spirit and scope of the exemplary embodiments of this application.

[0089] Furthermore, this application uses specific terms to describe its embodiments. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that "an embodiment," "one embodiment," or "an alternative embodiment" mentioned twice or more in different positions in this specification do not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application can be appropriately combined.

[0090] Furthermore, those skilled in the art will understand that aspects of this application can be described and illustrated through several patentable types or situations, including any new and useful combination of processes, machines, products, or substances, or any new and useful improvements thereof. Therefore, aspects of this application can be implemented entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. All of the above hardware or software can be referred to as a “unit,” “module,” or “system.” Furthermore, aspects of this application can take the form of a computer program product embodied in one or more computer-readable media, wherein computer-readable program code is contained therein.

[0091] The computer program code required for the operation of each part of this application can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, and Python; general programming languages ​​such as C; Visual Basic, Fortran2103, Perl, COBOL2102, PHP, and ABAP; dynamic programming languages ​​such as Python, Ruby, and Groovy; or other programming languages. This program code can run entirely on the user's computer, or as a standalone software package on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any network, such as a local area network (LAN) or wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as Software as a Service (SaaS).

[0092] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this application are not intended to limit the order of the processes and methods of this application. Although some currently considered useful embodiments of the invention have been discussed in the foregoing disclosure by way of various examples, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the substance and scope of the embodiments of this application. For example, although the implementation of the various components described above can be embodied in a hardware device, it can also be implemented as a purely software solution, such as an installation on an existing server or mobile device.

[0093] Similarly, it should be noted that, in order to simplify the description of the present application and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of the embodiments of the present application sometimes combines multiple features into a single embodiment, drawing, or description thereof. However, this approach of the present application should not be construed as reflecting an intention that the claimed subject matter requires more features than expressly recited in each claim. Rather, the subject of the invention should possess fewer features than in any single embodiment described above.

Claims

1. An image detection method for defects on the surface of precision-machined metals, characterized in that, include: Step S1. Use a 3D line laser sensor and motion device to perform laser scanning detection on the metal surface and collect metal surface data; Step S2. Based on the metal surface data, obtain the reference height of the metal surface through histogram statistics; The effective surface is defined by using the inner sidewalls at both ends of the metal surface as the boundary lines, and only the data of the surface between the inner sidewalls at both ends is displayed. Step S3. Calculate the difference between each laser line and the reference height, and adjust the height of each laser line based on the difference; after adjusting the height of each laser line, perform straight line fitting on each laser line, and calculate the tilt angle between the fitted line and the metal surface. When the tilt angle exceeds the threshold, straighten and correct the laser line. Step S4. Based on each adjusted laser line, perform linear interpolation and data regularization correction to obtain a grayscale image; The data linear interpolation completion includes: By using a spatial partitioning strategy, the average point distance of the entire point cloud can be estimated by calculating the minimum bounding box containing the entire point cloud and placing it in a grid. Based on the average point distance, traverse the data points and find the points whose distance from adjacent data points in the x-axis direction is greater than the average point distance. Mark these points as holes. Then, based on the height change rate of the data points at both ends of the hole, insert data points in the middle of the hole to fill it. Data alignment correction is: by comparing the flat areas in each row of data, i.e. those that are lower than the areas with higher flatness, the corresponding areas of each row of data are raised to the same height. Step S5. Perform data noise reduction filtering on the grayscale image; Step S6. Perform defect detection on the grayscale image after data noise reduction and filtering to obtain the depth data of the defects on the metal surface, and perform defect localization based on the position data of the motion device to obtain the absolute position of the defects; Step S7. Display the grayscale image as a global image and / or a local image and / or a global 3D image.

2. The image detection method for defects on the surface of finished metals according to claim 1, characterized in that, Using a position sensor, position data is collected each time the 3D line laser sensor performs a laser scan detection. Based on the position data, defect location is performed to obtain the absolute position of the defect on the metal surface.

3. The image detection method for defects on the surface of finished metals according to claim 1, characterized in that, In step S5, data denoising filtering includes median filtering, mean filtering, and image closing operation.

4. An image detection system for defects on the surface of precision-machined metals, characterized in that, An image detection method for performing any one of claims 1 to 3 on a finished metal surface; The image detection system for defects on the finished metal surface includes: 3D line laser sensor is used for laser scanning and detection of metal surfaces to collect metal surface data; The motion device is connected to the 3D line laser sensor to ensure that data can be collected from all metal surfaces; A position sensor is connected to the motion device to collect position data during each laser scan detection; The processor is connected to the 3D line laser sensor, the motion device, and the position sensor respectively to control the operation of the 3D line laser sensor, the motion device, and the position sensor; A data preprocessing module, connected to the processor, is used to obtain the reference height of the metal surface through histogram statistics based on the metal surface data, calculate the difference between each laser line and the reference height, adjust the height of each laser line based on the difference, and perform data linear interpolation completion and data regularization correction based on each adjusted laser line to obtain a grayscale image. The defect detection module, connected to the processor, is used to perform data noise reduction filtering on the grayscale image. Defect detection is performed on the grayscale image after data denoising and filtering to obtain the depth data of defects on the metal surface, and the defect is located based on the position data of the motion device to obtain the absolute position of the defect. The defect display module, connected to the processor, is used to display the grayscale image as a global image and / or a local image and / or a global 3D image.

Images