Nut thread defect detection method and system and medium

Abstract

The invention provides a nut thread defect detection method and system and a medium, and the method comprises the steps: collecting a nut image based on a high-resolution camera, and carrying out the preprocessing of the nut image, and obtaining a preprocessed image; scanning the outer surface of the screw thread of the screw cap based on a laser scanner to obtain three-dimensional point cloud data of the screw cap; extracting image features based on the preprocessed image, generating image data, and performing fusion processing on the image data and the three-dimensional point cloud data to obtain fusion data; thread parameter identification is carried out through a deep learning algorithm based on the fusion data, and thread defect information is obtained; transmitting the thread defect information to a terminal in real time according to a preset mode; through combination of image processing, a three-dimensional laser scanning technology and a deep learning algorithm, automatic detection of screw cap threads is realized, and thread defects including the size, the shape, the number and the like of the threads can be rapidly and accurately identified, so that the detection efficiency and precision of a production line are improved.

Description

Technical Field

[0001] The present application relates to the technical field of nut defect detection, and in particular to a nut thread defect detection method, system and medium. Background Art

[0002] Nuts are common standard components in mechanical fasteners, typically featuring external threads for use with bolts. During nut production, thread quality significantly impacts the nut's fastening performance and service life. Thread quality inspection is crucial for ensuring nut product compliance and reliability. Traditional inspection methods primarily include manual measurement, mechanical testing, and simple automated testing. While these methods can detect some thread defects, they still suffer from low accuracy, poor efficiency, and complex operation.

[0003] In industrial production, nut thread defects primarily manifest as thread errors (e.g., thread undulations), missing threads, non-compliant thread shapes (e.g., sharp angles, chamfers), and dimensional deviations. To improve production efficiency and quality, the use of high-precision automated inspection systems for nut thread testing has become an inevitable trend.

[0004] Currently common nut thread detection methods include:

[0005] Manual visual inspection: This method is not only inefficient, but also easily affected by human factors and has poor accuracy.

[0006] Mechanical contact measurement: Usually requires the measuring instrument to contact the thread, which may cause wear and tear errors, and difficulty in detecting small defects.

[0007] Optical image processing: This method uses an image acquisition system to capture images of the threads and then uses image processing algorithms to detect defects. However, this method performs poorly in complex environments and is easily affected by factors such as external lighting.

[0008] Although the above methods each have their own application scenarios, it is still difficult to simultaneously meet the high precision, high efficiency and stability requirements of thread detection. Summary of the Invention

[0009] The purpose of the embodiments of the present application is to provide a nut thread defect detection method, system and medium, which realizes automated detection of nut threads by combining image processing, three-dimensional laser scanning technology and deep learning algorithms, and can quickly and accurately identify thread defects, including thread size, shape, quantity, etc., thereby improving the detection efficiency and accuracy of the production line.

[0010] The present application also provides a method for detecting nut thread defects, comprising:

[0011] Capturing a nut image based on a high-resolution camera, and preprocessing the nut image to obtain a preprocessed image;

[0012] Scan the outer surface of the nut thread using a laser scanner to obtain three-dimensional point cloud data of the nut;

[0013] Extract image features based on the preprocessed image to generate image data, and fuse the image data with the three-dimensional point cloud data to obtain fused data;

[0014] Based on the fusion data, thread parameters are identified through deep learning algorithms to obtain thread defect information;

[0015] The thread defect information is transmitted to the terminal in real time according to the predetermined method.

[0016] Optionally, in the nut thread defect detection method described in the embodiment of the present application, a nut image is captured based on a high-resolution camera, and the nut image is preprocessed to obtain a preprocessed image, specifically including:

[0017] Set the light source parameters, high-resolution camera position, and nut position, and acquire the nut image based on the high-resolution camera;

[0018] Gray value processing is performed on the nut image to obtain a grayscale image;

[0019] The image noise of the grayscale image is removed based on the filtering algorithm, and the grayscale image after the noise is removed is enhanced to obtain an enhanced image;

[0020] Setting a segmentation threshold, and segmenting the thread area and the background area in the enhanced image based on the segmentation threshold;

[0021] The background area is cropped to obtain the region of interest, and a preprocessed image is generated based on the region of interest.

[0022] Optionally, in the nut thread defect detection method described in the embodiment of the present application, scanning the outer surface of the nut thread with a laser scanner to obtain three-dimensional point cloud data of the nut specifically includes:

[0023] Setting multiple scanning angles based on the laser scanner, scanning the outer surface of the nut thread according to the set scanning angles to obtain the three-dimensional coordinates of the nut thread;

[0024] Based on the three-dimensional coordinates of the nut thread, the three-dimensional coordinates of different angles at the same position of the nut thread are selected;

[0025] Perform error analysis on the three-dimensional coordinates of different angles at the same position of the nut thread to obtain the error value;

[0026] Compensate the three-dimensional coordinates based on the error value to obtain optimized three-dimensional coordinates;

[0027] All optimized 3D coordinates are integrated into 3D point cloud data of the nut.

[0028] Optionally, in the nut thread defect detection method described in the embodiment of the present application, extracting image features based on the preprocessed image to generate image data specifically includes:

[0029] The preprocessed image is analyzed based on the Canny edge detection algorithm to obtain the thread edge features and contour features;

[0030] Based on the Hough transform algorithm, the linear features and curve features of the thread are extracted from the image;

[0031] Analyze the texture features of threads from images based on local binary patterns;

[0032] Image data is generated based on thread edge features, contour features, straight line features, curve features and texture features.

[0033] Optionally, in the nut thread defect detection method described in the embodiment of the present application, fusing the image data with the three-dimensional point cloud data to obtain fused data specifically includes:

[0034] Synchronously calibrate the camera's intrinsic parameters and the laser scanner's extrinsic parameters based on a high-precision calibration plate;

[0035] Obtaining image pixel coordinates based on the image coordinate system, converting the image pixel coordinates into world coordinates, and obtaining the image pixel coordinates in the world coordinate system;

[0036] Convert the 3D point cloud data to world coordinates to obtain the 3D point cloud coordinates in the world coordinate system;

[0037] Mapping and analyzing the image pixel coordinates in the world coordinate system with the three-dimensional point cloud coordinates in the world coordinate system to obtain the correlation information between the three-dimensional point cloud data and the image data;

[0038] Information fusion is performed based on the correlation information of the three-dimensional point cloud data and the image data to obtain fused data.

[0039] Optionally, in the nut thread defect detection method described in the embodiment of the present application, thread parameter identification is performed based on the fused data using a deep learning algorithm to obtain thread defect information, specifically including:

[0040] Obtain thread data with defect annotations, input the thread data with defect annotations into the deep learning model for model training, and obtain training results;

[0041] Determining whether the training result has converged;

[0042] If convergence occurs, a defect recognition model is obtained, and the fused data is analyzed based on the defect recognition model to obtain thread parameter information, and thread defect information is generated based on the thread parameter information;

[0043] If convergence does not occur, correction information is generated and the parameters of the deep learning model are dynamically adjusted based on the correction information.

[0044] In a second aspect, an embodiment of the present application provides a nut thread defect detection system, the system comprising: a memory and a processor, the memory comprising a program for a nut thread defect detection method, and the processor executing the program for the nut thread defect detection method to implement the following steps:

[0045] Capturing a nut image based on a high-resolution camera, and preprocessing the nut image to obtain a preprocessed image;

[0046] Scan the outer surface of the nut thread using a laser scanner to obtain three-dimensional point cloud data of the nut;

[0047] Extract image features based on the preprocessed image to generate image data, and fuse the image data with the three-dimensional point cloud data to obtain fused data;

[0048] Based on the fusion data, thread parameters are identified through deep learning algorithms to obtain thread defect information;

[0049] The thread defect information is transmitted to the terminal in real time according to the predetermined method.

[0050] Optionally, in the nut thread defect detection system described in the embodiment of the present application, a nut image is captured based on a high-resolution camera, and the nut image is preprocessed to obtain a preprocessed image, specifically including:

[0051] Set the light source parameters, high-resolution camera position, and nut position, and acquire the nut image based on the high-resolution camera;

[0052] Gray value processing is performed on the nut image to obtain a grayscale image;

[0053] The image noise of the grayscale image is removed based on the filtering algorithm, and the grayscale image after the noise is removed is enhanced to obtain an enhanced image;

[0054] Setting a segmentation threshold, and segmenting the thread area and the background area in the enhanced image based on the segmentation threshold;

[0055] The background area is cropped to obtain the region of interest, and a preprocessed image is generated based on the region of interest.

[0056] Optionally, in the nut thread defect detection system provided in the embodiments of the present application, the outer surface of the nut thread is scanned based on the laser scanner to obtain three-dimensional point cloud data of the nut, and specifically includes:

[0057] A plurality of scanning angles are set based on the laser scanner, the outer surface of the nut thread is scanned according to the set scanning angles, and three-dimensional coordinates of the nut thread are obtained;

[0058] Three-dimensional coordinates of different angles at the same position of the nut thread are screened out based on the three-dimensional coordinates of the nut thread;

[0059] Error analysis is performed on the three-dimensional coordinates of different angles at the same position of the nut thread to obtain error values;

[0060] The three-dimensional coordinates are compensated based on the error values to obtain optimized three-dimensional coordinates;

[0061] All the optimized three-dimensional coordinates are integrated into three-dimensional point cloud data of the nut.

[0062] In a third aspect, the embodiments of the present application also provide a computer readable storage medium, the computer readable storage medium includes a nut thread defect detection method program, and the nut thread defect detection method program is executed by a processor to implement the steps of the nut thread defect detection method according to any one of the above.

[0063] As can be seen from the above, the nut thread defect detection method, system and medium provided in the embodiments of the present application, by acquiring a nut image based on a high-resolution camera, preprocessing the nut image to obtain a preprocessed image, scanning the outer surface of the nut thread based on a laser scanner to obtain three-dimensional point cloud data of the nut, extracting image features based on the preprocessed image to generate image data, fusing the image data and the three-dimensional point cloud data to obtain fused data, identifying thread parameters based on the fused data through a deep learning algorithm to obtain thread defect information, and transmitting the thread defect information to a terminal in real time in a predetermined manner, the automatic detection of the nut thread is realized by combining image processing, three-dimensional laser scanning technology and a deep learning algorithm, the thread defects including the size, shape and number of the thread can be quickly and accurately identified, and thus the detection efficiency and accuracy of the production line are improved. BRIEF DESCRIPTION OF DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0065] Figure 1 A flowchart of a nut thread defect detection method provided in an embodiment of the present application;

[0066] Figure 2 A flowchart of nut image preprocessing for a nut thread defect detection method provided in an embodiment of the present application;

[0067] Figure 3 A flowchart of three-dimensional point cloud data acquisition for a nut thread defect detection method provided in an embodiment of the present application;

[0068] Figure 4 A diagram showing the overall architecture of a nut thread detection system for a nut thread defect detection system provided in an embodiment of the present application;

[0069] Figure 5 A flowchart of thread defect identification and judgment of a nut thread defect detection system provided in an embodiment of the present application;

[0070] Figure 6 Schematic diagram of thread size and shape measurement of a nut thread defect detection system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work fall within the scope of protection of the present application.

[0072] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0073] Please refer to Figure 1 , Figure 1 This is a flow chart of a nut thread defect detection method in some embodiments of the present application. The nut thread defect detection method is used in a terminal device and includes the following steps:

[0074] S101, capturing a nut image using a high-resolution camera, and preprocessing the nut image to obtain a preprocessed image;

[0075] S102, scanning the outer surface of the nut thread using a laser scanner to obtain three-dimensional point cloud data of the nut;

[0076] S103, extracting image features based on the preprocessed image to generate image data, and fusing the image data with the three-dimensional point cloud data to obtain fused data;

[0077] S104, identifying thread parameters using a deep learning algorithm based on the fused data to obtain thread defect information;

[0078] S105, transmitting the thread defect information to the terminal in real time according to a predetermined method.

[0079] It should be noted that image acquisition uses a high-resolution camera and a customized light source to capture the nut at a fixed angle. The threads in the image are converted into pixel data and further analyzed by a computer image processing system.

[0080] A laser scanner illuminates the outer surface of a nut with a laser beam, generating a 3D point cloud dataset containing the spatial coordinates of every point on the nut's surface. This point cloud data, captured through laser scanning, accurately reflects the true 3D shape of the thread and can be combined with image data for comprehensive analysis.

[0081] Image data and three-dimensional point cloud data are synthesized through data fusion algorithms (such as ICP algorithm or point cloud registration algorithm), so that the thread detection system can simultaneously utilize surface texture features and three-dimensional geometric information to perform more accurate thread defect detection.

[0082] Please refer to Figure 2 , Figure 2 This is a flowchart of a nut image preprocessing method for detecting nut thread defects in some embodiments of the present application. According to an embodiment of the present invention, a nut image is captured using a high-resolution camera and preprocessed to obtain a preprocessed image, specifically including:

[0083] S201, setting light source parameters, high-resolution camera position, and nut position, and acquiring a nut image using the high-resolution camera;

[0084] S202, gray-value processing is performed on the nut image to obtain a grayscale image;

[0085] S203, removing image noise from the grayscale image based on a filtering algorithm, and performing enhancement processing on the grayscale image after noise removal to obtain an enhanced image;

[0086] S204, setting a segmentation threshold, and segmenting the thread area and the background area in the enhanced image based on the segmentation threshold;

[0087] S205, cropping a background region to obtain a region of interest, and generating a preprocessed image based on the region of interest.

[0088] It should be noted that the collected image is preprocessed, such as denoising, grayscale, edge detection, etc., to eliminate noise interference and improve image quality. Common image preprocessing algorithms include Gaussian filtering, Sobel operator and Canny edge detection.

[0089] Please refer to Figure 3 , Figure 3 is a three-dimensional point cloud data acquisition flowchart of a nut thread defect detection method in some embodiments of the application. According to the embodiment of the application, the outer surface of the nut thread is scanned based on the laser scanner, and the three-dimensional point cloud data of the nut is obtained, which specifically includes:

[0090] S301, based on the laser scanner, a plurality of scanning angles are set, and the outer surface of the nut thread is scanned according to the set scanning angle to obtain the three-dimensional coordinates of the nut thread;

[0091] S302, based on the three-dimensional coordinates of the nut thread, three-dimensional coordinates of different angles at the same position of the nut thread are selected;

[0092] S303, error analysis is performed on the three-dimensional coordinates of different angles at the same position of the nut thread to obtain error values;

[0093] S304, based on the error values, the three-dimensional coordinates are compensated to obtain optimized three-dimensional coordinates;

[0094] S305, all optimized three-dimensional coordinates are integrated into three-dimensional point cloud data of the nut.

[0095] It should be noted that by converting the three-dimensional point cloud data and the image data to the same world coordinate system, the mapping relationship between the three-dimensional point cloud and the image data is analyzed, and the coordinate error is analyzed, and the three-dimensional coordinates are dynamically compensated to improve the accuracy of the three-dimensional point cloud data.

[0096] According to the embodiment of the application, image features are extracted based on the preprocessed image to generate image data, which specifically includes:

[0097] Based on the Canny edge detection algorithm, the thread edge features and contour features are obtained from the preprocessed image;

[0098] Based on the Hough transform algorithm, the straight line features and curve features of the thread are extracted from the image;

[0099] Based on the local binary pattern, the texture features of the thread are analyzed from the image;

[0100] Based on the thread edge features, contour features, straight line features, curve features and texture features, the image data is generated.

[0101] It should be noted that the feature points extracted from the thread image mainly include the peak value, valley value, pitch, diameter, etc. The geometric features of the thread such as straight lines and curves are extracted from the image using technologies such as Hough transform.

[0102] According to an embodiment of the present invention, the image data and the three-dimensional point cloud data are fused to obtain fused data, which specifically includes:

[0103] Synchronously calibrate the camera's intrinsic parameters and the laser scanner's extrinsic parameters based on a high-precision calibration plate;

[0104] Obtaining image pixel coordinates based on the image coordinate system, converting the image pixel coordinates into world coordinates, and obtaining the image pixel coordinates in the world coordinate system;

[0105] Convert the 3D point cloud data to world coordinates to obtain the 3D point cloud coordinates in the world coordinate system;

[0106] Mapping and analyzing the image pixel coordinates in the world coordinate system with the three-dimensional point cloud coordinates in the world coordinate system to obtain the correlation information between the three-dimensional point cloud data and the image data;

[0107] Information fusion is performed based on the correlation information of the three-dimensional point cloud data and the image data to obtain fused data.

[0108] It should be noted that by converting the three-dimensional power data and image data into world coordinates and performing data fusion in the same coordinate system, the fusion accuracy is improved.

[0109] According to an embodiment of the present invention, thread parameter identification is performed based on fusion data using a deep learning algorithm to obtain thread defect information, specifically including:

[0110] Obtain thread data with defect annotations, input the thread data with defect annotations into the deep learning model for model training, and obtain training results;

[0111] Determine whether the training results converge;

[0112] If convergence occurs, a defect recognition model is obtained, and the fused data is analyzed based on the defect recognition model to obtain thread parameter information, and thread defect information is generated based on the thread parameter information;

[0113] If convergence does not occur, correction information is generated and the parameters of the deep learning model are dynamically adjusted based on the correction information.

[0114] It should be noted that thread defect detection, based on image features and 3D point cloud data, is performed using deep learning algorithms (such as convolutional neural networks (CNNs)). This system detects problems such as irregular thread shapes, missing threads, and substandard dimensions. This system not only achieves high accuracy but also significantly improves detection efficiency, overcoming the limitations of traditional detection methods. Experimental results demonstrate that the system achieves an accuracy of 99.5% and can inspect up to 300 nuts per minute, significantly improving production line efficiency.

[0115] According to an embodiment of the present invention, the method further includes: obtaining thread characteristic parameters, performing preliminary identification of thread defects based on a defect identification model, and analyzing whether the thread has geometric defects;

[0116] If there are geometric defects, analyze whether the threads have image defects;

[0117] If there is an image defect, record the defect type and locate the thread defect;

[0118] If there is no image defect or no geometric defect, the analysis result is output;

[0119] Analyze whether the thread meets the quality judgment standard based on the defect type;

[0120] If the quality judgment standard is met, the thread is judged to be qualified;

[0121] If the quality judgment standards are not met, the thread is judged to be unqualified.

[0122] Specifically, based on the 3D point cloud data, the geometric dimensions of the thread are measured using the following formula:

[0123]

[0124] D outer Indicates the outer diameter of the thread (external maximum diameter), that is, the maximum edge distance of the nut's external thread.

[0125] D inner Indicates the inside diameter of the thread (minimum internal diameter), that is, the distance to the deepest valley or the minimum drift diameter that engages with the mating bolt.

[0126] Among them, x max and y max is the maximum coordinate of the thread outer diameter, x min and y min The minimum coordinate of the inner diameter is used to calculate the outer diameter and inner diameter, and further determine the specifications of the thread.

[0127] like Figure 4-Figure 6As shown, in a second aspect, an embodiment of the present application provides a nut thread defect detection system, the system comprising: a memory and a processor, the memory comprising a program of a nut thread defect detection method, and when the program of the nut thread defect detection method is executed by the processor, the following steps are implemented:

[0128] Capturing a nut image based on a high-resolution camera, and preprocessing the nut image to obtain a preprocessed image;

[0129] Scan the outer surface of the nut thread using a laser scanner to obtain three-dimensional point cloud data of the nut;

[0130] Extract image features based on the preprocessed image to generate image data, and fuse the image data with the three-dimensional point cloud data to obtain fused data;

[0131] Based on the fusion data, thread parameters are identified through deep learning algorithms to obtain thread defect information;

[0132] The thread defect information is transmitted to the terminal in real time according to the predetermined method.

[0133] It should be noted that the system includes the following main modules:

[0134] Image acquisition module: This module includes a high-precision industrial camera, light source, and lens system to capture image data of the nut threads. The image acquisition module is connected to the computer processing unit to transmit image data in real time.

[0135] 3D laser scanning module: Use a laser scanner to scan the outer surface of the nut thread to obtain 3D point cloud data for determining the shape and size of the thread.

[0136] Image processing and analysis module: includes image preprocessing, feature extraction and defect analysis algorithms, combined with deep learning models for thread shape recognition and defect detection.

[0137] Defect judgment module: Based on the data provided by the image processing module and the standard size and shape requirements of the thread, it automatically determines whether it is qualified.

[0138] Display and feedback module: The test results are displayed on the display screen. If defects are found, they are fed back to the production line through the alarm device or automatic control system for the next step of processing.

[0139] According to an embodiment of the present invention, a nut image is captured using a high-resolution camera, and the nut image is preprocessed to obtain a preprocessed image, specifically including:

[0140] Set the light source parameters, high-resolution camera position, and nut position, and acquire the nut image based on the high-resolution camera;

[0141] Gray value processing is performed on the nut image to obtain a grayscale image;

[0142] The image noise of the grayscale image is removed based on the filtering algorithm, and the grayscale image after the noise is removed is enhanced to obtain an enhanced image;

[0143] Setting a segmentation threshold, and segmenting the thread area and the background area in the enhanced image based on the segmentation threshold;

[0144] The background area is cropped to obtain the region of interest, and a preprocessed image is generated based on the region of interest.

[0145] It should be noted that the collected images are preprocessed by denoising, grayscale conversion, and edge detection to eliminate noise interference and improve image quality. Common image preprocessing algorithms include Gaussian filtering, Sobel operator, and Canny edge detection.

[0146] According to an embodiment of the present invention, scanning the outer surface of the nut thread with a laser scanner to obtain three-dimensional point cloud data of the nut specifically includes:

[0147] Setting multiple scanning angles based on the laser scanner, scanning the outer surface of the nut thread according to the set scanning angles to obtain the three-dimensional coordinates of the nut thread;

[0148] Based on the three-dimensional coordinates of the nut thread, the three-dimensional coordinates of different angles at the same position of the nut thread are selected;

[0149] Perform error analysis on the three-dimensional coordinates of different angles at the same position of the nut thread to obtain the error value;

[0150] Compensate the three-dimensional coordinates based on the error value to obtain optimized three-dimensional coordinates;

[0151] All optimized 3D coordinates are integrated into 3D point cloud data of the nut.

[0152] It should be noted that by uniformly converting the three-dimensional point cloud data and image data into the same world coordinate system, the mapping relationship between the three-dimensional point cloud and the image data is analyzed, and then the coordinate error is analyzed, and the three-dimensional coordinates are dynamically compensated to improve the accuracy of the three-dimensional point cloud data.

[0153] According to an embodiment of the present invention, extracting image features based on the preprocessed image and generating image data specifically includes:

[0154] The preprocessed image is analyzed based on the Canny edge detection algorithm to obtain the thread edge features and contour features;

[0155] Based on the Hough transform algorithm, the linear features and curve features of the thread are extracted from the image;

[0156] Analyze the texture features of the thread from the image based on a local binary pattern;

[0157] Generate the image data based on the thread edge features, the contour features, the straight line features, the curve features and the texture features.

[0158] It should be noted that the feature points in the thread image are extracted, mainly including the peak value, the valley value, the pitch, the diameter and the like of the thread. The Hough transform and the like are adopted to extract the straight line features, the curve features and the like of the thread from the image.

[0159] According to the embodiment of the present application, the image data and the three-dimensional point cloud data are fused to obtain fused data, specifically including:

[0160] Synchronously calibrate the camera internal parameter and the laser scanner external parameter based on the high-precision calibration board;

[0161] Obtain the image pixel coordinates based on the image coordinate system, convert the image pixel coordinates into the world coordinates to obtain the image pixel coordinates in the world coordinate system;

[0162] Convert the three-dimensional point cloud data into the world coordinates to obtain the three-dimensional point cloud coordinates in the world coordinate system;

[0163] Map and analyze the image pixel coordinates in the world coordinate system and the three-dimensional point cloud coordinates in the world coordinate system to obtain the correlation information between the three-dimensional point cloud data and the image data;

[0164] Fuse the information based on the correlation information between the three-dimensional point cloud data and the image data to obtain the fused data.

[0165] It should be noted that the three-dimensional power supply data and the image data are converted into the world coordinates, and the data fusion is performed in the same coordinate system, so that the fusion precision is improved.

[0166] According to the embodiment of the present application, the thread parameter recognition is performed based on the fused data through the deep learning algorithm to obtain the thread defect information, specifically including:

[0167] Obtain the thread data with defect labels, input the thread data with defect labels into the deep learning model to perform model training to obtain a training result;

[0168] Judge whether the training result converges or not;

[0169] If the training result converges, a defect recognition model is obtained, the fused data is analyzed based on the defect recognition model to obtain thread parameter information, and the thread defect information is generated based on the thread parameter information;

[0170] If the training result does not converge, correction information is generated, and the parameters of the deep learning model are dynamically adjusted based on the correction information.

[0171] It should be noted that thread defect detection, based on image features and 3D point cloud data, is performed using deep learning algorithms (such as convolutional neural networks (CNNs)). This system detects problems such as irregular thread shapes, missing threads, and substandard dimensions. This system not only achieves high accuracy but also significantly improves detection efficiency, overcoming the limitations of traditional detection methods. Experimental results demonstrate that the system achieves an accuracy of 99.5% and can inspect up to 300 nuts per minute, significantly improving production line efficiency.

[0172] According to an embodiment of the present invention, the method further includes: obtaining thread characteristic parameters, performing preliminary identification of thread defects based on a defect identification model, and analyzing whether the thread has geometric defects;

[0173] If there are geometric defects, analyze whether the threads have image defects;

[0174] If there is an image defect, record the defect type and locate the thread defect;

[0175] If there is no image defect or no geometric defect, the analysis result is output;

[0176] Analyze whether the thread meets the quality judgment standard based on the defect type;

[0177] If the quality judgment standard is met, the thread is judged to be qualified;

[0178] If the quality judgment standards are not met, the thread is judged to be unqualified.

[0179] Specifically, based on the 3D point cloud data, the geometric dimensions of the thread are measured using the following formula:

[0180]

[0181] Among them, x max and y max is the maximum coordinate of the thread outer diameter, x min and y min The minimum coordinate of the inner diameter is used to calculate the outer diameter and inner diameter, and further determine the specifications of the thread.

[0182] A third aspect of the present invention provides a computer-readable storage medium, which includes a nut thread defect detection method program. When the nut thread defect detection method program is executed by a processor, the steps of the nut thread defect detection method as described above are implemented.

[0183] The present invention discloses a nut thread defect detection method, system and medium, which capture nut images with a high-resolution camera, pre-process the nut images to obtain pre-processed images; scan the outer surface of the nut thread with a laser scanner to obtain three-dimensional point cloud data of the nut; extract image features based on the pre-processed image to generate image data, and fuse the image data with the three-dimensional point cloud data to obtain fused data; identify thread parameters based on the fused data using a deep learning algorithm to obtain thread defect information; transmit the thread defect information to a terminal in real time according to a predetermined method; and realize automated detection of nut threads by combining image processing, three-dimensional laser scanning technology and deep learning algorithms, so as to quickly and accurately identify thread defects, including thread size, shape, quantity, etc., thereby improving the detection efficiency and accuracy of the production line.

[0184] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0185] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0186] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0187] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0188] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the existing technology, can be embodied in the form of a software product. The software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

Claims

1. A method for detecting nut thread defects, characterized in that: include: Capturing a nut image based on a high-resolution camera, and preprocessing the nut image to obtain a preprocessed image; Scan the outer surface of the nut thread using a laser scanner to obtain three-dimensional point cloud data of the nut; Extract image features based on the preprocessed image to generate image data, and fuse the image data with the three-dimensional point cloud data to obtain fused data; Based on the fusion data, thread parameters are identified through deep learning algorithms to obtain thread defect information; The thread defect information is transmitted to the terminal in real time according to the predetermined method.

2. The nut thread defect detection method according to claim 1, characterized in that: The nut image is captured based on a high-resolution camera, and the nut image is preprocessed to obtain a preprocessed image, specifically including: Set the light source parameters, high-resolution camera position, and nut position, and acquire the nut image based on the high-resolution camera; Gray value processing is performed on the nut image to obtain a grayscale image; The image noise of the grayscale image is removed based on the filtering algorithm, and the grayscale image after the noise is removed is enhanced to obtain an enhanced image; Setting a segmentation threshold, and segmenting the thread area and the background area in the enhanced image based on the segmentation threshold; The background area is cropped to obtain the region of interest, and a preprocessed image is generated based on the region of interest.

3. The nut thread defect detection method according to claim 2, characterized in that: The laser scanner is used to scan the outer surface of the nut thread to obtain the 3D point cloud data of the nut, including: Setting multiple scanning angles based on the laser scanner, scanning the outer surface of the nut thread according to the set scanning angles to obtain the three-dimensional coordinates of the nut thread; Based on the three-dimensional coordinates of the nut thread, the three-dimensional coordinates of different angles at the same position of the nut thread are selected; Perform error analysis on the three-dimensional coordinates of different angles at the same position of the nut thread to obtain the error value; Compensate the three-dimensional coordinates based on the error value to obtain optimized three-dimensional coordinates; All optimized 3D coordinates are integrated into 3D point cloud data of the nut.

4. The method for detecting nut thread defects according to claim 3, wherein: Extract image features based on preprocessed images and generate image data, specifically including: The preprocessed image is analyzed based on the Canny edge detection algorithm to obtain the thread edge features and contour features; Based on the Hough transform algorithm, the linear features and curve features of the thread are extracted from the image; Analyze the texture features of threads from images based on local binary patterns; Image data is generated based on thread edge features, contour features, straight line features, curve features and texture features.

5. The method for detecting nut thread defects according to claim 4, characterized in that: The image data and the 3D point cloud data are fused to obtain fused data, which specifically includes: Synchronously calibrate the camera's intrinsic parameters and the laser scanner's extrinsic parameters based on a high-precision calibration plate; Obtaining image pixel coordinates based on the image coordinate system, converting the image pixel coordinates into world coordinates, and obtaining the image pixel coordinates in the world coordinate system; Convert the 3D point cloud data to world coordinates to obtain the 3D point cloud coordinates in the world coordinate system; Mapping and analyzing the image pixel coordinates in the world coordinate system with the three-dimensional point cloud coordinates in the world coordinate system to obtain the correlation information between the three-dimensional point cloud data and the image data; Information fusion is performed based on the correlation information of the three-dimensional point cloud data and the image data to obtain fused data.

6. The method for detecting nut thread defects according to claim 5, characterized in that: Based on the fusion data, thread parameters are identified through deep learning algorithms to obtain thread defect information, including: Obtain thread data with defect annotations, input the thread data with defect annotations into the deep learning model for model training, and obtain training results; Determining whether the training result has converged; If convergence occurs, a defect recognition model is obtained, and the fused data is analyzed based on the defect recognition model to obtain thread parameter information, and thread defect information is generated based on the thread parameter information; If convergence does not occur, correction information is generated and the parameters of the deep learning model are dynamically adjusted based on the correction information.

7. A nut thread defect detection system, characterized in that: The system includes: a memory and a processor, wherein the memory includes a program of a nut thread defect detection method, and when the program of the nut thread defect detection method is executed by the processor, the following steps are implemented: Capturing a nut image based on a high-resolution camera, and preprocessing the nut image to obtain a preprocessed image; Scan the outer surface of the nut thread using a laser scanner to obtain three-dimensional point cloud data of the nut; Extract image features based on the preprocessed image to generate image data, and fuse the image data with the three-dimensional point cloud data to obtain fused data; Based on the fusion data, thread parameters are identified through deep learning algorithms to obtain thread defect information; The thread defect information is transmitted to the terminal in real time according to the predetermined method.

8. The nut thread defect detection system according to claim 7, characterized in that: The nut image is captured based on a high-resolution camera, and the nut image is preprocessed to obtain a preprocessed image, specifically including: Set the light source parameters, high-resolution camera position, and nut position, and acquire the nut image based on the high-resolution camera; Gray value processing is performed on the nut image to obtain a grayscale image; The image noise of the grayscale image is removed based on the filtering algorithm, and the grayscale image after the noise is removed is enhanced to obtain an enhanced image; Setting a segmentation threshold, and segmenting the thread area and the background area in the enhanced image based on the segmentation threshold; The background area is cropped to obtain the region of interest, and a preprocessed image is generated based on the region of interest.

9. The nut thread defect detection system according to claim 8, characterized in that: The laser scanner is used to scan the outer surface of the nut thread to obtain the 3D point cloud data of the nut, including: Setting multiple scanning angles based on the laser scanner, scanning the outer surface of the nut thread according to the set scanning angles to obtain the three-dimensional coordinates of the nut thread; Based on the three-dimensional coordinates of the nut thread, the three-dimensional coordinates of different angles at the same position of the nut thread are selected; Perform error analysis on the three-dimensional coordinates of different angles at the same position of the nut thread to obtain the error value; Compensate the three-dimensional coordinates based on the error value to obtain optimized three-dimensional coordinates; All optimized 3D coordinates are integrated into 3D point cloud data of the nut.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a nut thread defect detection method program, and when the nut thread defect detection method program is executed by a processor, the steps of the nut thread defect detection method according to any one of claims 1 to 6 are implemented.

Images