Board dark crack detection method and system

Abstract

The invention relates to the technical field of plate quality detection, and discloses a plate dark crack detection method and system. The method comprises the following steps: firstly, acquiring 3D point cloud data and surface image data of a plate through a depth camera, and preprocessing the 3D point cloud data and the surface image data, including the steps of filtering, denoising, downsampling, image enhancement, gray adjustment and the like; and then, inputting the preprocessed data into a plate dark crack detection model trained by a deep learning algorithm based on M3DM, and outputting the probability that the plate has dark cracks by the model. And finally, according to the output probability value, the plates with the dark cracks are sorted out through automatic sorting equipment. The system further comprises a data acquisition module, a data preprocessing module, a dark crack detection module, a sorting control module, a user interaction module and the like, accurate detection of the plate dark cracks is achieved, the detection efficiency and accuracy are improved, the three-dimensional features of the surface of the plate are comprehensively captured, the problems existing in a traditional detection method are effectively solved, and the detection accuracy is improved. And a new technical means is provided for plate quality detection.

Description

Technical Field

[0001] The present invention relates to the technical field of plate quality detection, and in particular to a plate hidden crack detection method and system. Background Art

[0002] During the processing and use of sheet metal, dark cracks are an important factor affecting the quality and safety of sheet metal. Traditional methods for detecting dark cracks in sheet metal mainly rely on manual visual inspection, which is not only inefficient but also easily affected by human factors, resulting in inaccurate detection results. With the development of industrial automation and artificial intelligence technology, the demand for automated detection of dark cracks in sheet metal is becoming increasingly urgent. However, since dark cracks in sheet metal are usually difficult to observe directly and have various forms, automated detection faces great challenges. At present, although there are some sheet metal detection methods based on image processing, most of them can only process two-dimensional image data, making it difficult to fully capture the three-dimensional features of the sheet metal surface, resulting in limited detection results. Summary of the invention

[0003] In view of the above-mentioned technical deficiencies, the purpose of the present invention is to provide a method and system for detecting hidden cracks in plate materials, so as to solve the problems of inaccurate detection and low efficiency in the prior art.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a method for detecting hidden cracks in a plate, the method comprising: Step S100: Acquire real-time plate 3D point cloud data and plate surface image data; Step S200: performing data preprocessing on the real-time plate 3D point cloud data and plate surface image; Step S300: inputting the real-time plate 3D point cloud data and plate surface image data into the plate dark crack detection model, and outputting the probability of the plate having dark cracks; Step S400: judging whether the plate has dark cracks according to the probability value of the plate having dark cracks and a preset threshold; Step S500: using automated sorting equipment to sort out the plates with hidden cracks.

[0005] Preferably, in a possible implementation manner of the first aspect, the 3D point cloud data and the surface image data are acquired by a depth camera.

[0006] Preferably, in a possible implementation manner of the first aspect, the data preprocessing includes filtering, denoising and downsampling the plate 3D point cloud data; and performing image enhancement and grayscale adjustment on the plate surface image data.

[0007] Preferably, in a possible implementation manner of the first aspect, the plate dark crack detection model is trained using a deep learning algorithm based on M3DM.

[0008] Preferably, in a possible implementation manner of the first aspect, the M3DM-based deep learning algorithm training process includes: The plate 3D point cloud data and plate surface image data are used as training samples, and the plate 3D point cloud data and plate surface image data include plates with dark cracks and plates without dark cracks; Build a deep learning network architecture based on M3DM; The training samples are input into the deep learning network for training, and the network parameters are adjusted through the back-propagation algorithm until the performance of the model on the validation set reaches the preset standard.

[0009] Preferably, in a possible implementation manner of the first aspect, the M3DM-based deep learning network architecture specifically includes: A branch network for processing 3D point cloud data, which uses Point Transformer to extract point cloud features and is used to extract spatial features from 3D point cloud data; A branch network for processing surface image data, wherein the branch network includes a convolutional neural network structure and is used to extract texture and color features from the surface image data; The feature fusion layer is used to fuse the features extracted by the two branch networks and integrate the information of different modalities through the unsupervised feature fusion module; The classifier layer receives the fused features and adopts the decision layer fusion strategy to output the probability value of the presence of dark cracks in the plate.

[0010] Preferably, in a possible implementation manner of the first aspect, step S300 specifically includes: Input the preprocessed plate 3D point cloud data and surface image data into the trained plate dark crack detection model; The model extracts and fuses point cloud features and image features of the input data; Based on the extracted and fused features, the model outputs the probability value of the presence of dark cracks in the plate through the classifier.

[0011] Preferably, in a possible implementation manner of the first aspect, step S400 specifically includes: According to the probability value output by the plate dark crack detection model, a preset threshold is set; When the probability value of dark cracks in the plate exceeds the threshold, the plate is determined to have dark cracks.

[0012] In a second aspect, the present invention provides a plate hidden crack detection system, the system comprising: A data acquisition module is used to obtain 3D point cloud data and surface image data of the plate through a depth camera; The data preprocessing module is used to filter, denoise and downsample the collected 3D point cloud data, and to enhance and adjust the grayscale of the surface image data; The dark crack detection module is used to input the preprocessed data into the plate dark crack detection model trained by the deep learning algorithm based on M3DM, and output the probability of the existence of dark cracks in the plate; A sorting control module, used to control the automated sorting equipment to sort out the plates with dark cracks according to the probability value output by the dark crack detection module; The user interaction module is used to display the detection results, set thresholds and provide a user operation interface.

[0013] The beneficial effect of the present invention is that by acquiring the 3D point cloud data and surface image data of the plate, preprocessing them, and then inputting them into the plate dark crack detection model trained by the deep learning algorithm based on M3DM, accurate detection of dark cracks in the plate is achieved. This method not only improves the detection efficiency and accuracy, but also can fully capture the three-dimensional features of the plate surface, effectively solving the problems existing in traditional detection methods. In addition, the plates with dark cracks are sorted by automated sorting equipment, further improving the safety and reliability of plate processing and use. The detection method and system of the present invention have broad application prospects, can be used for dark crack detection of various plates, and provide a new technical means for plate quality detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 A flow chart of a method for detecting hidden cracks in a plate is provided for this application.

[0016] Figure 2 A structural diagram of a plate hidden crack detection system is provided for this application.

[0017] Explanation of the accompanying drawings: 1-data acquisition module, 2-data preprocessing module, 3-dark crack detection module, 4-sorting control module, 5-user interaction module. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Embodiment 1: Figure 1 As shown, the present invention provides a method for detecting hidden cracks in a plate, the method comprising: Step S100: Acquire real-time plate 3D point cloud data and plate surface image data.

[0020] Specifically, 3D point cloud data and surface image data are acquired through a depth camera, which is deployed above the dark crack detection area of ​​the plate. After the depth camera completes data collection, it transmits the 3D point cloud data and surface image data to the control center.

[0021] In this embodiment, the depth camera adopts the HVC-SLB1000B model. The HVC-SLB1000B camera uses structured light projection technology to project specific structured light patterns onto the surface of the plate through a projector. These structured light patterns are composed of a series of light with specific phases and frequencies. When they interact with the surface of the plate, they produce specific reflection and scattering patterns. Subsequently, the camera inside the camera captures these projected structured light patterns and records the light information after reflection and scattering. After acquiring the image data, the HVC-SLB1000B camera will perform image processing steps. Including algorithms such as phase extraction and triangulation. Phase extraction mainly extracts the phase information of the structured light pattern from the captured image. Triangulation uses the known structured light pattern and the relative position relationship between the camera and the projector to determine the coordinates of each point on the surface of the plate in three-dimensional space by calculating the phase difference of each point in the image. After image processing, the HVC-SLB1000B camera generates high-precision 3D point cloud data on the surface of the plate. It contains the three-dimensional coordinates (X, Y, Z) of each point on the surface of the plate, as well as possible additional information such as color and intensity. The HVC-SLB1000B camera obtains image data of the plate surface through the color camera inside the camera. The color camera can capture the color, texture and other visual features of the plate surface and generate a two-dimensional image.

[0022] Step S200: preprocessing the real-time plate 3D point cloud data and plate surface image.

[0023] Specifically, data preprocessing includes filtering, denoising and downsampling the plate 3D point cloud data; and image enhancement and grayscale adjustment of the plate surface image data.

[0024] In this embodiment, the plate 3D point cloud data is preprocessed first, and a function named preprocess_point_cloud is defined. The function receives the point cloud object pcd as input and returns the preprocessed point cloud object.

[0025] The function implements three processing steps: filtering, denoising and downsampling. Filtering uses the remove_statistical_outlier method to remove outliers in the point cloud. This method determines which points are outliers based on the average distance and standard deviation between each point and its neighboring points. In this embodiment, the number of neighboring points is set to 20 and the standard deviation multiplier is 2.0. After removing the outliers, a new point cloud object inlier_cloud is obtained. Denoising uses the remove_radius_outlier method to remove noise points in the point cloud. This method determines which points are noise points based on the number of neighboring points within a given radius for each point. In this embodiment, the number of neighboring points is set to 10 and the radius is set to 0.02, and the inlier_cloud obtained in the previous step is processed to obtain the denoised point cloud object denoised_cloud. Downsampling uses the voxel_down_sample method to downsample the point cloud. The number of points in the point cloud is reduced by constructing a voxel grid, thereby reducing the complexity of the data. In this embodiment, the size of the voxel grid is set to 0.005, and the downsampled point cloud object down_pcd is obtained, and the number of data points of the processed point cloud is printed out. After completing the above processing steps, the function returns the downsampled point cloud object down_pcd.

[0026] Next, call the preprocess_point_cloud function to preprocess the loaded point cloud data and obtain the preprocessed point cloud object preprocessed_pcd. For the preprocessing of the plate surface image data, first define a function named preprocess_image, which receives a parameter image_path, which represents the image file to be processed. Inside the function, use the cv2.imread function to read the image file under the specified path, and store the read image data in the variable image. Next, perform image enhancement processing. First, convert the color image to a grayscale image using the cv2.cvtColor function and specify the conversion code as cv2.COLOR_BGR2GRAY. After obtaining the grayscale image, use the cv2.equalizeHist function to perform histogram equalization processing on it to obtain the enhanced grayscale image equalized_image. Then perform grayscale adjustment. In order to linearly transform the grayscale value range from [0, 255] to [50, 200], use the cv2.convertScaleAbs function and set the scaling factor alpha to 0.6 and the offset beta to 50. In this way, the adjusted grayscale image adjusted_image is obtained. Finally, the function returns the adjusted grayscale image adjusted_image.

[0027] Step S300: input the real-time plate 3D point cloud data and plate surface image data into the plate dark crack detection model, and output the probability of the plate having dark cracks.

[0028] Specifically, the plate dark crack detection model is trained using a deep learning algorithm based on M3DM. The deep learning algorithm training process based on M3DM includes: using historical plate 3D point cloud data and plate surface image data as training samples, the historical plate 3D point cloud data and plate surface image data contain plates with dark cracks and plates without dark cracks and corresponding annotations, and the historical plate 3D point cloud data and plate surface image data are obtained from the control center; building a deep learning network architecture based on M3DM; inputting training samples into the deep learning network for training, and adjusting network parameters through the back propagation algorithm until the performance of the model on the validation set reaches the preset standard.

[0029] The deep learning network architecture based on M3DM specifically includes: a branch network for processing 3D point cloud data, which uses Point Transformer to extract point cloud features and is used to extract spatial features from 3D point cloud data; a branch network for processing surface image data, which contains a convolutional neural network structure and is used to extract texture and color features from surface image data; a feature fusion layer, which is used to fuse the features extracted by the two branch networks and integrate information from different modalities through an unsupervised feature fusion module; a classifier layer, which receives the fused features, adopts a decision layer fusion strategy, and outputs the probability value of the presence of dark cracks in the plate.

[0030] The preprocessed 3D point cloud data and surface image data of the plate are input into the trained plate dark crack detection model; the model extracts and fuses the point cloud features and image features of the input data; based on the extracted and fused features, the model outputs the probability value of the existence of dark cracks in the plate through a classifier.

[0031] In this embodiment, during the model training phase, plate 3D point cloud data and plate surface image data are used as training samples. The samples cover plates with dark cracks and plates without dark cracks to ensure the generalization ability and accuracy of the model. Subsequently, a deep learning network architecture based on M3DM is constructed, which consists of two main branch networks: one for processing 3D point cloud data and the other for processing surface image data. Point Transformer is used to extract point cloud features. The input point cloud p is a point position sequence with N points. After sampling the farthest point, the point cloud is divided into M groups, each with S points. The points in each group are then encoded as feature vectors, and the M vectors are input into Point Transformer. The output g is M point features, which are then organized into point feature groups, each with a single point feature, which can be regarded as the feature of the center point. Since the center point of each point is unevenly distributed in space after sampling the farthest point, this leads to an unbalanced density of the point features, and the features are interpolated back to the original point cloud. Given the center point with M groups Related M point features , using inverse distance weights to interpolate features to each point in the input point cloud , the process is:

[0032]

[0033] After interpolation, the point coordinates and camera parameters are used to Projected onto the 2D plane, the projection point is , the projected feature map is expressed as , D is the 2D plane area of ​​the RGB image, which has the same size as the input RGB image. Finally, the average pooling operation is used to obtain the patch features on the 2D plane feature map. After the feature extraction is completed, the feature fusion layer is used to fuse the features extracted by the two branch networks, integrate the information of different modalities, and use the piece-by-piece contrast loss to train the feature fusion module so that the features of different modalities from the same position have more corresponding information, while the features of different positions have less corresponding information. Finally, the fused features are sent to the classifier layer for the final classification decision. The classifier layer adopts the decision layer fusion strategy, using 3 storage libraries to store the original color features, position features and fused features. Each storage library is used to predict the anomaly score and segmentation map, and then two learnable one-class support vector machines are used to make the final decision on the anomaly score and segmentation map. The probability value of the existence of dark cracks in the plate is output according to the anomaly score and segmentation map.

[0034] After the model training is completed, the preprocessed 3D point cloud data and surface image data of the plate are used as input and passed to the trained plate dark crack detection model. Inside the model, the point cloud data is processed by the Point Transformer branch to extract the spatial features in the point cloud, while the surface image data is processed by the convolutional neural network branch to extract features such as texture, color, and possible crack traces in the image. After the feature fusion layer in the model receives the features extracted by the two branch networks, it integrates the information from different modalities through the unsupervised feature fusion module.

[0035] After feature fusion, the model sends these comprehensive features to the classifier layer. The classifier layer adopts the decision layer fusion strategy, combines the original color features, position features and fused features in the three repositories, and predicts the anomaly score and segmentation map for each feature. Then, two learnable one-class support vector machines (One-Class SVM) are used to comprehensively analyze the anomaly score and segmentation map, and finally output the probability value of the presence of dark cracks in the plate. This probability value is a value between 0 and 1, reflecting the possibility of the presence of dark cracks in the plate. The closer the probability value is to 1, the greater the possibility of the presence of dark cracks in the plate; conversely, the closer the probability value is to 0, the smaller the possibility of the presence of dark cracks in the plate.

[0036] Step S400: judging whether the plate has dark cracks according to the probability value of the plate having dark cracks and a preset threshold.

[0037] Specifically, a preset threshold is set according to the probability value output by the plate dark crack detection model; when the dark crack probability value of the plate exceeds the threshold, the plate is determined to have dark cracks.

[0038] In this embodiment, the threshold is set to 0.4. When the plate dark crack detection model completes the detection of the plate and outputs the probability value of the plate having dark cracks, the system compares this probability value with the preset threshold. If the plate dark crack probability value is greater than the threshold of 0.4, it will be determined that the plate has dark cracks and needs to be sorted out. If the plate dark crack probability value is less than or equal to 0.4, it is considered that the plate does not have dark cracks and continues to stay on the production line for subsequent processing or treatment.

[0039] Step S500: using automated sorting equipment to sort out the plates with hidden cracks.

[0040] In this embodiment, the automated sorting equipment uses a machine vision system to locate the plate, ensuring that the position information of the plate can be accurately captured and the movement and position change of the plate can be monitored in real time. Then, the control center of the sorting equipment calculates the best sorting path and action based on the judgment result of the plate having dark cracks output in step S400 and the plate position information of the machine vision system, and the control center sends a command to the robot arm to start the sorting operation. After the control center issues the command, the robot arm performs the sorting action, removes the plate with dark cracks from the production line, and places it in the specified position.

[0041] Embodiment 2: Figure 2 As shown, the present invention provides a plate hidden crack detection system, the system comprising: Data acquisition module 1, used to obtain 3D point cloud data and surface image data of the plate through a depth camera; Data preprocessing module 2, used for filtering, denoising and downsampling the collected 3D point cloud data, and performing image enhancement and grayscale adjustment on the surface image data; Dark crack detection module 3, used to input the preprocessed data into the plate dark crack detection model trained by the deep learning algorithm based on M3DM, and output the probability of the existence of dark cracks in the plate; A sorting control module 4 is used to control the automated sorting equipment to sort out the plates with dark cracks according to the probability value output by the dark crack detection module; The user interaction module 5 is used to display the detection results, set the threshold value and provide a user operation interface.

[0042] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for detecting hidden cracks in a plate, characterized in that: The method comprises: Step S100: Acquire real-time plate 3D point cloud data and plate surface image data; Step S200: performing data preprocessing on the real-time plate 3D point cloud data and plate surface image; Step S300: inputting the real-time plate 3D point cloud data and plate surface image data into the plate dark crack detection model, and outputting the probability of the plate having dark cracks; Step S400: judging whether the plate has dark cracks according to the probability value of the plate having dark cracks and a preset threshold; Step S500: using automated sorting equipment to sort out the plates with hidden cracks.

2. The plate hidden crack detection method according to claim 1, characterized in that: The 3D point cloud data and surface image data are acquired through a depth camera.

3. The plate hidden crack detection method according to claim 1, characterized in that: The data preprocessing includes filtering, denoising and downsampling the plate 3D point cloud data; and image enhancement and grayscale adjustment of the plate surface image data.

4. The plate hidden crack detection method according to claim 1, characterized in that: The plate dark crack detection model is trained using a deep learning algorithm based on M3DM.

5. The plate hidden crack detection method as claimed in claim 4, characterized in that: The deep learning algorithm training process based on M3DM includes: The plate 3D point cloud data and plate surface image data are used as training samples, and the plate 3D point cloud data and plate surface image data include plates with dark cracks and plates without dark cracks; Build a deep learning network architecture based on M3DM; The training samples are input into the deep learning network for training, and the network parameters are adjusted through the back-propagation algorithm until the performance of the model on the validation set reaches the preset standard.

6. The plate hidden crack detection method according to claim 5, characterized in that: The M3DM-based deep learning network architecture specifically includes: A branch network for processing 3D point cloud data, which uses Point Transformer to extract point cloud features and is used to extract spatial features from 3D point cloud data; A branch network for processing surface image data, wherein the branch network includes a convolutional neural network structure and is used to extract texture and color features from the surface image data; The feature fusion layer is used to fuse the features extracted by the two branch networks and integrate the information of different modalities through the unsupervised feature fusion module; The classifier layer receives the fused features and adopts the decision layer fusion strategy to output the probability value of the presence of dark cracks in the plate.

7. The plate hidden crack detection method according to claim 6, characterized in that: The step S300 specifically includes: Input the preprocessed plate 3D point cloud data and surface image data into the trained plate dark crack detection model; The model extracts and fuses point cloud features and image features of the input data; Based on the extracted and fused features, the model outputs the probability value of the existence of dark cracks in the plate through the classifier.

8. The plate hidden crack detection method according to claim 1, characterized in that: The step S400 specifically includes: According to the probability value output by the plate dark crack detection model, a preset threshold is set; When the probability value of dark cracks in a plate exceeds the threshold, the plate is determined to have dark cracks.

9. A plate hidden crack detection system, characterized in that: The system comprises: A data acquisition module is used to obtain 3D point cloud data and surface image data of the plate through a depth camera; The data preprocessing module is used to filter, denoise and downsample the collected 3D point cloud data, and to enhance and adjust the grayscale of the surface image data; The dark crack detection module is used to input the preprocessed data into the plate dark crack detection model trained by the deep learning algorithm based on M3DM, and output the probability of the existence of dark cracks in the plate; A sorting control module, used to control the automated sorting equipment to sort out the plates with dark cracks according to the probability value output by the dark crack detection module; The user interaction module is used to display the detection results, set thresholds and provide a user operation interface.

Images