Unsupervised tire defect detection method fusing image and point cloud features

Abstract

The invention relates to the technical field of tire detection, in particular to an image and point cloud feature fused unsupervised tire defect detection method, and provides the following scheme: firstly, obtaining a tire model, a surface image and point cloud data; processing the point cloud data to extract point cloud tread features; registering the image and the point cloud data through rigid body transformation; image tread features are extracted from the registered image; and constructing a defect detection network, performing training by taking the tire model, the image and the point cloud tread features as input, and outputting a defect result. The method further comprises the steps of preprocessing, coordinate transformation, voxelization representation and clustering analysis of the point cloud data, and transformation and iterative calculation of the image point cloud by using a rotation matrix and a translation matrix. The defect detection network is composed of an input layer, a vector fusion layer and an output layer, and can effectively identify and mark defects on the surface of the tire.

Description

Technical Field

[0001] The present invention relates to the technical field of tire detection, and in particular to an unsupervised tire defect detection method integrating image and point cloud features. Background Art

[0002] Tires are key components of vehicles, and their performance and safety are directly related to driving safety and riding comfort. Traditional tire defect detection methods mainly rely on manual and mechanical inspections, which have problems such as low efficiency, high cost, and poor accuracy. Although some existing automated detection technologies based on images and machine learning have been improved, they still face problems such as sample scarcity, model overfitting, and inaccurate recognition of complex texture defects. The invention provides an unsupervised tire defect detection method that integrates image and point cloud features, aiming to solve various problems of tire defect detection in the prior art, such as high cost, low efficiency, and susceptibility to human factors in manual inspection, as well as poor performance of image-based algorithms in dealing with complex tire surface textures and defect diversity.

[0003] For example, a Chinese patent with publication number CN105869135A discloses a tire defect detection method and device. The tire defect detection method includes: obtaining an X-ray image of a tire; performing image preprocessing on the X-ray image to distinguish a first image from a background image in the displayed X-ray image to obtain a processed image; performing cluster analysis on pixels of the processed image to obtain an area where pixels representing the first image are located; judging whether the number of pixels in the area is greater than a preset threshold, and determining that the area is an image of a tire defect.

[0004] The above patents all have the problem raised by this background technology: since tire defects do not occur frequently and the types and forms of defects vary, the data sets available for training are limited in diversity and quantity. The scarcity of such samples may cause the trained model to overfit on the training set, that is, the model overfits the training data and loses the ability to generalize to unseen data, resulting in increased false detection rate and missed detection rate. To solve the above problems, this application designs an unsupervised tire defect detection method that integrates image and point cloud features. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide an unsupervised tire defect detection method that integrates image and point cloud features in view of the deficiencies of the prior art. First, the tire model, surface image and point cloud data are obtained; the point cloud data is processed to extract the point cloud tread features; the image and point cloud data are registered through rigid body transformation; the image tread features are extracted from the registered image; a defect detection network is constructed, and the tire model, image and point cloud tread features are used as input for training, and the defect results are output. The method also includes preprocessing, coordinate transformation, voxel representation and cluster analysis of point cloud data, and transformation and iterative calculation of image point clouds using rotation matrices and translation matrices. The defect detection network consists of an input layer, a vector fusion layer and an output layer, and can effectively identify and annotate defects on the tire surface.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] An unsupervised tire defect detection method integrating image and point cloud features comprises the following steps:

[0008] S1: Obtain tire model, surface image and point cloud data of the tire to be tested;

[0009] S2: Processing the point cloud data to extract the point cloud tread features;

[0010] S3: The surface image and point cloud data are converted into a unified coordinate system through rigid body transformation, and the surface image and point cloud data are registered according to the rotation matrix and translation matrix;

[0011] S4: processing the surface image after the point cloud tread feature registration to extract the image tread features;

[0012] S5: constructing a defect detection network, taking the tire model, image tread features and point cloud tread features as input parameters for the defect detection, training the input parameters through the defect detection network, and outputting tire defect results.

[0013] The specific steps of processing the point cloud data are as follows:

[0014] S2.1: pre-processing the laser scanning point cloud according to the laser scanning point cloud of the tire to be tested;

[0015] S2.2: Find the most suitable three-dimensional plane through plane fitting, perform coordinate transformation on the pre-processed three-dimensional point cloud data, and transfer the laser scanning point cloud from the radar coordinate system to the world coordinate system.

[0016] The S2 further includes:

[0017] S2.3: Divide the world coordinate system into grids at equal intervals, voxelize the laser scanning point cloud, project the voxel index of the laser scanning point cloud to the corresponding grid, connect the points in the voxel, strengthen the local feature information, and obtain the voxel features of the laser scanning point cloud;

[0018] S2.4: Cluster analysis is performed on the voxel features of the laser scanning point cloud according to a clustering algorithm, grids containing the voxel features of the laser scanning point cloud under the same central cluster are merged, and internal point cloud features are extracted.

[0019] The specific steps of S3 are as follows:

[0020] S3.1: Convert the surface image into an image point cloud, and calculate the corresponding nearest point of each point in the image point cloud in the laser scanning point cloud;

[0021] S3.2: Calculate the minimum average distance based on the corresponding point pairs of the image point cloud and the laser scanning point cloud, and determine the rotation matrix and the translation matrix through the minimum average distance.

[0022] The S3 further includes:

[0023] S3.3: transform the image point cloud according to the rotation matrix and the translation matrix, obtain the iterative image point cloud, and calculate the average distance between the iterative image point cloud and the laser scanning point cloud;

[0024] S3.4: If the average distance is less than the point cloud distance threshold, stop the iterative calculation; if the average distance is greater than or equal to the point cloud distance threshold, use the iterative image point cloud as the new image point cloud to recalculate the minimum average distance, rotation matrix and translation matrix;

[0025] S3.5: Convert the iterative image point cloud after the iteration into a surface image again.

[0026] The S4 includes:

[0027] S4.1: compressing the surface image to obtain a compressed reconstructed image, and extracting a tread feature image of the compressed reconstructed image by calculating the image box dimension;

[0028] S4.2: According to the cascaded residual regression tree, the positions of the key points of the tread are obtained, and the key points of the tread are marked and cropped in the tread feature image to obtain the gradient map of the key points of the tread;

[0029] S4.3: Extract the corner point features and grayscale centroid of the tread key point row gradient map, and calculate the image tread features according to the angle between the corner point features and the grayscale centroid. The calculation formula of the image tread features is:

[0030]

[0031] Among them, Vc represents the image tread feature, i represents the serial number of a single corner feature of the image, n represents the number of corner features of the image, Ψx represents the angle between the i-th corner feature and the grayscale centroid, ω i represents the weighted value of the i-th corner feature, F(·) represents the Fourier transform function, x c and c Represents the horizontal and vertical coordinates of the grayscale centroid, c i represents the i-th corner feature of the image, c p Represents the average value of image corner features.

[0032] The defect detection network includes an input layer, a vector fusion layer and an output layer;

[0033] The input layer includes an encoding module and a vector extraction module, wherein the encoding module is used to represent the model features of the tire by embedding vectors, and the vector extraction module includes an image vector extraction submodule and a point cloud vector extraction submodule, which are used to vectorize the image tread features and the point cloud tread features;

[0034] The vector fusion layer includes a vector fusion module, a convolution module and a perception module. The vector fusion module is used to fuse the model features, the vectorized tread features and the vectorized point cloud tread features in the vector fusion layer to obtain a fused feature vector. The convolution module is used to process the fused feature vector and calculate the convolution output. The perception module is used to compress and nonlinearly transform the convolution output to obtain point cloud feature points and image feature points.

[0035] The output layer includes an abnormality score calculation module and a defect judgment module. The abnormality score calculation module is used to calculate the signed distance between the point cloud feature points and the image feature points, and generate the abnormality score of each point by weighted fusion of the score distribution of the signed distance. The defect judgment module is used to judge whether there is a defect on the tire surface according to the defect threshold by comparing the abnormality score of each point with the defect threshold, and mark the defect position.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] By combining feature extraction of surface images and point cloud data, the present invention can more comprehensively capture subtle differences in the tire surface, does not require a large amount of labeled data for training, reduces the cost and time of data preparation, and avoids potential deviations caused by labeling errors, thereby improving the accuracy of defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0039] Figure 1 This is a flow chart of an unsupervised tire defect detection method integrating image and point cloud features according to Embodiment 1 of the present invention;

[0040] Figure 2 This is a structural diagram of a tire defect detection platform according to Embodiment 1 of the present invention;

[0041] Figure 3 Schematic diagram of the corner feature judgment area in Example 1 of the present invention. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0043] Embodiment 1:

[0044] See also Figure 1 The present invention provides an embodiment: an unsupervised tire defect detection method integrating image and point cloud features, wherein the specific steps of the method are as follows:

[0045] S1: Obtain tire model, surface image and point cloud data of the tire to be tested;

[0046] S2: Processing the point cloud data to extract the point cloud tread features;

[0047] S3: The surface image and point cloud data are converted into a unified coordinate system through rigid body transformation, and the surface image and point cloud data are registered according to the rotation matrix and translation matrix;

[0048] In this step, in order to align the surface image with the laser scanning point cloud data, the two are converted to a unified coordinate system through rigid body transformation to achieve accurate fusion of multi-source data;

[0049] S4: processing the surface image after the point cloud tread feature registration to extract the image tread features;

[0050] S5: constructing a defect detection network, taking the tire model, image tread features and point cloud tread features as input parameters for defect detection, training the input parameters through the defect detection network, and outputting tire defect results;

[0051] See also Figure 2, structural diagram of a tire defect detection platform in an embodiment of the present invention, in this step, the tire defect detection platform includes a conveyor belt, a rotating turntable, a mechanical arm, a data acquisition source and an optical character recognition module, the conveyor belt is used to transport tires, and the tires are transported from the production line to the detection platform through the conveyor belt, and the tires are transported to the next process after the detection is completed, so as to ensure the smooth movement of the tires in the entire detection process, the rotating turntable is used to fix and rotate the tires, so as to facilitate comprehensive detection of different angles of the tires, the mechanical arm is used to grab and move the tires, and ensure that the tires can be correctly placed on the detection platform, the data acquisition source is used to collect the surface image and three-dimensional contour data of the tire, the optical character recognition module is used to automatically identify the characters and digital information on the side of the tire through image processing technology, and record it in the system database for traceability and management, the data acquisition source includes an industrial camera, a laser scanner and a light source, the industrial camera is used to capture high-resolution tire surface images and detect surface defects, the laser scanner is used to obtain the three-dimensional contour data of the tire, and provide depth information to detect internal structural defects, and the light source provides uniform illumination to ensure the clarity of the image and contour data;

[0052] The specific steps of processing the point cloud data are as follows:

[0053] S2.1: pre-processing the laser scanning point cloud according to the laser scanning point cloud of the tire to be tested;

[0054] S2.2: During the point cloud data acquisition process, the posture and position of the tire are different when scanning at different angles. According to the least squares method, the direction and position of the plane are adjusted to minimize the sum of the squares of the distances from all points to the plane. The most suitable three-dimensional plane is found through plane fitting. The pre-processed three-dimensional point cloud data is converted to convert the laser scanning point cloud from the radar coordinate system to the world coordinate system. When scanning at different angles, the tire surface point cloud data is uniformly represented in the same coordinate system, which provides spatial consistency for subsequent data processing and feature extraction.

[0055] S2.3: Divide the world coordinate system into grids at equal intervals, voxelize the laser scanning point cloud, project the voxel index of the laser scanning point cloud to the corresponding grid, connect the points in the voxel, strengthen the local feature information, and obtain the voxel features of the laser scanning point cloud. The voxelization process divides the space into uniform small cube units, and gathers the points in the same voxel into a voxel point. This representation method not only reduces the amount of data, but also strengthens the local feature information, so that the detailed features of the tire surface can be better preserved and represented;

[0056] S2.4: Perform preliminary clustering of the voxel features of the laser scanning point cloud according to the DBSCAN clustering algorithm to obtain core points, and determine the number of cluster center clusters by the number of core points. DBSCAN classifies points in high-density areas into the same category by analyzing the density of points, and can effectively identify and ignore noise points. Determine the number of preliminary cluster center clusters by the number of core points. The clustering results of this step help identify the main areas of the tire surface structure and lay the foundation for subsequent accurate clustering;

[0057] S2.5: Use the core points after DBSCAN clustering as the initial center cluster of the K-means clustering algorithm, calculate the Euclidean distance from the voxel features of the laser scanning point cloud to each of the initial center clusters, use the core points obtained by DBSCAN clustering as the initial center cluster of the K-means algorithm, and use the K-means algorithm to further optimize the clustering results. K-means performs clustering by calculating the Euclidean distance from the voxel features to the initial center cluster. The selection of the initial center cluster has a great influence on the final convergence effect of the K-means algorithm. By using the DBSCAN clustering results as the initial point, the convergence speed and clustering quality of the K-means algorithm can be improved;

[0058] S2.6: Perform clustering according to the Euclidean distance, and calculate the average clustering error of the initial center cluster. Update the center cluster according to the average clustering error, and determine whether the updated center cluster meets the convergence condition. If it does not meet the convergence condition, continue to update the center cluster. If it meets the convergence condition, output the clustering result. In each iteration, the K-means algorithm reallocates each point to the nearest center cluster and updates the position of the center cluster. This process continues to iterate until the average clustering error meets the convergence condition and the clustering result is stable. Through this process, the algorithm can more accurately extract the features of the tire surface and finally output the clustering result.

[0059] S2.7: Extract the clustered laser scanning point cloud voxel features, merge the grids where the laser scanning point cloud voxel features are located under the same central cluster, ensure that the feature points belonging to the same category are clustered together, and make the defect identification of the tire surface more accurate. The point cloud features finally extracted can reflect the morphological characteristics and potential defects of the tire surface, extract the internal point cloud features, and the calculation formula of the internal point cloud features is:

[0060]

[0061] Among them, T a represents the internal point cloud features corresponding to the voxel features of the laser scanning point cloud in the same central cluster after clustering, i represents a single voxel feature of the laser scanning point cloud in the central cluster, M represents the total number of voxel features of the laser scanning point cloud in the central cluster, and η irepresents the eigenvalue of the voxel feature of the i-th laser scanning point cloud, d represents the clustering threshold corresponding to the central cluster, r2 represents the farthest distance scanned by the laser scanner, r1 represents the closest distance scanned by the laser scanner, and p i_x represents the voxel horizontal coordinate of the i-th laser scanning point cloud voxel feature, p i_y The voxel ordinate represents the feature of the i-th laser scanning point cloud voxel, p i_z The vertical coordinate of the voxel representing the feature of the i-th laser scanning point cloud voxel;

[0062] Laser scanning point cloud data usually contains a large amount of raw point data, which may be affected by noise, external environment and equipment errors, so as to improve the overall quality of point cloud data. This process can effectively reduce the errors caused by external factors, so that the point cloud data can more truly reflect the actual situation of the tire surface. To ensure the accuracy of subsequent analysis, the preprocessing step first removes noise points and abnormal points. The specific steps for preprocessing laser scanning point cloud are as follows:

[0063] S2.1.1: Calculate the y-axis coordinates of all laser scanning point clouds. If the y-axis coordinate of a point is less than 0, delete the point from the laser scanning point cloud.

[0064] S2.1.2: traverse and query the laser scanning point cloud, and delete the laser scanning point cloud data with the same radar rotation angle and echo distance;

[0065] S2.1.3: Set the fitting plane equation, calculate the coefficient of the fitting plane equation according to the laser scanning point cloud, and calculate the distance from all laser scanning point clouds to the fitting plane, compare the distance from the laser scanning point cloud to the fitting plane with the plane distance threshold, if it is less than or equal to the plane distance threshold, save the laser scanning point cloud, if it is greater than the plane distance threshold, delete the laser scanning point cloud;

[0066] S2.1.4: Gather the laser scanning point cloud in the normal direction and adjust the position and coordinates of the sampling points to denoise the laser scanning point cloud. The calculation formula for denoising the laser scanning point cloud is:

[0067]

[0068] Among them, NP 3D represents the denoised laser scanning point cloud, P 3D represents the laser scanning point cloud before denoising, α and β represent the adjustment weight coefficients, H c represents the smoothing filter influence factor, H s represents the feature preservation influence factor, N represents the total number of laser scanning point clouds, n represents a single laser scanning point cloud, d vrepresents the average distance of the laser scanning point cloud, ξ represents the dynamic coefficient of the laser scanning point cloud distance, represents the variance of the laser scanning point cloud distance, λ represents the bilateral filtering factor, and g represents the normal vector of the laser scanning point cloud;

[0069] The specific steps of S3 are as follows:

[0070] S3.1: Convert the two-dimensional surface image data into a three-dimensional image point cloud. Using the internal and external parameters obtained through camera calibration and the depth information in the image, calculate the coordinates of each pixel in the three-dimensional space and generate an image point cloud. Each point in the image point cloud represents the position of an image pixel in the three-dimensional space. After the image point cloud is generated, calculate the corresponding nearest point in the laser scanning point cloud for each point in the image point cloud. For each point in the image point cloud, find the point in the laser scanning point cloud that is closest to it, so as to establish a corresponding relationship between the two;

[0071] S3.2: Calculate the minimum average distance based on the corresponding point pairs of the image point cloud and the laser scanning point cloud, and determine the rotation matrix and translation matrix through the minimum average distance;

[0072] S3.3: According to the calculated rotation matrix RR and translation matrix TT, perform a rigid body transformation on the image point cloud to obtain the transformed iterative image point cloud. Calculate the average distance DD between the iterative image point cloud and the laser scanning point cloud. If the average distance is less than the preset point cloud distance threshold, stop the iteration, indicating that the registration has reached the required accuracy. Otherwise, use the iterative image point cloud as the new image point cloud, re-execute S3.2, and continue iterative optimization. This iterative process ensures the registration accuracy of the image point cloud and the laser scanning point cloud. The calculation formula for the average distance is:

[0073]

[0074] Among them, D represents the average distance between the iterative image point cloud and the laser scanning point cloud, j represents the serial number of a single point cloud data, M represents the total number of point cloud data, represents the jth laser scanning point cloud, represents the jth iterative image point cloud, represents the Euclidean distance function, R represents the rotation matrix, and T represents the translation matrix;

[0075] S3.4: After the registration is completed, the final image point cloud is converted back to the surface image through back projection. Using the camera's internal and external parameters and the transformed point cloud coordinates, the 3D image point cloud is remapped to the 2D image plane to generate a surface image that is fully aligned with the laser scanning point cloud. This conversion process enables the image and point cloud to be represented in the same coordinate system, providing consistent input data for subsequent feature extraction and defect detection;

[0076] The specific steps of processing the surface image after the point cloud tread feature registration are as follows:

[0077] S4.1: In order to reduce the computational complexity and retain the key features of the image, the surface image is first compressed. Through compressed sampling, the redundant information in the image is removed and the main features of the tread texture are retained. Then, the compressed image is reconstructed to generate a compressed reconstructed image. In order to quantify the complexity and detail information of the image, the box dimension of the compressed reconstructed image is calculated. The box dimension is an indicator to measure the fractal characteristics of the image, through which the characteristic image related to the tread shape and texture can be extracted;

[0078] S4.2: After obtaining the tread feature image, use the cascaded residual regression tree to obtain the key point positions of the tread. By predicting the key points layer by layer and continuously adjusting the key point positions, the key points can be accurately located in the complex tread image. After determining the key points, these key points are marked in the tread feature image, and the local area containing the key points is cropped to generate a row gradient map of the tread key points. The row gradient map shows the gradient information of the row where the key points are located;

[0079] S4.3: Further feature extraction is performed on the gradient map of the key points of the tread. First, the corner features in the image are detected. Corner points usually appear in areas with drastic changes in the image and represent important structural features in the image. After using the corner detection algorithm to find the corner points in the image and calculating their position and intensity, the grayscale centroid is calculated. The grayscale centroid represents the distribution center of the intensity in the image and can be determined by calculating the weighted average position of the image grayscale value. The directional angle between the corner point feature and the grayscale centroid is used as part of the tread feature, and the feature vector of the tread is further calculated. This angle can describe the directionality of the structural features in the image and quantitatively analyze the texture information of the tire surface. The calculation formula for the image tread feature is:

[0080]

[0081] Among them, V c represents the image tread feature, i represents the serial number of a single corner feature of the image, n represents the number of corner features of the image, Ψ i Represents the angle between the i-th corner feature and the grayscale centroid, ω i represents the weighted value of the i-th corner feature, F(·) represents the Fourier transform function, x c and c Represents the horizontal and vertical coordinates of the grayscale centroid, c i represents the i-th corner feature of the image, c p Represents the average value of image corner features;

[0082] The specific steps of S4.1 are as follows:

[0083] S4.1.1: De-noising the surface image, wherein the noise includes scene light noise, operation noise of each unit itself, circuit switching noise, and image sensor photoelectric tube noise;

[0084] S4.1.2: Perform sparse representation on the denoised surface image and perform mask sampling through Fourier transform to obtain a sparse surface image;

[0085] S4.1.3: Cut the sparse surface image, cut the original image of size h*w into blocks of size p*q, obtain the block image matrix, construct the observation matrix sequence, match different sampling rates according to the composition complexity of different regions of the block image, and perform sparse sampling compression in the Fourier domain in a distributed multi-point collaborative manner. The calculation formula for sparse sampling compression is:

[0086]

[0087] Where G represents the compressed reconstructed image, a represents a single block image matrix, A represents the total number of block image matrices, and ρ a represents the sampling rate of the a-th block image matrix, θ represents the observation matrix sequence, represents the vector dot multiplication operation, K represents the division rate, F[·] represents the Fourier transform function, I represents the external image of the ultrasonic motor, and γ represents the noise;

[0088] S4.1.4: Divide the preprocessed compressed reconstructed image into segmented images with rows = columns = γ according to matrix binarization, obtain the centroid position of the segmented image, calculate the segmented image box dimension according to the centroid position, and compare the segmented image box dimension with the tread threshold;

[0089] S4.1.5: If it is greater than or equal to the tread threshold, keep the segmented image; if it is less than the tread threshold, delete the segmented image, traverse all segmented images, fuse the remaining segmented images, and obtain the tread feature image, where γ represents the length of the row and column;

[0090] The specific steps of S4.2 are as follows:

[0091] S4.2.1: Construct a cascaded residual regression tree, take the average shape of all facial row gradient images as the initial shape for regression, input it into the leaf node of the first cascaded residual regression tree, calculate the difference between the true shape and the average shape of each tread row gradient image, take the variance of the difference of the images in the leaf node, and take the result as the residual of the leaf node;

[0092] S4.2.2: Save the residuals of all leaf nodes, complete the construction of the first tree, update the current shape of each tread row gradient map to the initial residual shape, input it into the leaf node of the second joint residual regression tree, repeatedly calculate the residuals of the leaf nodes, extract the residuals of the leaf nodes of the second joint residual regression tree according to the least squares method, obtain the position of the key points of the face, mark and crop them in the facial row gradient map, and obtain the tread key point row gradient map;

[0093] See also Figure 3 , a schematic diagram of a corner feature judgment area according to an embodiment of the present invention, wherein the steps of extracting the corner feature are as follows:

[0094] S6.3.1: Randomly select any pixel in the image, mark it as point P, and calculate the gray value I of P P ;

[0095] S6.3.2: Draw a circle with P as the center and a radius of 3 pixels, and select 16 pixels on the edge of the circle;

[0096] S6.3.3: Set the grayscale threshold t. If the grayscale values ​​of 12 consecutive pixels out of 16 pixels are less than I P -t or greater than I P +t, then point P is determined to be a corner feature of the image;

[0097] S6.3.4: Select the next pixel and repeat steps S6.3.1 to S6.3.3 until all pixels are processed and all corner features are counted;

[0098] The defect detection network includes an input layer, a vector fusion layer and an output layer;

[0099] The input layer includes an encoding module and a vector extraction module. The encoding module is used to represent the model features of the tire by embedding vectors. The model of each tire is input as category information, and the features of different models of tires are represented by embedding vectors. This part of the input helps the network to perform differentiated processing according to the unique characteristics of different models of tires. The vector extraction module includes an image vector extraction submodule and a point cloud vector extraction submodule, which are used to vectorize the image tread features and the point cloud tread features. The image vector extraction submodule uses a pre-trained Wide ResNet101 model to process the input image data. Wide ResNet 101 is a deep convolutional neural network that extracts local and global features from images through multi-layer convolution operations. These features include texture, shape, edge and other potentially important information in the image. The model generates convolution feature maps of multiple levels, each of which captures image information at different scales. The high-dimensional feature vector finally output is used to describe the details of the tire surface to ensure that the key features are effectively retained. The point cloud vector extraction submodule is processed by the PointNet++ model, which can process irregular three-dimensional point cloud data and extract point cloud feature vectors with structural and local geometric features. The PointNet++ model uses a hierarchical multi-scale receptive field to extract local information from the point cloud and aggregates this information layer by layer to form global features. The generated point cloud feature vector is high-dimensional data that contains the spatial structure information of the point cloud, which is crucial for detecting defects on three-dimensional surfaces.

[0100] The vector fusion layer includes a vector fusion module, a convolution module and a perception module. The vector fusion module is used to fuse the model features, vectorized tread features and vectorized point cloud tread features in the vector fusion layer. By introducing the embedded vector of the tire model, it is input into the fusion layer together with the joint features of the image and the point cloud to provide additional context information. The embedded vector of the tire model provides important prior information for the network, which helps the network to optimize the effect of feature fusion according to the unique structure and characteristics of different tire models and obtain the fused feature vector. The convolution module is used to process the fused feature vector and calculate the convolution output. The perception module is used to compress and nonlinearly transform the convolution output to obtain point cloud feature points and image feature points.

[0101] The output layer includes an abnormal score calculation module and a defect judgment module. The abnormal score calculation module is used to calculate the signed distance between the point cloud feature points and the image feature points, and generate the abnormal score of each point by weighted fusion of the score distribution of the signed distance. The defect judgment module is used to determine whether there is a defect on the tire surface by comparing the abnormal score of each point with the defect threshold according to the defect threshold, and mark the defect position. The legacy score calculation module specifically includes:

[0102] Signed distance calculation: In this module, the output of the network is the signed distance of each feature point. The NIF model outputs the signed distance of the point to the expected surface by taking the coordinates of the input point and the corresponding feature vector. A positive value indicates that the point is outside the surface, and a negative value indicates that the point is inside the surface.

[0103] Anomaly score distribution map generation: Take the absolute value of the signed distance of all point cloud blocks to generate the anomaly score distribution map of the point cloud. For the image part, the anomaly score is calculated by the reconstruction error of the image feature vector, and the score distribution of the two is weighted and fused to generate a comprehensive anomaly score distribution map.

[0104] Score fusion: Anomaly score fusion takes into account the different modal features of images and point clouds. The final score distribution map combines the information of the two data sources, providing a more comprehensive reference for defect detection.

[0105] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. An unsupervised tire defect detection method integrating image and point cloud features, characterized in that: The following steps are involved: S1: Obtain tire model, surface image and point cloud data of the tire to be tested; S2: Processing the point cloud data to extract the point cloud tread features; S3: The surface image and point cloud data are converted into a unified coordinate system through rigid body transformation, and the surface image and point cloud data are registered according to the rotation matrix and translation matrix; S4: processing the surface image after the point cloud tread feature registration to extract the image tread features; S5: constructing a defect detection network, taking the tire model, image tread features and point cloud tread features as input parameters for the defect detection, training the input parameters through the defect detection network, and outputting tire defect results.

2. The unsupervised tire defect detection method by fusing image and point cloud features according to claim 1, characterized in that: The specific steps of processing the point cloud data are as follows: S2.1: pre-processing the laser scanning point cloud according to the laser scanning point cloud of the tire to be tested; S2.2: Find the most suitable three-dimensional plane through plane fitting, perform coordinate transformation on the pre-processed three-dimensional point cloud data, and transfer the laser scanning point cloud from the radar coordinate system to the world coordinate system.

3. The unsupervised tire defect detection method by integrating image and point cloud features according to claim 2, characterized in that: The S2 further includes: S2.3: Divide the world coordinate system into grids at equal intervals, voxelize the laser scanning point cloud, project the voxel index of the laser scanning point cloud to the corresponding grid, connect the points in the voxel, strengthen the local feature information, and obtain the voxel features of the laser scanning point cloud; S2.4: Cluster analysis is performed on the voxel features of the laser scanning point cloud according to a clustering algorithm, grids containing the voxel features of the laser scanning point cloud under the same central cluster are merged, and internal point cloud features are extracted.

4. The unsupervised tire defect detection method by fusing image and point cloud features according to claim 3, characterized in that: The specific steps of S3 are as follows: S3.1: Convert the surface image into an image point cloud, and calculate the corresponding nearest point of each point in the image point cloud in the laser scanning point cloud; S3.2: Calculate the minimum average distance based on the corresponding point pairs of the image point cloud and the laser scanning point cloud, and determine the rotation matrix and the translation matrix through the minimum average distance.

5. The unsupervised tire defect detection method by fusing image and point cloud features according to claim 4, characterized in that: The S3 further includes: S3.3: transform the image point cloud according to the rotation matrix and the translation matrix, obtain the iterative image point cloud, and calculate the average distance between the iterative image point cloud and the laser scanning point cloud; S3.4: If the average distance is less than the point cloud distance threshold, stop the iterative calculation; if the average distance is greater than or equal to the point cloud distance threshold, use the iterative image point cloud as the new image point cloud to recalculate the minimum average distance, rotation matrix and translation matrix; S3.5: Convert the iterative image point cloud after the iteration into a surface image again.

6. The unsupervised tire defect detection method by fusing image and point cloud features according to claim 5, characterized in that: The S4 includes: S4.1: compressing the surface image to obtain a compressed reconstructed image, and extracting a tread feature image of the compressed reconstructed image by calculating the image box dimension; S4.2: According to the cascaded residual regression tree, the positions of the key points of the tread are obtained, and the key points of the tread are marked and cropped in the tread feature image to obtain the gradient map of the key points of the tread; S4.3: Extract the corner point features and grayscale centroid of the tread key point row gradient map, and calculate the image tread features according to the angle between the corner point features and the grayscale centroid. The calculation formula of the image tread features is: Among them, V c represents the image tread feature, i represents the serial number of a single corner feature of the image, n represents the number of corner features of the image, Ψ represents the angle between the i-th corner feature and the grayscale centroid, ω i represents the weighted value of the i-th corner feature, F(·) represents the Fourier transform function, x c and c Represents the horizontal and vertical coordinates of the grayscale centroid, c i represents the i-th corner feature of the image, c p Represents the average value of image corner features.

7. The unsupervised tire defect detection method by integrating image and point cloud features according to claim 1, characterized in that: The defect detection network includes an input layer, a vector fusion layer and an output layer; The input layer includes an encoding module and a vector extraction module, wherein the encoding module is used to represent the model features of the tire by embedding vectors, and the vector extraction module includes an image vector extraction submodule and a point cloud vector extraction submodule, which are used to vectorize the image tread features and the point cloud tread features; The vector fusion layer includes a vector fusion module, a convolution module and a perception module. The vector fusion module is used to fuse the model features, the vectorized tread features and the vectorized point cloud tread features in the vector fusion layer to obtain a fused feature vector. The convolution module is used to process the fused feature vector and calculate the convolution output. The perception module is used to compress and nonlinearly transform the convolution output to obtain point cloud feature points and image feature points. The output layer includes an abnormality score calculation module and a defect judgment module. The abnormality score calculation module is used to calculate the signed distance between the point cloud feature points and the image feature points, and generate the abnormality score of each point by weighted fusion of the score distribution of the signed distance. The defect judgment module is used to judge whether there is a defect on the tire surface according to the defect threshold by comparing the abnormality score of each point with the defect threshold, and mark the defect position.

Images