A Tire Bubble Defect Detection Method and System Based on 3D Point Cloud

Abstract

A method and system for detecting tire bubble defects based on 3D point clouds. The method includes the following steps: sampling the original point cloud, sampling at a certain ratio using different sampling methods, extracting feature information to obtain the points after feature extraction; arranging the points after feature extraction to obtain a subset of feature vectors of N points, and voting and grouping the N points; detecting whether there are defective targets among the points after feature extraction, and if there are defective targets, making proposals and refining. This application performs geometric sampling on the point cloud to retain the edge points of the defects; at the same time, it directly operates on the original point cloud to preserve the local feature information of the point cloud, indirectly increasing its detection accuracy. Compared with other networks, it has a good effect on detecting target defects.

Description

Technical Field

[0001] The present application relates to the field of tire detection, and specifically to a tire bubble defect detection system and method based on 3D point cloud. Background Art

[0002] In recent years, 3D point cloud technology has become an increasingly important element in the field of computer vision and has been applied in many areas, especially in autonomous driving scenarios and target detection. Compared with 2D technology, 3D point cloud has a natural advantage in expressing the overall appearance of the target object, is insensitive to changes in lighting, has good robustness, and its accuracy is directly determined by the point cloud sensor.

[0003] Automobile tires have a complex structure and are easily affected by materials and production processes during the formation process. Bubble defects are the detection object of this application. Their existence will seriously endanger the safety of automobile driving and must be eliminated in time. In view of the advantages of 3D point clouds, Kinect2.0 is used as a sensor to obtain dense point clouds of targets. The difference between point clouds and images is that point clouds are irregular, which makes it impossible for traditional convolutional neural networks (CNNs) to directly process point clouds. Whether it is voxel, volume, grid or bird's-eye view (BEV) point cloud processing methods, there is a phenomenon of synthesizing local point areas into one point. With the help of RPN, proposals are generated for the target, which is real-time, but this method will lose local information, which is not obvious in some places with subtle spatial changes, and is not conducive to the detection of smaller targets. Summary of the invention

[0004] The present application discloses a tire bubble defect detection method and system based on 3D point cloud: sampling the original point cloud, using different sampling methods to sample at a certain ratio, extracting feature information, and obtaining points after feature extraction; arranging the points after feature extraction to obtain a feature vector subset of N points, and voting and grouping the N points; detecting whether there are defective targets in the points after feature extraction, and if there are defective targets, proposing and refining them.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] A tire bubble defect detection method based on 3D point cloud comprises the following steps:

[0007] S1. Sampling the original point cloud, using different sampling methods to sample at a preset ratio, extracting feature information, and obtaining points after feature extraction;

[0008] S2. Arrange the points after feature extraction to obtain a feature vector subset of N points, and vote and group the points in the feature vector subset;

[0009] S3. Detect whether there are defective targets among the points after feature extraction. If there are defective targets, propose and refine them.

[0010] Preferably, the sampling method includes:

[0011] Sample the original point cloud with the SA layer in Pointnet++ as the backbone and obtain feature information. One SA layer includes sampling, grouping, and local feature extraction (PointNet) of feature points. In the sampling layer of SA, sample a preset proportion of points according to FPS sampling and C-FPS sampling respectively, use the focal loss function to balance the number gap, and complete the feature extraction of the input point cloud with the help of a multi-layer perceptron.

[0012] Preferably, the FPS sampling is the farthest point sampling method, and the C-FPS sampling is the sampling method based on extracting curvature features in the Euclidean space.

[0013] 4. According to the method for detecting tire bubble defects based on 3D point cloud as claimed in claim 2, wherein the voting method includes: local feature voting and global feature voting.

[0014] Preferably, the local feature voting includes:

[0015] Retain some points and their features after the FPS sampling, take the output after passing through the max pooling layer as a global feature, and add it as the first part of the new feature vector set to the feature vector subset;

[0016] For the density feature of points, use its kernel density estimation density feature, and extract the density feature and other features into the feature vector subset with the help of a multi-layer perceptron and max pooling operation.

[0017] Preferably, the global feature voting includes:

[0018] Extract the points after the C-FPS sampling as local features. After processing the original point cloud with fusion sampling in multiple SA layers, obtain a subset processed by the C-FPS sampling method and the corresponding features. Take the subset as the initial center point. Under the supervision of the loss function, the initial center point moves relatively to generate the final center point, and the output is the local feature.

[0019] This application also provides a tire bubble defect detection system based on 3D point cloud, including: a fusion sampling module, a point voting module, and a proposal refinement module;

[0020] The fusion sampling module is connected to the point voting module. The fusion sampling module is used to sample the original point cloud and extract feature information to obtain the points after feature extraction;

[0021] The point voting module is also connected to the proposal refinement module. The point voting module is used to arrange the points after feature extraction, output a subset of feature vectors of N points, and vote and group the points in the subset of feature vectors through a multi-layer perceptron (MLP);

[0022] The proposal refinement module is used to detect whether there are defective targets in the point cloud. If there are such defective targets, proposals and refinements are performed.

[0023] Preferably, the fusion sampling module includes: an FPS sampling device and a C-FPS sampling device.

[0024] Preferably, the point voting module includes: a local feature voting device and a global feature voting device;

[0025] The local voting feature device is used to retain some points and their features after FPS sampling, take the output after passing through the max pooling layer as a global feature, and add it as the first part of the new feature vector set to the subset of feature vectors; for the density feature of the points, use its kernel density estimation density feature, and extract the density feature and other features into the subset of feature vectors by means of a multi-layer perceptron and max pooling operations;

[0026] The global feature voting device is used to extract the points after C-FPS sampling as local features. After processing the original point cloud by fusion sampling in multiple SA layers, a subset after C-FPS sampling processing and corresponding features are obtained. Taking the subset as the initial center point, the initial center point moves relatively under the supervision of the loss function to generate a final center point, and the output is the local feature.

[0027] The beneficial effects of this application are:

[0028] (1) Geometric sampling of the point cloud to retain the edge points of the defect;

[0029] (2) Operating directly on the original point cloud, preserving the local feature information of the point cloud, indirectly increasing its detection accuracy, and having a good effect on detecting target defects compared with other networks. Description of the Drawings

[0030] In order to more clearly illustrate the technical solutions of this application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0031] Figure 1Schematic flow chart of a tire bubble defect detection method based on 3D point cloud in this application;

[0032] Figure 2 Schematic diagram of the arrangement of points at normal and bubble defect - existing points in the embodiment of this application;

[0033] Figure 3 Schematic diagram of the improved bounding box encoding method in the embodiment of this application;

[0034] Figure 4 Schematic structure diagram of a tire bubble defect detection system based on 3D point cloud in this application;

[0035] Figure 5 Schematic diagram of the change trend of the loss function in different training stages in the embodiment of this application. Detailed implementation manners

[0036] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

[0037] To make the above - mentioned objects, features, and advantages of this application more obvious and understandable, the following further detailed description of this application will be made in conjunction with the accompanying drawings and specific implementation manners.

[0038] Embodiment 1

[0039] In the first embodiment, as Figure 1 shown, a tire bubble defect detection method based on 3D point cloud includes:

[0040] S1. Sample the original point cloud, use different sampling methods to sample according to a certain proportion, extract feature information, and obtain the points after feature extraction.

[0041] A common point cloud sampling method is FPS (Farthest Point Sampling method), which is a uniform sampling method. Although it can retain some local information, the sampling effect is average for places with curvature. And because the target for detecting defects is small, FPS sampling may pick up fewer foreground points, resulting in poor detection effects. For this reason, a C - FPS (sampling method based on extracting curvature features in Euclidean space) is proposed, which effectively retains the foreground points when the detection target is small.

[0042] In a Euclidean space, Ns represents the neighborhood of point N i , i ∈ R with S points, and the surface normal vector at point N i is vi , point x j is another point within the neighborhood, and its surface normal vector is v j , then there is As Figure 2 shown, when there are no defects on the point cloud surface, in the ideal state, the normal vectors of each point are parallel, and the included angle is 0°; when there are defects on the point cloud surface, in the edge neighborhood of the defect, some values of θ (i,j) will change. Set the threshold θ (i,j) * to 10°. Those exceeding the threshold are regarded as foreground points, and those less than the threshold are considered background points. Therefore, the points in each neighborhood with such characteristics are retained, greatly increasing the foreground points in the instance and removing a large number of useless background points, providing favorable conditions for subsequent detection.

[0043] However, if all use C-FPS to sample the point cloud, a large number of foreground points will be concentrated in the target area, generating duplicate proposal boxes in a small range, resulting in a decrease in accuracy and efficiency. In order to obtain sufficient points during the sampling process, this application uses a fusion sampling strategy. In the sampling layer of SA, a certain proportion of points are collected by both the FPS and the proposed C-FPS sampling methods, as shown in formula (1):

[0044] N total = λN C-FPS + βN FPS (1)

[0045] Among them, λ is the preset proportion of C-FPS sampling, β is the preset proportion of FPS sampling, and λ + β = 1.

[0046] This method retains more foreground points for smaller target detection, while having enough background points for classification, and does not use the FP layer to retain foreground points. The sampled points are provided as input to the subsequent grouping step. Generally speaking, the problem that the number gap between foreground and background points is large due to the small detection target still exists, and we use the focal loss function to balance the number gap.

[0047] Complete the feature extraction of the input point cloud with the help of a multi-layer perceptron. After completing the C-P fusion sampling, add density and other features of the point to points. Estimate its density feature through kernel density estimation, represented by , and other features of the point are represented by .

[0048] Considering that the number of point clouds in each group of the dataset is about 24,000, while the number of points in the corresponding annotation boxes is about 200. Initially, 1 / 20 of the original number of points is taken as the network input. To make up for the shortage of foreground points as much as possible, the number of SA layers is set to 3 appropriately, as shown in Table 1, the SA layer parameter table. As shown in Table 2, the performance comparison of SA layers with different numbers of layers, which is the comparison between using four SA layers and three SA layers. When n = 4, the parameters increase accordingly, so the running time and the training time also increase accordingly, while the mAP also improves compared to when n = 3.

[0049] Table 1

[0050]

[0051] Table 2

[0052]

[0053] The input point cloud can be represented as N×3. Taking λ = β = 1 / 2, that is, each of FPS and C-FPS uses half of the number of points. After passing through the first feature extraction layer in the C-P fusion sampling layer, the outputs of the two methods are respectively represented as and Taking half of the number of points again, after passing through the second feature extraction layer, it can be represented as and Taking half of the points again until passing through the nth feature extraction layer. At this time, the output is N n points, which consists of two parts: one part is generated by FPS, represented by and the other part is generated by C-FPS, represented by Since the sampling methods used by the two are different, the features they possess will also be different. Use C n and C n' to distinguish them.

[0054] S2. Arrange the points after feature extraction to obtain a subset of feature vectors of N points, and vote and group the points in the subset of feature vectors; to ensure the detection accuracy and fully capture the local and global features of the points, arrange and combine the points after feature extraction at different scales to form a new set of feature vectors. Its output is a subset of feature vectors of N points, and each point in the subset will correspond to a vote, and vote on each vote through the MLP network. The voting layer of the points can be divided into two branches: the local feature voting (LFE branch) and the global feature voting (GFE branch).

[0055] The LFE branch is used to extract global features at different scales: retain part of the points N FPSAnd its features, the output after the max-pooling layer is used as a global feature, and this global feature is added as the first part of the new feature vector set to the feature vector set; noticing that there are density changes in some local regions of the detected defective objects, the density feature of points and other features are added as the subsequent parts of the feature vector set. For the density feature of points, its kernel density is used to estimate the density feature, and the density feature and other features are extracted into the vector set with the help of a multi-layer perceptron and max-pooling operations.

[0056] The role of the GFE branch is to extract the points after C-FPS processing as local features. After processing the original point cloud using the fusion sampling strategy in multiple SA layers, a subset {x i |x∈(1,2,…n)} and its corresponding features are obtained through the C-FPS sampling method. This subset is used as the initial center point. Under the supervision of the loss function, the initial center point moves relatively to generate the final center point, and the output is the local feature, and the features in the LFE branch are added to the local feature.

[0057] S3. Detect whether there are defective objects in the points after feature extraction. If there are defective objects, proposals and refinement are performed; during the point cloud sampling process, due to the non-ideal positions between point clouds, which are caused by the accuracy error of the depth camera and the error caused by shooting factors, some points in the point cloud are not accurately located on the target surface position they should represent, and even deviate far away. Moreover, the existence of errors will also cause some background points to be detected as foreground points. Such error points are called pseudo-foreground points, and thus proposals are generated through voting, which undoubtedly has a great impact on the detection results. And detecting such pseudo-defects in a point cloud without target defects will seriously reduce the efficiency.

[0058] The detection type of this application is defect detection, with a single target, belonging to binary classification object detection. Among the required classification categories, there are only bubble defects and the background. To avoid generating and refining proposal boxes for each frame of 3D point cloud collected, in order to save detection time, a simple method is proposed to determine whether there are defective objects in this frame of point cloud, and the classification of the target object is placed before the 3D box proposal. If the target object is classified, then proposals and refinement are performed, otherwise the process does not take effect.

[0059] First, define the clustered points separated as CP = {C i , F i}, i = 1, Λ, n and the centroid b of this clustered point. Among them, C i ∈R 3 represents the coordinates of the clustered point, and F i ∈R represents the feature of the clustered point. If there are defective objects in the point cloud, the number of generated clustered points will be much larger than the small number of clustered points generated by the point cloud without defective objects. Therefore, there is (Ci ) exist -(C i ) no >T, and use C i -b to convert the cluster points into the centroid local normalization coordinate system. Since the cluster points of the defective objects are relatively concentrated, there exist Taking the centroid point of the cluster points as the center point, the average value of the neighborhood points near the center point and the center point is less than the threshold r. However, the cluster points without defective objects will be scattered irregularly in the entire point cloud space. Therefore, it shows the characteristics of a small number and a large r i .

[0060] Each point in the cluster points generates a 3D bounding box. Since directly encoding the 8 corner points of the bounding box is relatively complex, and the encoding method of the bounding box needs to use the ground plane given by the sensor, while the ground plane is a curved surface that cannot be given, we still use the four corner points and two heights of the bounding box for encoding, reducing the high dimension to 10, as Figure 3 shown. The difference is that the two heights refer to the distances from the upper and lower planes of the bounding box to the sensor (this distance can be obtained by the sensor), and the four corner points are the four points on the upper plane of the bounding box. Therefore, the target to be regressed is (x1 * ~x4 * ,y1 * ~y4 * ,l1 * ,l2 * ). By comparing the IoU between the bounding box and the ground truth box to distinguish the information useful to us, an IoU less than 0.4 is considered a background point, and an IoU greater than 0.6 is classified as a foreground point. For the redundant bounding boxes, 3DNMS is used to improve the accuracy.

[0061] After the 3D bounding box is proposed, the target to be regressed is (x u ,y u ,z u ,l u ,w u ,h u ,θ u ), μ ∈ (g, a) 7 parameters. Among them, (x, y, z) represents the center coordinate value of the box, (l, w, h) represents the size of the box, θ is the angle, the subscript g represents the ground truth box, and the subscript a represents the bounding box. The regression residuals of the ground truth box and the bounding box can be expressed as formula (2):

[0062]

[0063] Among them, The superscript gt represents the ground truth box, and the superscript an represents the bounding box. The regression residual loss between the two is represented by a smooth function:

[0064] L reg = ∑ Smooth-L1(Δ(x,y,z,l,w,h,θ))

[0065] Due to the large gap in the number of foreground and background points, the focal loss function is adopted for the classification loss, L cls = -α an (1 - p an ) γ logp an , p an is the class probability of the bounding box. The target defect does not have an orientation requirement like a vehicle, so the proportion of the classification loss function is reduced to 0.001. In addition, the corner loss is the Euclidean distance between the 4 corners on the bounding box and the ground truth box (GT box) and the distances between the upper and lower planes of the bounding box and the sensor. Therefore, it can be expressed as The total loss function is:

[0066] L total = L reg + L cls + L corner

[0067] Embodiment 2

[0068] In this Embodiment 2, as Figure 4 shown, a tire bubble defect detection system based on 3D point cloud includes: a fusion sampling module, a point voting module, and a proposal refinement module;

[0069] The fusion sampling module includes an FPS sampling device and a C-FPS sampling device, which are connected to the point voting module and are used to sample the original point cloud and extract feature information to obtain the points after feature extraction;

[0070] The point voting module includes a local feature voting device and a global feature voting device, and is also connected to the proposal refinement module, which is used to arrange the points after feature extraction, output a subset of feature vectors of N points, and vote and group the points in the feature vector subset through an MLP network; among them, the local voting feature device is used to retain some points and their features after FPS sampling, take the output after passing through the max pooling layer as a global feature, and add it as the first part of the new feature vector set to the feature vector subset; for the density feature of the points, use its kernel density estimation density feature, and extract the density feature and other features into the feature vector subset by means of a multi-layer perceptron and max pooling operations; the global feature voting device is used to extract the points after C-FPS processing as local features. After processing the original point cloud by fusion sampling in multiple SA layers, a subset processed by the C-FPS sampling method and its corresponding features are obtained. Taking the subset as the initial center point, under the supervision of the loss function, the initial center point moves relatively to generate the final center point, and the output is the local feature.

[0071] The proposal refinement module is used to detect whether there are defective targets in the point cloud. If there are such defective targets, proposals and refinements are carried out.

[0072] Embodiment III

[0073] In this Embodiment III, in order to be able to independently establish a bubble defect data set, an experimental device is built. The length and width of the bubble defects vary from a few millimeters to a few centimeters. In this application, clay is used to simulate various bubble defects of different shapes to maintain the diversity of the data set.

[0074] The data set contains 2,500 samples. The data set is divided into a training set and a test set, with a ratio of 9:1. The training set has 2,250 samples and the test set has 250 samples. During training, the global parameters are: the sample batch is 4, the maximum number of sample training times is 210, and the initial learning rate is set to 0.001.

[0075] To verify the quality of the proposed network model, this application uses parameters such as average precision (AP), mean average precision (mAP), recall, and average recall (AR) to evaluate the model. Since there are only two categories, background and detection target, in the object detection classification described in this application, AP and mAP, recall and AR represent the same meaning. As shown in Tables 3 and 4, when the IoU thresholds are 0.3 and 0.6 respectively, the model is evaluated every 10 training epochs. When the thresholds are different, the pattern of mAP also varies to some extent. At the beginning of training in the table, mAP shows large oscillations between training epochs 10 to 20 and 40 to 50, reaching up to 30%. After the training epoch reaches 70, there are no longer large oscillations, but it rises steadily. After the training epoch reaches 90, there are small oscillations, which means the performance of the model reaches its peak at this time, and the maximum oscillation value reaches 80%. In the table, mAP oscillates to a certain extent before the training epoch reaches 50, and then rises steadily until it stabilizes. The experimental results also prove that a larger learning rate will cause mAP to oscillate, and as the learning rate decays continuously, mAP also rises steadily. As can be seen from Table 5, the target AR decreases as the threshold increases.

[0076] Table 3

[0077]

[0078] Table 4

[0079]

[0080] Table 5

[0081]

[0082] As Figure 5 shown, it is the change of the loss function during training. The solid line represents the total loss value, which is initially high. As the training epoch increases, the function value continuously decreases, showing oscillations between training epochs 40 - 90. After the 100th training epoch, the downward trend of the loss function slows down and converges to about 0.01. The double solid line represents the size loss, with an initial value of 0.02, slightly lower than the total loss. Its trend is basically the same as that of the total loss. After ignoring the direction loss, it can be considered that the size loss guides the trend of the total loss, and the value converges to 0.0078. The dashed line represents the center loss, which finally converges to 0.0003, and its value is about 1 / 10 of the total loss, so the fluctuations are not obvious. The average size of the sample boxes annotated in the dataset is (0.045, 0.047, 0.03). After training, the size loss accounts for 16% - 26% of the average size, while the center loss accounts for 0.1% - 6.38%.

[0083] The embodiments described above are only descriptions of the preferred embodiments of the present application, and do not limit the scope of the present application. Without departing from the design spirit of the present application, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present application shall fall within the protection scope determined by the claims of the present application.

Claims

1. A method for detecting tire bubble defects based on 3D point cloud, characterized in that, It includes the following steps: S1. Sample the original point cloud, sample it according to a preset ratio using different sampling methods, extract feature information, and obtain the points after feature extraction; S2. Arrange the points after feature extraction to obtain a subset of feature vectors of N points, and vote and group the points in the subset of feature vectors; S3. Detect whether there are defective targets among the points after feature extraction. If there are such defective targets, conduct proposal and refinement; The sampling methods include: Sampling the original point cloud with the SA layer in Pointnet++ as the backbone and obtaining feature information. One SA layer includes sampling, grouping, and local feature extraction of feature points. In the sampling layer of SA, sample a preset ratio of points according to FPS sampling and C-FPS sampling respectively, use the focal loss function to balance the gap in the number of points, and complete the feature extraction of the input point cloud with the help of a multi-layer perceptron; The FPS sampling is the farthest point sampling method, and the C-FPS sampling is the sampling method based on extracting curvature features in the Euclidean space; Using the fusion sampling strategy, in the sampling layer of SA, sample a certain proportion of points by the two sampling methods of FPS and the proposed C-FPS respectively, as shown in formula (1): N total = λN C-FPS + βN FPS where λ is the preset ratio of C-FPS sampling, β is the preset ratio of FPS sampling, and λ + β = 1; The voting method includes: local feature voting and global feature voting; The local feature voting includes: Retain some points and their features after the FPS sampling, take the output after passing through the max pooling layer as a global feature, and add it as the first part of the new feature vector set to the subset of feature vectors; For the density feature of the points, use its kernel density estimation density feature, and extract the density feature and other features into the subset of feature vectors with the help of a multi-layer perceptron and max pooling operations; The global feature voting includes: Extract the points after the C-FPS sampling as local features. After processing the original point cloud with the fusion sampling in multiple SA layers, obtain a subset processed by the C-FPS sampling method and the corresponding features. Take the subset as the initial center point. Under the supervision of the loss function, the initial center point moves relatively to generate the final center point, and the output is the local feature; To avoid generating and refining proposal boxes for each frame of 3D point cloud collected to save detection time, a simple method is proposed to judge whether there are defective targets in this frame of point cloud, and classify the target object before the 3D box proposal. If the target object is classified, then conduct proposal and refinement, otherwise the process does not take effect; First, define the clustered points separated as CP = {C i , F i}, where i = 1, Λ, n and the centroid b of the clustered points. Here, C i ∈ R 3 represents the coordinates of the clustered points, and F i ∈ R represents the features of the clustered points. If there are defective targets in the point cloud, the number of generated clustered points is much larger than the small number of clustered points generated by the point cloud without defective targets. Therefore, (C i ) exist - (C i ) no > T, and use C i - b to convert the clustered points into the centroid local normalized coordinate system. Since the clustered points of the defective targets are relatively concentrated, there exists in the Euclidean space containing defective targets with the centroid point of the clustered points as the center point, and the average value of the neighborhood points near the center point and the center point is less than the threshold r. However, for the clustered points without defective targets, they will be scattered irregularly in the entire point cloud space. Therefore, it shows the characteristics of a small number and a large r i . Each point in the cluster point generates a 3D bounding box. Since directly encoding the 8 corner points of the bounding box is relatively complex, and the encoding method of the bounding box needs to use the ground plane given by the sensor, while the ground plane is a curved surface that cannot be given, the four corner points and two heights of the bounding box are still used for encoding, reducing the high dimension to 10; differently, the two heights refer to the distances from the upper and lower planes of the bounding box to the sensor, and the four corner points are the four points on the upper plane of the bounding box. Therefore, the target to be regressed is (x1 * ~x4 * ,y1 * ~y4 * ,l1 * ,l2 * ); Useful information is distinguished by comparing the IoU between the bounding box and the ground truth box. If the IoU is less than 0.4, it is considered a background point. If the IoU is greater than 0.6, it is classified as a foreground point; for the redundant bounding boxes, 3DNMS is used to improve the accuracy; After the 3D bounding box is proposed, the target to be regressed is (x u , y u , z u , l u , w u , h u , θ u ), μ ∈ (g, a) seven parameters. Among them, (x, y, z) represents the center coordinate value of the box, (l, w, h) represents the size of the box, θ is the angle, the subscript g represents the ground truth box, and the subscript a represents the bounding box; the regression residuals of the ground truth box and the bounding box can be expressed as: Δθ = sin(θ gt - θ an ) Among them, The superscript gt represents the ground truth box, and the superscript an represents the bounding box. The regression residual loss between the two is represented by a smoothing function: L reg = ∑ Smooth-L1(Δ(x, y, z, l, w, h, θ)) Due to the large gap in the number of foreground and background points, the focal loss function is adopted for the classification loss, L cls = -α an (1 - p an ) γ log p an , where p an is the class probability of the bounding box; the target defect does not have an orientation requirement like a vehicle, so the proportion of the classification loss function is reduced to 0.001; in addition, the corner loss is the Euclidean distance between the 4 corners on the bounding box and the ground truth box and the distances between the upper and lower planes of the bounding box and the sensor, so it can be expressed as The total loss function is: L total = L reg + L cls + L corner 。 2. A tire bubble defect detection system based on 3D point cloud, wherein the system applies the method described in claim 1, characterized in that, It includes: A fusion sampling module, a point voting module, and a proposal refinement module; The fusion sampling module is connected to the point voting module. The fusion sampling module is used to sample the original point cloud and extract feature information to obtain the points after feature extraction; The point voting module is also connected to the proposal refinement module. The point voting module is used to arrange the points after feature extraction, output a subset of feature vectors of N points, and vote and group the points in the subset of feature vectors through a multi-layer perceptron network; The proposal refinement module is used to detect whether there are defective targets in the point cloud. If there are such defective targets, proposals and refinements are carried out.

3. The tire bubble defect detection system based on 3D point cloud according to claim 2, characterized in that, The fusion sampling module includes: an FPS sampling device and a C-FPS sampling device.

4. The tire bubble defect detection system based on 3D point cloud according to claim 2, characterized in that, The point voting module includes: a local feature voting device and a global feature voting device; The local feature voting device is used to retain some points and their features after FPS sampling, take the output after passing through the max pooling layer as a global feature, and add it as the first part of the new feature vector set to the subset of feature vectors; for the density feature of the points, use its kernel density estimation density feature, and extract the density feature and other features into the subset of feature vectors by means of a multi-layer perceptron and max pooling operations; The global feature voting device is used to extract the points after C-FPS sampling as local features. After processing the original point cloud by fusion sampling in multiple SA layers, a subset after C-FPS sampling processing and the corresponding features are obtained. Taking the subset as the initial center point, the initial center point moves relatively under the supervision of the loss function to generate a final center point, and the output is the local feature.

Images