Rubber texture recognition method based on machine vision

Abstract

The present invention provides a method for rubber grain recognition based on machine vision. The method involves capturing an image of the rubber to be identified using a camera array, reconstructing a point cloud model of the rubber based on the image data and depth information, and projecting it at an appropriate angle to obtain a two-dimensional image of the rubber. The two-dimensional rubber image is then input into a pre-trained machine vision recognition model. This model, combined with a multi-scale adjustment loss function, enhances adaptability to rubbers of varying sizes and improves detection accuracy, thereby outputting identification information for the rubber grain. This method can improve the accuracy and adaptability of real-time rubber grain recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a rubber grain recognition method based on machine vision, which realizes automatic rubber grain recognition based on a machine vision algorithm. Background Art

[0002] Rubber is widely used in industries such as transportation, water conservancy, construction, healthcare, chemicals, and aerospace, providing sealing, isolation, shock absorption, cushioning, and flexible connections. Rubber products are diverse and offer a broad future. In practice, the friction coefficient between the rubber and the contact surface can be adjusted by adding textured structures to meet specific needs.

[0003] However, during the manufacturing and use of rubber products, defects such as broken and damaged rubber surfaces can occur due to process flow, improper worker operation, and excessive use. This can lead to quality degradation, affect the actual use of rubber products, and in severe cases, endanger user safety. Therefore, the detection and identification of rubber surface textures is particularly important.

[0004] Currently used recognition solutions fall into two main categories. The first is manual inspection, which offers high accuracy but requires a high level of experience from the technicians involved and results in low efficiency. The second approach is traditional digital image processing algorithms, which utilize binarization and edge detection algorithms to identify rubber texture images. This improves efficiency to a certain extent, but due to limitations in image processing algorithms, accuracy remains low.

[0005] In recent years, with the continuous development of computer technology, emerging technologies, represented by machine vision, have been widely used by researchers across various industries. Machine vision has also found applications in digital image data processing, such as image recognition and object detection using convolutional neural networks. Convolutional neural networks (CNNs) capture relevant data and input them into network models, eliminating the tedious manual feature selection process during network training. They also enable continuous model optimization based on actual application scenarios, improving both detection and recognition efficiency.

[0006] Therefore, how to achieve rapid, accurate and automated recognition of rubber textures is a technical problem that needs to be solved urgently in this field. Summary of the Invention

[0007] The purpose of the present invention is to provide a rubber grain recognition method based on machine vision, which realizes automatic recognition of rubber grain based on machine vision algorithms. It aims to take advantage of the ability of machine vision algorithms to automatically extract key features from data without human intervention, and apply machine vision to the recognition of rubber grains to solve the problems of low efficiency and poor accuracy of traditional recognition methods.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] Step 1: Use an RGB-D camera array to capture images of the rubber to be identified, to obtain multiple rubber image data including textures and corresponding depth information;

[0010] Step 2: Reconstruct the corresponding point cloud data based on each image and the corresponding depth information to obtain multiple point cloud data, and use the multiple point cloud data to perform point cloud fusion to obtain a point cloud model of the rubber;

[0011] Step 3: Project the rubber point cloud model into a two-dimensional image;

[0012] Step 4: Input the obtained two-dimensional image data into a pre-trained machine vision recognition model to obtain the rubber texture recognition result.

[0013] Furthermore, before point cloud reconstruction, a process of pre-processing the collected images is also included, which is characterized by:

[0014] Collecting rubber image data;

[0015] Perform Gaussian filtering on the collected image data to obtain the filtered image. The specific calculation formula is as follows:

[0016] G(x,y)=f(x,y)*h(i,j)=∑ i ∑ j f(xi,yj)h(i,j)

[0017] Where h(i,j) represents the Gaussian kernel function of size i*j, f(x,y) is the original image data, (x,y) is the pixel point, and * represents the convolution operation.

[0018] Perform a contrast linear stretch on the filtered image.

[0019] Furthermore, the specific process of reconstructing the 3D point cloud of the rubber image data is as follows:

[0020] Feature points are extracted from each image captured by the RGB-D camera array, and then feature point matching is performed between the captured images based on the feature points to determine the position of the same feature point in each image.

[0021] Based on the camera intrinsic parameter matrix and pixel depth, the pixel coordinates in the two-dimensional image are converted to point cloud coordinates. The calculation formula is as follows:

[0022]

[0023] Among them, P(u,v) is the point cloud coordinate of pixel point P, u and v are the pixel coordinates of pixel point P in the two-dimensional image, D(u,v) is the depth information corresponding to pixel point P, K is the intrinsic parameter matrix of the camera, is the coordinate matrix.

[0024] Based on the feature points, point cloud fusion is performed to eliminate redundant and repeated point cloud data, thereby reconstructing the final rubber point cloud model.

[0025] Furthermore, the training process of the machine vision recognition model includes:

[0026] Rubber images with different texture states are collected and classified labels are assigned to the rubber images with different texture states. The original data set is constructed, and the collected images are divided into a training set and a test set. The training set is enhanced, and the original image data is rotated, noise is added, brightness is adjusted, and downsampling is performed to obtain enhanced training data. The obtained label information is processed using the k-means algorithm to obtain a bounding box (i.e., anchor) suitable for the rubber texture image.

[0027] The training set is input into the feature extraction part of the preset machine vision recognition model to extract the multi-scale feature map of the rubber image.

[0028] The multi-scale feature map is input into the prediction layer of the machine vision recognition model to obtain the detection frame of the rubber image and output the recognition map of the rubber texture. After training, the machine vision recognition model is obtained.

[0029] Input the test set images into the trained machine vision recognition model and output the recognition results. Determine whether the training is complete based on the precision and recall rates. When the precision and recall rates do not meet the threshold, adjust the parameter values in the model and retrain until the threshold is met.

[0030] Furthermore, according to the different states of the rubber texture, it can be classified into intact, worn and broken.

[0031] Furthermore, the training set data is resized and normalized before training to speed up the convergence of the network in the subsequent training process.

[0032] Furthermore, when extracting multi-scale features, a channel attention mechanism is introduced. For the feature vector f obtained at each scale, global average pooling and global maximum pooling operations are performed respectively. The weighted sum of the global average pooling results and the global maximum pooling results is taken. Without reducing the channel dimension, a convolution kernel of size 3 is used to implement fast 1D convolution to obtain the attention channel weights, and finally the channel attention feature map F is obtained. The convolution kernel of size 3 represents the coverage of local cross-scale interactions. The specific calculation formula is as follows:

[0033] F=w1P avg (f)+w2P max (f)

[0034] Among them, F is the output feature vector, P avg (f) is the global average pooling of the feature vector f, P max (f) is the global maximum pooling of the feature vector f.

[0035] Furthermore, the loss function in model training is defined as:

[0036] L total =a*BCE class +b*BCE conf +c*CIoU+d*MIoU

[0037] Among them, BCE class is the classification loss. For each predicted box, the loss is calculated as follows:

[0038]

[0039] C is the number of categories; p c It is a binary encoding of the true label. If the predicted box corresponds to a certain category, the value of the category position is 1, and the rest are 0. is the probability that the model predicts the box belongs to category c.

[0040] BCE conf Is a confidence loss function used to determine whether the predicted box contains the rubber texture object. If the intersection over union (IoU) of the predicted box and any true value is greater than a preset threshold, the prediction is a positive sample; otherwise, it is a negative sample. The intersection over union (IoU) is a metric used to measure the performance of object detection algorithms. It indicates the degree of overlap between the predicted box and the true bounding box. When IoU = 0, it means that the predicted box does not overlap with the true bounding box. When IoU = 1, it means that the predicted box completely overlaps with the true bounding box.

[0041] The loss is calculated as follows:

[0042]

[0043] p o is the true label (0 or 1) of the predicted box as a positive sample, It is the probability that the model judges the prediction box to be a positive sample.

[0044] CIoU is the prediction box regression loss function, which is calculated as follows:

[0045]

[0046] ρ u and ρ c are the center point distances between the predicted box and the true bounding box, Φ u and Φ c are the aspect ratios of the predicted box and the true box, respectively.

[0047] MIoU is a multi-scale adjustment loss function, which is calculated as follows:

[0048]

[0049] M=w*h+w t *h t -w i *h i

[0050] Among them, w and h are the width and height of the prediction box, w i 、h i is the width and height of the intersection area between the predicted box and the real box, w t 、h t is the width and height of the real box, w u 、h u The width and height of the minimum rectangular area including the predicted box and the real box, x and y represent the horizontal and vertical coordinates of the center point of the predicted box, t 、y t Represents the horizontal and vertical coordinates of the center point of the ground truth box.

[0051] a, b, c, d are the total loss function L total The weights of each loss function in .

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] 1. The system captures rubber images using an RGB-D camera array to obtain multiple images and corresponding depth information. Based on these images and depth information, a point cloud model of the rubber is reconstructed, and then projected at an appropriate angle to obtain a two-dimensional image of the rubber. This improves rubber recognition accuracy and ensures optimal viewing angles for the rubber. It avoids issues such as overlapping and obstructed viewing of the rubber caused by improper camera placement or angle settings, which can result in a poor viewing range.

[0054] 2. Introducing the channel attention mechanism in model training allows the model to focus on the rubber texture part of the input image data and pay less attention to the background part. This helps the model better understand and classify the rubber texture data, improve the accuracy of model recognition, and improve recognition efficiency.

[0055] 3. A multi-scale adjustment loss function is set in the model to enhance the adaptability of the recognition model to rubber of different sizes and improve the detection accuracy and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0057] Figure 1 This is a flow chart of an embodiment of a method for identifying rubber grains based on machine vision according to the present application;

[0058] Figure 2 This is a flowchart of an embodiment of the present application for preprocessing image data;

[0059] Figure 3 It is a flowchart of training a machine vision recognition model according to an embodiment of the present application. DETAILED DESCRIPTION

[0060] The following will clearly describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative work are within the scope of protection of this application.

[0061] Example 1

[0062] See also Figure 1 , Figure 1The figure is a flow chart of an embodiment of the rubber grain recognition method based on machine vision of the present application. As shown in the figure, the method includes:

[0063] Step 1: Use an RGB-D camera array to capture the rubber image to be identified to obtain multiple rubber image data including texture and corresponding depth information.

[0064] Since the camera array can capture multiple images of the rubber from different angles, the collected image data can completely cover the appearance details of the rubber to be identified, which is conducive to the subsequent identification of the rubber texture.

[0065] Step 2: Reconstruct the corresponding point cloud data based on each image and the corresponding depth information to obtain multiple point cloud data, and use the multiple point cloud data to perform point cloud fusion to obtain a point cloud model of the rubber.

[0066] Step 3: Project the point cloud model of the rubber into a two-dimensional image.

[0067] While point cloud models can accurately represent the geometry and texture details of rubber, point clouds are three-dimensional datasets composed of a large number of points. Directly processing the rubber point cloud model consumes a significant amount of computing resources. Therefore, to improve efficiency, the rubber point cloud model is reprojected into a two-dimensional image. Because rubber may be stacked, the captured image may be obscured by other rubber structures. Manual or automated selection of the projection angle of the rubber point cloud model can avoid image incompleteness caused by occlusion, thereby selecting the data that best represents the rubber's current state and improving the accuracy of subsequent model recognition.

[0068] Step 4: Input the obtained two-dimensional image data into a pre-trained machine vision recognition model to obtain the rubber texture recognition result.

[0069] Please refer to Figure 2 , Figure 2 It is a flowchart of another embodiment of the present application.

[0070] Specifically, in some embodiments, step one may include a process of pre-processing the image:

[0071] S21: collecting rubber image data;

[0072] S22: Perform Gaussian filtering on the collected image data to obtain a filtered image. The specific calculation formula is as follows:

[0073] G(x,y)=f(x,y)*h(i,j)=∑ i Σ j f(xi,yj)h(i,j)

[0074] Where h(i,j) represents the Gaussian kernel function of size i*j, f(x,y) is the original image data, (x,y) is the pixel point, and * represents the convolution operation.

[0075] S23: Perform contrast linear stretching on the filtered image.

[0076] In the above steps, the image data is first Gaussian filtered to remove obvious random noise. This prevents the subsequent model recognition from mistaking noise for useful information, which could reduce recognition accuracy. Furthermore, because Gaussian filtering can blur the image, a contrast linear stretch is performed to improve contrast and clarity.

[0077] By preprocessing the images, the quality and usability of the collected images can be improved, high-quality data can be provided for the recognition model, and the recognition accuracy and efficiency can be improved.

[0078] Furthermore, the specific process of reconstructing the 3D point cloud of the rubber image data is as follows:

[0079] Feature points are extracted from each image captured by the RGB-D camera array, and then feature point matching is performed between the captured images based on the feature points to determine the position of the same feature point in each image.

[0080] Based on the camera intrinsic parameter matrix and pixel depth, the pixel coordinates in the two-dimensional image are converted to point cloud coordinates. The calculation formula is as follows:

[0081]

[0082] Among them, P(u,v) is the point cloud coordinate of pixel point P, u and v are the pixel coordinates of pixel point P in the two-dimensional image, D(u,v) is the depth information corresponding to pixel point P, K is the intrinsic parameter matrix of the camera, is the coordinate matrix.

[0083] Based on the feature points, point cloud fusion is performed to eliminate redundant and repeated point cloud data, thereby reconstructing the final rubber point cloud model. Point cloud fusion can adopt any existing known technology and will not be described in detail here.

[0084] Please refer to Figure 3 , Figure 3 This is a flowchart of the training machine vision recognition model of this application.

[0085] S31: Collect rubber images with different texture states and assign classification labels to the rubber images with different texture states to construct an original dataset. The rubber images can be classified into intact, worn, and broken according to the different texture states.

[0086] The collected images are divided into training and test sets. The training set is enhanced by rotating, adding noise, adjusting brightness, and downsampling the original image data to obtain enhanced training data. This method can expand the training data sample when the original training data is limited, thereby improving the robustness and generalization ability of model training.

[0087] Preferably, the training set data can be resized and normalized before training to speed up the convergence of the network in the subsequent training process.

[0088] The obtained label information is processed using the k-means algorithm to obtain a bounding box (i.e., anchor) suitable for the rubber texture image.

[0089] S32: Input the training set into the feature extraction part of the preset machine vision recognition model to extract a multi-scale feature map of the rubber image.

[0090] Multi-scale features can represent the different receptive fields of an image, thereby enhancing the model's ability to detect objects of different sizes. The extraction of multi-scale features is achieved by building a pyramid network structure in the model. The specific method is as follows:

[0091] Use the basic network ResNet to extract the feature representation of the image; through continuous convolutional layers and pooling layers, feature maps of different scales are generated. After each pooling layer, the spatial size of the feature map is halved; in the feature pyramid network, lateral connections are used to combine high-level (low-resolution) feature maps with low-level (high-resolution) feature maps; use upsampling technology to upsample the high-level feature maps to the same spatial size as the low-level feature maps; fuse the upsampled high-level feature maps with the low-level feature maps; thus, a series of feature maps of different scales are obtained.

[0092] S33: Inputting the multi-scale feature map into the prediction layer of the machine vision recognition model to obtain a detection frame of the rubber image, outputting an identification map of the rubber texture, and obtaining a machine vision recognition model through training;

[0093] S34: Input the test set images into the trained machine vision recognition model and output the recognition results. Determine whether the training is completed based on the precision and recall rates. When the precision and recall rates do not meet the threshold, adjust the parameter values in the model and retrain until the threshold is met.

[0094] Preferably, when extracting multi-scale features, a channel attention mechanism can be introduced so that the model focuses on the rubber part in the input image data and pays less attention to the background part, helping the model to better understand and classify the rubber texture data, thereby improving the performance of the model.

[0095] For the feature vector f obtained at each scale, global average pooling and global maximum pooling operations are performed respectively. The weighted sum of the global average pooling results and the global maximum pooling results is performed. Without reducing the channel dimension, a fast 1D convolution is performed using a convolution kernel of size 3 to obtain the attention channel weights. Finally, the feature map F of the channel attention is obtained. The convolution kernel of size 3 represents the coverage of local cross-scale interactions. The specific calculation formula is as follows:

[0096] F=w1P avg (f)+w2P max (f)

[0097] Among them, F is the output feature vector, P avg (f) is the global average pooling of the feature vector f, P max (f) is the global maximum pooling of the feature vector f.

[0098] Preferably, the loss function in model training is defined as:

[0099] L total =a*BCE class +b*BCE conf +c*CIoU+d*MIoU

[0100] Among them, BCE class is the classification loss. For each predicted box, the loss is calculated as follows:

[0101]

[0102] C is the number of categories; p c It is a binary encoding of the true label. If the predicted box corresponds to a certain category, the value of the category position is 1, and the rest are 0. is the probability that the model predicts the box belongs to category c.

[0103] BCE conf Is a confidence loss function used to determine whether the predicted box contains the target rubber object. If the intersection over union (IoU) of the predicted box and any true value is greater than a preset threshold, the prediction is a positive sample, otherwise it is a negative sample. The intersection over union (IoU) is a metric used to measure the performance of the target detection algorithm. It indicates the degree of overlap between the predicted box and the true bounding box. When IoU = 0, it means that the predicted box does not overlap with the true bounding box. When IoU = 1, it means that the predicted box completely overlaps with the true bounding box.

[0104] The loss is calculated as follows:

[0105]

[0106] po is the true label (0 or 1) of the predicted box as a positive sample, It is the probability that the model judges the prediction box to be a positive sample.

[0107] CIoU is the prediction box regression loss function, which is calculated as follows:

[0108]

[0109] ρ u and ρ c are the center point distances between the predicted box and the true bounding box, Φ u and Φ c are the aspect ratios of the predicted box and the true box, respectively.

[0110] MIoU is a multi-scale adjustment loss function. Traditional loss functions fail to consider the scale-varying nature of the detected target. However, rubber products vary significantly in size due to different application scenarios. Using a single-scale loss function can lead to inaccuracies in the prediction boxes for rubber products of varying sizes. Therefore, a multi-scale adjustment loss function is proposed to enhance the model's adaptability to different sizes, thereby improving detection accuracy. The specific calculation method for the multi-scale adjustment loss function is as follows:

[0111]

[0112] M=w*h+w t *h t -w i *h i

[0113] Among them, w and h are the width and height of the prediction box, w i 、h i is the width and height of the intersection area between the predicted box and the real box, w t 、h t is the width and height of the real box, w u 、h u The width and height of the minimum rectangular area including the predicted box and the real box, x and y represent the horizontal and vertical coordinates of the center point of the predicted box, t 、y t Represents the horizontal and vertical coordinates of the center point of the ground truth box.

[0114] According to the actual rubber texture image data, adjust the total loss function L total The weights a, b, c, and d of each loss function are used to obtain the optimal loss function and improve the model's adaptability to rubber texture recognition of different sizes.

[0115] The present invention provides a machine vision-based rubber texture recognition method. This method uses a machine vision algorithm to identify rubber textures. An RGB-D camera array captures rubber images to obtain multiple image data and corresponding depth information. Based on these multiple rubber image data and depth information, a point cloud model of the rubber is reconstructed, and then projected at an appropriate angle to obtain a two-dimensional image of the rubber. This method improves the accuracy of rubber recognition, ensuring an optimal viewing angle for the rubber. It avoids issues such as overlap and obstruction of the rubber view caused by improper camera placement or monitoring angle settings, which can lead to poor monitoring coverage. Furthermore, a channel attention mechanism is introduced during model training, allowing the model to focus on the rubber texture in the input image data and pay less attention to the background. This helps the model better understand and classify rubber texture data, improving model recognition accuracy and efficiency. Furthermore, a multi-scale adjustment loss function is incorporated into the model to enhance the recognition model's adaptability to rubbers of varying sizes, improving both recognition accuracy and adaptability.

[0116] The above is an illustration of the present disclosure and should not be considered as a limitation thereof. Although several exemplary embodiments of the present disclosure have been described, it will be readily understood by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure as defined by the claims. It should be understood that the above is an illustration of the present disclosure and should not be considered as limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present disclosure is defined by the claims and their equivalents.

[0117] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative uses of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0118] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0119] Finally, it should be noted that those skilled in the art will easily understand that the embodiments described above are only preferred embodiments of the present invention, which are used to illustrate the technical solutions of this application rather than to limit them. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A rubber texture recognition method based on machine vision, characterized in that: The method comprises: Step 1: Use an RGB-D camera array to capture images of the rubber to be identified, to obtain multiple rubber image data including textures and corresponding depth information; Step 2: Reconstruct the corresponding point cloud data based on each image and the corresponding depth information to obtain multiple point cloud data, and use the multiple point cloud data to perform point cloud fusion to obtain a point cloud model of the rubber; Step 3: Select a suitable angle and project the rubber point cloud model into a two-dimensional image; Step 4: Input the obtained two-dimensional image data into the pre-trained machine vision recognition model to obtain the rubber texture recognition result; The training process of the machine vision recognition model includes: Input the training set into the feature extraction part of the preset machine vision recognition model to extract the multi-scale feature map of the rubber image; When extracting multi-scale features, the channel attention mechanism is introduced. For the feature vector f obtained at each scale, global average pooling and global maximum pooling operations are performed respectively. The weighted sum of the global average pooling results and the global maximum pooling results is performed. Without reducing the dimension of the channel, a convolution kernel of size 3 is used to implement fast 1D convolution to obtain the attention channel weight. Finally, the feature map F of the channel attention is obtained, where the convolution kernel of size 3 represents the coverage of local cross-scale interactions. The specific calculation formula is as follows: Among them, F is the output feature vector, P avg (f) is the global average pooling of the feature vector f, P max (f) is the global maximum pooling of the feature vector f, w1 and w2 are the weights when summing; The loss function in model training is defined as: Among them, BCE class is the classification loss. For each predicted box, the loss is calculated as follows: C is the number of categories; p c It is a binary encoding of the true label. If the predicted box corresponds to a certain category, the value of the category position is 1, and the rest are 0. is the probability that the model predicts that the box belongs to category c; BCE class It is a confidence loss function used to determine whether the predicted box contains the target object. If the intersection over union (IoU) of the predicted box and any true value is greater than a preset threshold, the prediction is a positive sample, otherwise it is a negative sample. The intersection over union (IoU) is an indicator used to measure the performance of the target detection algorithm. It indicates the degree of overlap between the predicted box and the true bounding box. When IoU=0, it means that the predicted box does not overlap with the true bounding box. When IoU=1, it means that the predicted box completely overlaps with the true bounding box. The loss is calculated as follows: p o is the true label of the predicted box as a positive sample, is the probability that the model judges the prediction box to be a positive sample; CIoU is the prediction box regression loss function, which is calculated as follows: ρ u and ρ c are the center point distances between the predicted box and the true bounding box, Φ u and Φ c are the aspect ratios of the predicted box and the true box respectively; MIoU is a multi-scale adjustment loss function, which is calculated as follows: Among them, w and h are the width and height of the prediction box, w i 、h i is the width and height of the intersection area between the predicted box and the real box, w t 、h t is the width and height of the real box, w u 、h u The width and height of the minimum rectangular area including the predicted box and the real box, x and y represent the horizontal and vertical coordinates of the center point of the predicted box, t 、y t Represents the horizontal and vertical coordinates of the center point of the real box; a, b, c, d are the total loss function L total The weights of each loss function in .

2. The method according to claim 1, characterized in that Before point cloud reconstruction, the collected images are preprocessed, including: Collecting rubber image data; Perform Gaussian filtering on the collected image data to obtain the filtered image. The specific calculation formula is as follows: Where h(i,j) represents the Gaussian kernel function of size i*j, f(x,y) is the original image data, (x,y) is the pixel point, and * represents the convolution operation; Perform a contrast linear stretch on the filtered image.

3. The method according to claim 1, characterized in that Point cloud reconstruction of rubber image data, including: Extract feature points from each image captured by the RGB-D camera array, and then perform feature point matching between the captured images to determine the location of the same feature point in each image; Based on the camera intrinsic parameter matrix and pixel depth, the pixel coordinates in the two-dimensional image are converted to point cloud coordinates. The calculation formula is as follows: Among them, P(u,v) is the point cloud coordinate of pixel point P, u and v are the pixel coordinates of pixel point P in the two-dimensional image, D(u,v) is the depth information corresponding to pixel point P, K is the intrinsic parameter matrix of the camera, is the coordinate matrix; Based on the feature points, point cloud fusion is performed to eliminate redundant and repeated point cloud data, thereby reconstructing the final rubber point cloud model.

4. The method according to claim 1, wherein The training process of the machine vision recognition model includes: Rubber images with different texture states are collected and assigned classification labels. An original dataset is constructed, and the collected images are divided into a training set and a test set. The training set is enhanced by rotating, adding noise, adjusting brightness, and downsampling the original image data to obtain enhanced training data. The obtained label information is processed using the k-means algorithm to obtain a bounding box suitable for the rubber texture image. The multi-scale feature map is input into the prediction layer of the machine vision recognition model to obtain the detection frame of the rubber image and output the recognition map of the rubber texture. After training, the machine vision recognition model is obtained. Input the test set images into the trained machine vision recognition model and output the monitoring results. Determine whether the training is complete based on the precision and recall rates. When the precision and recall rates do not meet the threshold, adjust the parameter values in the model and retrain until the threshold is met.

5. The method according to claim 4, characterized in that According to the different states of the rubber texture, it is classified into intact, worn and broken.

6. The method according to claim 4, characterized in that Before training, the training set data is subjected to size conversion and data normalization preprocessing.

Images