A detection method and system for detecting tire defects

Abstract

This invention discloses a detection method and system for detecting tire defects, comprising the following steps: acquiring original image sequences of the tire under four different illumination angles; calculating the normal vector data N and albedo data ρ of the target surface based on the original image sequence using a photometric stereo algorithm and a preset light source direction matrix L; constructing a feature image containing three independent physical dimensions based on the normal vector data N and albedo data ρ: a first feature map representing macroscopic geometric deformation, a second feature map representing the spectral properties of the material, and a third feature map representing microscopic texture deviation; merging the first, second, and third feature maps into multi-channel image data, and using it as input data for training or inference of a deep learning model to output whether the tire has defects and the type of defects; this invention constructs high signal-to-noise ratio input data to determine whether the tire has defects.

Description

Technical Field

[0001] This invention relates to the field of tire appearance defect detection technology, and in particular to a detection method and system for detecting tire defects. Background Technology

[0002] Currently, tire appearance defect detection for vehicles largely relies on purely visual inspection, that is, judging whether the tire appearance is up to standard by human observation. With the development of artificial intelligence, detection based on intelligent algorithms is gradually emerging, such as using machine learning models to analyze and detect defects in visible light images.

[0003] However, existing analytical models have the following drawbacks: False positives due to feature coupling: Traditional grayscale or color imaging cannot physically distinguish between "material defects" (such as ink, dirt, discoloration) and "geometric defects" (such as bulges, dents, scratches); in RGB images, these are all manifested as changes in grayscale values, making model convergence difficult and resulting in a high false positive rate; Interference from tire curvature: The tire surface has a cylindrical macroscopic geometry, and its low-frequency curvature changes often occupy most of the dynamic range of the image data, causing minute defect features (high-frequency signals) to be submerged and difficult to display in the image; Single data dimension: Existing photometric stereo applications typically only output normal maps or curvature maps as a single judgment criterion, lacking comprehensive utilization of different physical properties (gloss, albedo, roughness), and failing to meet the high-dimensional feature requirements of data-driven algorithms (such as deep learning). Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned defects in the prior art and provide a detection method and system for detecting tire defects. The original image is decoupled into feature channels of three orthogonal dimensions: geometry, material, and texture through a photometric stereo model, and targeted frequency domain / spatial domain filtering preprocessing is performed to construct high signal-to-noise ratio input data specifically for deep learning model training, thereby determining whether the tire has defects.

[0005] To achieve the above objectives, the present invention provides a detection method for detecting tire defects, comprising the following steps;

[0006] Step S1: Obtain the original image sequence of the tire under four different lighting angles;

[0007] Step S2: Based on the photometric stereo algorithm, using the preset light source direction matrix L, calculate the normal vector data N and albedo data ρ of the target surface according to the original image sequence;

[0008] Step S3: Based on the normal vector data N and albedo data ρ, construct a feature image containing three independent physical dimensions, namely: a first feature image representing macroscopic geometric deformation, a second feature image representing the spectral properties of the material, and a third feature image representing microscopic texture deviation.

[0009] Step S4: Combine the first feature map, the second feature map, and the third feature map into multi-channel image data, and use it as input data for deep learning model training or inference to output whether the tire has defects and the type of defects.

[0010] Preferably, in step S1, a time-frequency flash source is used to illuminate the tire, and a line scan camera is used in conjunction with the time-frequency flash source to collect the light reflected from the tire; the time-frequency flash source adopts an X-shaped layout, with four time-frequency flash sources distributed at the four corners of the tire, namely: upper left corner, lower left corner, lower right corner and upper right corner; the line scan camera collects four original images of the tire illuminated by the time-frequency flash source.

[0011] Preferably, in step S2, the light source direction matrix L is defined as 4 rows × 3 columns, with each row representing the direction vector of a light source; the symbol T is used to represent the tilt angle of the light source, and the symbol P is used to represent the azimuth angle of the light source.

[0012] The light source direction matrix L is as follows:

[0013]

[0014] The light source tilt angle T is a modifiable variable. In the same photometric stereo algorithm, the light source tilt angles in the four directions remain consistent. The value range of the light source tilt angle T is between 0° and 90°. The light source azimuth angle P includes a first light source azimuth angle P1, a second light source azimuth angle P2, a third light source azimuth angle P3, and a fourth light source azimuth angle P4. The first light source azimuth angle P1, the second light source azimuth angle P2, the third light source azimuth angle P3, and the fourth light source azimuth angle P4 represent the azimuth angles of the light sources in the four directions, respectively.

[0015] The meaning of the negative sign: The second column (Y component) of the matrix is ​​"-sinTsinP"; the positive direction of the coordinate system defined by the system is: X is positive when the image is to the right, Y is positive when the image is downward, and Z is positive when the image is perpendicular to the tire surface upward. This direction setting is a conventional setting in the field of machine vision.

[0016] Correspondence: The order in which the time-division frequency flash source is lit and the order in which the line scan camera collects data must strictly correspond one-to-one with the row order of matrix L, from the first light source azimuth angle P1, the second light source azimuth angle P2, the third light source azimuth angle P3 to the fourth light source azimuth angle P4.

[0017] Preferably, the tilt angle T of the light source is 60°; the azimuth angles are fixed as follows: P1=45°, P2=135°, P3=225° and P4=315°.

[0018] Preferably, in step S3, based on the image brightness I and the light source direction matrix L acquired by the line scan camera, the equation I=L·ρ·N is solved by the least squares method to separate the albedo ρ and the normal vector N; then, the first feature map, the second feature map and the third feature map are constructed.

[0019] Preferably, the method for constructing the first feature map is as follows: after performing large-scale Gaussian smoothing on the X and Y components of the normal vector N data, the gradient modulus is calculated and mapped to the gray space. ;in, For large-scale smoothing, the aim is to extract low-frequency geometric features.

[0020] Preferably, the method for constructing the second feature map is as follows: converting the albedo data to the Lab color space and calculating the Euclidean distance between each pixel and a preset reference neutral color;

[0021] First, obtain the albedo image ρ{RGB} containing color information or the average image of the original image. Then, using the standard color space conversion algorithm, use "cvtColor" in OpenCV to convert it from the RGB space to the CIE-Lab color space and extract the a component (magenta-green) and b component (yellow-blue).

[0022] Then, the chromatic Euclidean distance of the pixel relative to neutral gray (a=128, b=128) is calculated using the following formula: This eliminates the influence of brightness (L component) and extracts only the color deviation characteristics of the pure material.

[0023] Preferably, the method for constructing the third feature map is as follows: calculate the absolute value of the difference between the original normal vector and the smoothed normal vector to remove the macroscopic geometric curvature background of the target object surface and retain high-frequency texture features; The difference between the original normal and the smoothed normal is calculated as described above, which is equivalent to a frequency domain high-pass filter; the macroscopic curvature of the tire is removed while the microscopic texture is preserved.

[0024] Preferably, in step S4, the first feature map, the second feature map, and the third feature map are merged into a three-channel tensor and input into a deep learning model for defect identification; the deep learning model is a convolutional neural network (CNN) and a visual Transformer (ViT).

[0025] The present invention also provides a detection system for detecting tire defects, comprising:

[0026] Image acquisition module: The image acquisition module is used to control the time-division frequency flash source and the line scan camera, and to acquire images using the line scan camera;

[0027] Processor: The processor is connected to the image acquisition module and is used to execute the detection method for detecting tire defects as described in any one of claims 1 to 9; and to process the acquired image data and output the processed results;

[0028] Memory: The memory is connected to the processor and is used to store the computer program for detecting tire defects, the images acquired by the line scan camera, and the processed results.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0030] 1. This invention uses a photometric stereo algorithm to decouple physical features before data is input into a deep learning model, separating each feature into a first feature map representing macroscopic geometric deformation, a second feature map representing the spectral properties of the material, and a third feature map representing microscopic texture deviation. This greatly reduces the complexity of the hypothesis space of the deep learning model, enabling it to converge quickly even with small sample conditions.

[0031] 2. The first feature map representing macroscopic geometric deformation, the second feature map representing material spectral properties, and the third feature map representing microscopic texture deviations, as described in this invention, are approximately orthogonal in a physical sense. This orthogonality makes the manifold distribution of data in the feature space clearer. For deep learning models, this means that different categories of defects (such as "black spots" vs. "black pits") already possess extremely high separability at the input end, effectively avoiding gradient confusion generated during the training process of deep learning models.

[0032] 3. This invention reduces and compresses high-dimensional photometric 3D data (4 or 8 Raw images) into a standard three-channel tensor, which can seamlessly adapt to existing pre-trained weights. This means that this invention can directly utilize the general edge detection and texture extraction capabilities of deep learning models, and quickly transform general visual capabilities into industrial defect detection capabilities in specific fields through transfer learning, avoiding the huge computational costs of redesigning the network structure or training from scratch. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a schematic diagram of the detection method steps for detecting tire defects provided by the present invention;

[0035] Figure 2 This is a top view of the tire defect detection provided by the present invention.

[0036] The diagram includes:

[0037] 1. First time-frequency flash source; 2. Second time-frequency flash source; 3. Third time-frequency flash source; 4. Fourth time-frequency flash source; 5. Line scan camera; 6. Tire. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are one embodiment of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] Example 1

[0040] Please refer to Figures 1 to 2 This invention provides a detection method for detecting tire defects.

[0041] Step S1: As Figure 2 As shown, a tire is placed at the bottom, and four time-frequency flash sources are provided at the top. The tire is illuminated by these flash sources, and a line scan camera is used in conjunction with the flash sources to collect the light reflected from the tire. The flash sources are arranged in an X-shape, with the four flash sources distributed at the four corners of the tire: upper left, lower left, lower right, and upper right. The line scan camera captures four original images of the tire illuminated by the flash sources, obtaining a sequence of original images of the tire under four different illumination angles.

[0042] Furthermore, the time-division frequency flash source can output light of multiple colors, or it can output only white light to illuminate the tire; the time-division frequency flash source illuminates the tire in sequence, and the line scan camera collects the reflected light from the tire in sequence.

[0043] Step S2: Based on the photometric stereo algorithm, using the preset light source direction matrix L, calculate the normal vector data N and albedo data ρ of the target surface according to the original image sequence;

[0044] In step S2, the light source direction matrix L is defined as 4 rows × 3 columns, with each row representing the direction vector of a light source;

[0045] The light source direction matrix L is as follows:

[0046]

[0047] Use the symbol T to represent the light source tilt angle (Slant) and the symbol P to represent the light source azimuth angle (Azimuth).

[0048] Using the Lambert reflection model in the photometric stereo algorithm, the normal vector field (Normal Map) and albedo map of the target surface are calculated.

[0049] The light source tilt angle T is set as a modifiable variable in the program and adjusted according to the actual placement of the light source. In the same photometric stereo algorithm, the light source tilt angles in all four directions remain consistent; the value range of the light source tilt angle T is between 0° and 90°; the recommended value in this embodiment is preferably between 45° and 60°. In this embodiment, 60° is recommended to obtain the best shadow effect.

[0050] In this embodiment, four light sources are provided, which corresponds to four light source azimuth angles. Specifically, the light source azimuth angle P includes a first light source azimuth angle P1, a second light source azimuth angle P2, a third light source azimuth angle P3, and a fourth light source azimuth angle P4. The first light source azimuth angle P1, the second light source azimuth angle P2, the third light source azimuth angle P3, and the fourth light source azimuth angle P4 represent the azimuth angles of the four light sources. In this embodiment, the azimuth angles are fixed as follows: P1=45°, P2=135°, P3=225°, and P4=315°.

[0051] The meaning of the negative sign: The second column (Y component) of the matrix is ​​"-sinTsinP"; the positive direction of the coordinate system defined by the system is: X is positive when the image is to the right, Y is positive when the image is downward, and Z is positive when the image is perpendicular to the tire surface upward. This direction setting is a conventional setting in the field of machine vision.

[0052] Correspondence: The order in which the time-division frequency flash source is lit and the order in which the line scan camera collects data must strictly correspond one-to-one with the row order of matrix L, from the first light source azimuth angle P1, the second light source azimuth angle P2, the third light source azimuth angle P3 to the fourth light source azimuth angle P4.

[0053] Step S3: Based on the normal vector data N and albedo data ρ, construct a feature image containing three independent physical dimensions, namely: a first feature image representing macroscopic geometric deformation, a second feature image representing the spectral properties of the material, and a third feature image representing microscopic texture deviation.

[0054] In step S3, based on the image brightness I and the light source direction matrix L acquired by the line scan camera, the equation I=L·ρ·N is solved by the least squares method to separate the albedo ρ and the normal vector N; then the first feature map, the second feature map and the third feature map are constructed.

[0055] The method for constructing the first feature map is as follows: after performing large-scale Gaussian smoothing on the X and Y components of the normal vector N data, the gradient modulus is calculated and mapped to the gray space. ;in, For large-scale smoothing, the aim is to extract low-frequency geometric features.

[0056] The first feature map highlights geometric deformation: based on the normal vector field, the gradient modulus is calculated after suppressing noise through low-pass filtering to characterize macroscopic geometric defects such as bulges and pits.

[0057] The method for constructing the second feature map is as follows: convert the albedo data to the Lab color space and calculate the Euclidean distance between each pixel and a preset reference neutral color;

[0058] First, obtain the albedo image ρ{RGB} containing color information or the average image of the original image. Then, using the standard color space conversion algorithm, use "cvtColor" in OpenCV to convert it from the RGB space to the CIE-Lab color space and extract the a component (magenta-green) and b component (yellow-blue).

[0059] Then, the chromatic Euclidean distance of the pixel relative to neutral gray (a=128, b=128) is calculated using the following formula: This eliminates the influence of brightness (L component) and extracts only the color deviation characteristics of the pure material.

[0060] The second feature map highlights material discoloration: based on the albedo map, the chromaticity distance of the pixel in the Lab color space is calculated to characterize material defects such as oil stains and discoloration.

[0061] The method for constructing the third feature map is as follows: calculate the absolute value of the difference between the original normal vector and the smoothed normal vector to remove the macroscopic geometric curvature background of the target object surface and retain high-frequency texture features; The difference between the original normal and the smoothed normal is calculated as described above, which is equivalent to a frequency domain high-pass filter; the macroscopic curvature of the tire is removed while the microscopic texture is preserved.

[0062] The third feature map highlights micro-textures: the normal vector field is subjected to high-pass filtering or differential processing to remove the macro-curvature background, extract high-frequency deviation values, and characterize micro-textures such as embossing and scratches.

[0063] Step S4: Combine the first feature map, the second feature map, and the third feature map into multi-channel image data, and use it as input data for deep learning model training or inference to output whether the tire has defects and the type of defects.

[0064] In step S4, the first feature map, the second feature map, and the third feature map are merged into a three-channel tensor and input into a deep learning model for defect identification; the deep learning model is a convolutional neural network (CNN) and a visual Transformer (ViT).

[0065] This invention analyzes the principles and advantages of adaptability in deep learning model training:

[0066] The three-channel image construction method based on photometric stereo feature decoupling proposed in this invention is not a simple image enhancement technique, but rather a specialized design tailored to the training characteristics of deep learning models such as Convolutional Neural Networks (CNNs) and Visual Transformers (ViTs). Its rationale and core advantages in model training are detailed below:

[0067] 1. Introduction of "inductive bias" at the physical level

[0068] In traditional machine vision deep learning, models typically require a large amount of parameters and data to learn how to separate "lighting," "material," and "shape" information from RGB images. This invention, through a photometric stereo algorithm, completes physical-level feature decoupling before data is input into the model.

[0069] Reason for molding: The surface defects of rubber products (tire) (such as bulges) and background interference (such as ink) are often non-linearly entangled in pixel space.

[0070] Advantages: By "hard-coding" this prior physical knowledge into the input data, we are essentially introducing a strong inductive bias into the model. This eliminates the need for the model to learn the physical laws of lighting from scratch, significantly reducing the complexity of the hypothesis space and enabling rapid convergence even with small sample sizes.

[0071] 2. Characteristic orthogonality and dimensionality reduction of "manifolds"

[0072] The first, second, and third feature maps (macroscopic gradient, chromaticity, and texture deviation) constructed in this invention are approximately orthogonal in a physical sense. The first feature map only responds to geometric deformation; the second feature map only responds to spectral reflectance; and the third feature map only responds to high-frequency texture components.

[0073] This orthogonality makes the manifold distribution of data in the feature space clearer. For deep learning models, this means that different categories of defects (such as "black spots" vs. "black pits") already have high separability at the input, effectively avoiding gradient confusion that occurs during model training.

[0074] 3. Perfect ecosystem compatibility with pre-trained large models

[0075] Current mainstream large-scale vision models (such as ResNet-50, ViT-Base, EfficientNet, etc.) are all pre-trained based on three-channel (RGB) images from the ImageNet dataset.

[0076] Advantages: This invention reduces and compresses high-dimensional photometric 3D data (4 or 8 raw images) into a standard three-channel tensor, which can seamlessly adapt to existing pre-trained weights. This means that this invention can directly utilize the general edge detection and texture extraction capabilities of deep learning models, and quickly transform general visual capabilities into specific domain-specific industrial defect detection capabilities through transfer learning, avoiding the huge computational costs of redesigning the network structure or training from scratch.

[0077] 4. Robustness for Long-tail Defects

[0078] In industrial settings, defect samples often exhibit a long-tailed distribution (i.e., there are many normal samples and few abnormal samples, which are also mixed).

[0079] Advantages: The "de-skewing" and "deviation map" (third feature map) design in this invention is essentially a form of background suppression based on physical rules. For unsupervised anomaly detection models (such as reconstruction-based AutoEncoder or feature embedding-based PatchCore), this channel provides an extremely clean background distribution. The model only needs to learn "smooth gradients" and "uniform chromaticity" to be considered normal; any abrupt signal (whether it's a never-before-seen scratch or foreign object) will generate a huge distance projection in the feature space, thus significantly improving the detection rate of unseen defects.

[0080] Example 2

[0081] Based on the same inventive concept, Embodiment 2 of this application also provides a detection system for detecting tire defects, the system comprising:

[0082] Image acquisition module: The image acquisition module is used to control the time-division frequency flash source and the line scan camera. The time-division frequency flash source illuminates the tire, and the line scan camera acquires the reflected light from the tire to form an image.

[0083] Processor: The processor is connected to the image acquisition module and is used to execute the detection method for detecting tire defects described in Embodiment 1; and to process the acquired image data and output the processed results;

[0084] Memory: The memory is connected to the processor and is used to store the computer program for detecting tire defects, the images acquired by the line scan camera, and the processed results.

[0085] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A detection method for detecting defects of a tire, characterized by: Includes the following steps; Step S1: Obtain the original image sequence of the tire under four different lighting angles; Step S2: Based on the photometric stereo algorithm, using the preset light source direction matrix L, calculate the normal vector data N and albedo data ρ of the target surface according to the original image sequence; Step S3: Based on the normal vector data N and albedo data ρ, construct a feature image containing three independent physical dimensions, namely: a first feature image representing macroscopic geometric deformation, a second feature image representing the spectral properties of the material, and a third feature image representing microscopic texture deviation. Step S4: Combine the first feature map, the second feature map, and the third feature map into multi-channel image data, and use it as input data for deep learning model training or inference to output whether the tire has defects and the type of defects.

2. The detection method for detecting tire defects according to claim 1, characterized in that: In step S1, a time-frequency flash source is used to illuminate the tire, and a line scan camera is used in conjunction with the time-frequency flash source to collect the light reflected from the tire. The time-frequency flash source adopts an X-shaped layout, with four time-frequency flash sources distributed at the four corners of the tire: the upper left corner, the lower left corner, the lower right corner, and the upper right corner. The line scan camera collects four original images of the tire illuminated by the time-frequency flash source.

3. The detection method for detecting tire defects according to claim 2, characterized in that: In step S2, the light source direction matrix L is defined as 4 rows × 3 columns, with each row representing the direction vector of a light source; the symbol T represents the tilt angle of the light source, and the symbol P represents the azimuth angle of the light source. The light source direction matrix L is as follows: The light source tilt angle T is a modifiable variable. In the same photometric stereo algorithm, the light source tilt angles in the four directions remain consistent. The value range of the light source tilt angle T is between 0° and 90°. The light source azimuth angle P includes a first light source azimuth angle P1, a second light source azimuth angle P2, a third light source azimuth angle P3, and a fourth light source azimuth angle P4. The first light source azimuth angle P1, the second light source azimuth angle P2, the third light source azimuth angle P3, and the fourth light source azimuth angle P4 represent the azimuth angles of the light sources in the four directions, respectively. The meaning of the negative sign: The second column (Y component) of the matrix is: "-sinTsinP"; The positive direction of the coordinate system defined by the system is: X is positive when the image is to the right, Y is positive when the image is downward, and Z is positive when the image is perpendicular to the tire surface upward. This direction setting is a conventional setting in the field of machine vision. Correspondence: The order in which the time-division frequency flash source is lit and the order in which the line scan camera collects data must strictly correspond one-to-one with the row order of matrix L, from the first light source azimuth angle P1, the second light source azimuth angle P2, the third light source azimuth angle P3 to the fourth light source azimuth angle P4.

4. The detection method for detecting tire defects according to claim 3, characterized in that: The tilt angle T of the light source is 60°; the azimuth angles are fixed as follows: P1=45°, P2=135°, P3=225° and P4=315°.

5. The detection method for detecting tire defects according to claim 1, characterized in that: In step S3, based on the image brightness I and the light source direction matrix L acquired by the line scan camera, the equation I=L·ρ·N is solved by the least squares method to separate the albedo ρ and the normal vector N; then the first feature map, the second feature map and the third feature map are constructed.

6. The detection method for detecting tire defects according to claim 5, characterized in that: The method for constructing the first feature map is as follows: after performing large-scale Gaussian smoothing on the X and Y components of the normal vector N data, the gradient modulus is calculated and mapped to the gray space. ;in, For large-scale smoothing, the aim is to extract low-frequency geometric features.

7. A detection method for detecting tire defects according to claim 5, characterized in that: The method for constructing the second feature map is as follows: convert the albedo data to the Lab color space and calculate the Euclidean distance between each pixel and a preset reference neutral color; First, obtain the albedo image ρ{RGB} containing color information or the average image of the original image. Then, using the standard color space conversion algorithm, use "cvtColor" in OpenCV to convert it from the RGB space to the CIE-Lab color space and extract the a component (magenta-green) and b component (yellow-blue). Then, the chromatic Euclidean distance of the pixel relative to neutral gray (a=128, b=128) is calculated using the following formula: This eliminates the influence of brightness (L component) and extracts only the color deviation characteristics of the pure material.

8. A method for detecting tire defects according to claim 5, characterized in that: The method for constructing the third feature map is as follows: calculate the absolute value of the difference between the original normal vector and the smoothed normal vector to remove the macroscopic geometric curvature background of the target object surface and retain high-frequency texture features; The difference between the original normal and the smoothed normal is calculated as described above, which is equivalent to a frequency domain high-pass filter; the macroscopic curvature of the tire is removed while the microscopic texture is preserved.

9. A method for detecting tire defects according to claim 1, characterized in that: In step S4, the first feature map, the second feature map, and the third feature map are merged into a three-channel tensor and input into a deep learning model for defect identification; the deep learning model is a convolutional neural network (CNN) and a visual Transformer (ViT).

10. A detection system for detecting tire defects, characterized in that: include: Image acquisition module: The image acquisition module is used to control the time-division frequency flash source and the line scan camera, and to acquire images using the line scan camera; Processor: The processor is connected to the image acquisition module and is used to execute the detection method for detecting tire defects as described in any one of claims 1 to 9; and to process the acquired image data and output the processed results; Memory: The memory is connected to the processor and is used to store the computer program for detecting tire defects, the images acquired by the line scan camera, and the processed results.

Images