Tire front and back detection method and system

Abstract

The invention discloses a tire front and back detection method, which is realized by adopting a staged-multi-branch hybrid deep learning architecture, and specifically comprises the following steps: in the first stage, positioning a tire region, processing an original image by adopting a lightweight target detection network, and outputting a tread sub-image ROIT and a sidewall sub-image ROIS; in the second stage, multi-modal feature parallel extraction is carried out, a tread sub-image ROIT is processed by using a pattern direction discrimination sub-network, and the inherent directional features of the tread pattern are obtained; meanwhile, processing the sidewall sub-image ROIS by utilizing a mark identification sub-network to obtain one or more pieces of detected mark information; in the third stage, fusion decision and confidence evaluation are carried out, and a final tire positive and negative judgment result and comprehensive confidence are output; the invention further discloses a tire front and back detection system. The problems that in the prior art, manual visual inspection is relied on, or an existing machine vision scheme is insufficient in robustness and low in accuracy in a complex industrial environment are solved.

Description

Technical Field

[0001] This invention relates to the field of tire inspection technology, and in particular to a method and system for tire front and back inspection. Background Technology

[0002] In automated tire assembly processes, ensuring tires are installed in the correct orientation is crucial. Tire tread patterns have a clear directional design; incorrect installation can lead to reduced drainage and grip, posing safety hazards. Currently, many production lines still rely on manual visual inspection for orientation determination. This method suffers from low efficiency, high labor intensity, susceptibility to subjective factors, and difficulty in integrating with automated processes. Existing machine vision solutions often analyze the entire image, including complex backgrounds, making them susceptible to interference from changes in ambient lighting and equipment obstruction, resulting in insufficient accuracy and robustness. Therefore, there is an urgent need for a technical solution that can accurately and reliably determine tire installation orientation automatically in complex industrial environments. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for detecting the front and back of a tire, so as to solve the problems of insufficient robustness and low accuracy of existing technologies that rely on manual visual inspection or existing machine vision solutions in complex industrial environments.

[0004] To achieve the above objectives, the technical solution adopted by this invention is: a tire front and back detection method, implemented using a phased-multi-branch hybrid deep learning architecture, specifically including:

[0005] The first stage involves locating the tire region and using a lightweight object detection network to process the original image, outputting the tread sub-image ROI_T and the sidewall sub-image ROI_S.

[0006] The second stage involves parallel extraction of multimodal features. The tread sub-image ROI_T is processed using a pattern direction discrimination sub-network to obtain the inherent directional features of the tire pattern. At the same time, the sidewall sub-image ROI_S is processed using a marker recognition sub-network to obtain one or more detected marker information.

[0007] The third stage involves fusion decision-making and confidence assessment, outputting the final tire front / back discrimination result and overall confidence level.

[0008] As a further improvement of the present invention, in the first stage, the target detection network is based on the YOLOv5s model, with its backbone network being a Focus structure and a CSPNet structure, and its neck network being a PANet structure. The size and proportion of the output layer anchor boxes are obtained by pre-clustering analysis based on a large number of tire image datasets. The bounding box coordinates of the tire in the image are output, and the original image is cropped accordingly to obtain the tread sub-image ROI_T and the sidewall sub-image ROI_S, which contain only the tire body and have eliminated background interference.

[0009] As a further improvement of the present invention, in the second stage, the pattern orientation discrimination sub-network adopts MobileNetV3 as the basic backbone network, and a lightweight self-attention module is introduced after the backbone network. The self-attention module first performs channel dimensionality reduction on the input feature map through 1x1 convolution, and then obtains the feature sequence through reconstruction operation, and calculates its query, key, and value vectors; by calculating and normalizing the similarity matrix of query and key, an attention weight map is obtained, and finally this weight is weighted and summed with value; the feature map enhanced by the self-attention module is fed into a global average pooling layer, flattened and connected to a fully connected layer, and finally outputs a binary classification probability distribution, i.e., the probabilities of forward and reverse directions, through the Softmax function.

[0010] As a further improvement of the present invention, in the second stage, the marker recognition sub-network adopts a single-stage target detection network for locating and classifying various deterministic markers in the fetal side image; in the data augmentation stage, Mosaic enhancement is performed for small targets; in the neck network, the fusion of shallow feature maps from the backbone network is strengthened to utilize its higher spatial resolution to improve the detection capability of small targets; for markers with clear directionality, not only their position is labeled in the annotation of the training data, but also their direction angle is labeled, so that the network can learn and output the direction of the directional markers.

[0011] As a further improvement of the present invention, in the third stage, the front and back of the tire are distinguished by constructing a priority rule base, the direction probability P_texture output by the pattern direction discrimination sub-network, and the tag list output by the tag recognition sub-network.

[0012] As a further improvement to the present invention, the priority rule base is specifically as follows:

[0013] If a directional marker is identified, the result is obtained by directly comparing the direction of the directional marker with the preset standard direction. If no directional marker is identified but a production identification mark is identified, the result is determined based on the position of the mark. If no definite marker is identified, the result is determined entirely by the output probability P_texture of the pattern direction discrimination subnetwork.

[0014] As a further improvement of the present invention, in the third stage, the overall confidence level is achieved by assigning dynamic weights to the outputs of the pattern direction discrimination subnetwork and the tag recognition subnetwork; when the overall confidence level is lower than a preset threshold, it is determined to be uncertain, and the re-detection process is automatically triggered or an alarm is raised to the superior MES system for manual intervention.

[0015] This invention also provides a tire front-back detection system, including a vision detection module and an electronic control signal system. The vision detection module includes a photoelectric sensor and an industrial control computer. When a tire is conveyed to the detection station via the feeding power roller line, the photoelectric sensor detects its position and immediately sends a signal to the electronic control signal system. The electronic control signal system sends a command to the industrial control computer to trigger the camera to acquire images. The image is analyzed using the tire front-back detection method described above, which is deployed in the industrial control computer, to determine the tire installation direction. After the detection is completed, the electronic control signal system sends a command to the discharge power roller line to convey the tire to the discharge section.

[0016] The beneficial effects of this invention are:

[0017] 1. Accuracy and robustness: By using a decision-level multimodal feature fusion mechanism, which integrates macroscopic pattern and microscopic marking information, the shortcomings of single features being prone to errors under dirt and lighting changes are effectively overcome. The detection accuracy and environmental robustness are significantly improved compared with manual and traditional vision solutions.

[0018] 2. High efficiency and embedded compatibility: Adopting a customized lightweight network hybrid architecture, it achieves millisecond-level inference speed while ensuring industrial-grade high precision. It can be directly deployed in embedded vision controllers to meet the real-time requirements of high-cycle production lines.

[0019] 3. System Integration and Intelligence: Through multi-camera collaborative data acquisition and dynamic confidence assessment mechanisms, and deep integration with the production line PLC and MES systems, a traceable, optimizable, and OTA remote upgrade-capable adaptive intelligent quality control closed loop is formed, injecting new certainty and intelligence into the production process. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the detection system in an embodiment of the present invention;

[0021] Figure 2 This is a flowchart of the detection method in an embodiment of the present invention.

[0022] Figure label:

[0023] 1. Feeding motorized roller conveyor; 2. Photoelectric sensor; 3. Industrial computer; 4. Industrial camera; 5. Discharge motorized roller conveyor. Detailed Implementation

[0024] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0025] Example 1

[0026] like Figure 1 As shown, a tire forward and reverse detection method and system are described. The tire begins to move as it passes through the feeding motor roller conveyor 1. When the tire reaches the photoelectric sensor 2, the photoelectric sensor 2 sends a signal back to the electronic control signal system. The electronic control signal system then stops the movement of the feeding motor roller conveyor 1 to ensure the tire remains stationary at the detection station, and simultaneously sends an image acquisition command to the industrial control computer 3.

[0027] The software execution flow of the core detection method (deployed and run on industrial PC 3) is as follows:

[0028] After receiving instructions from the electronic control signal system, the industrial computer 3 controls the industrial cameras 4 installed above and to the side of the inspection station to simultaneously acquire images of the tire tread and sidewall. The acquired images are immediately sent to the vision inspection algorithm deployed in the industrial computer 3 for processing. The execution flow of this algorithm includes the following steps:

[0029] A. Precise tire area positioning:

[0030] The industrial computer 3 uses a pre-trained lightweight object detection network (such as YOLOv5s) to analyze the raw images captured by the camera. This network can quickly and accurately locate the tire in the image and output its bounding box coordinates. The system then uses these coordinates to crop out a tread sub-image (denoted as ROI_T) and a sidewall sub-image (denoted as ROI_S) containing only the tire from the raw image. The core function of this step is to actively eliminate interference from the complex background of the production line through object detection technology, providing a clean and high-quality input area for subsequent high-precision analysis, thereby significantly improving the accuracy and stability of subsequent steps.

[0031] B. Parallel extraction of multimodal features:

[0032] This stage is divided into two parallel branches, processing the macroscopic directional features of the tread pattern and the microscopic deterministic features of the sidewall markings, respectively. B1. Tread Pattern Direction Discrimination Sub-network: The cropped tread sub-image ROI_T is input into this sub-network. This network uses the lightweight MobileNetV3 as its backbone for feature extraction, followed by a lightweight self-attention module. This self-attention module enables the network to automatically calculate the importance of different regions in the image, and is particularly adept at capturing and strengthening long-range dependent features representing the global direction of the tire tread pattern (such as continuous V-shaped or wavy patterns), while suppressing interference from irrelevant noise such as uneven lighting and localized dirt. Finally, the network outputs the probability value P_texture and its confidence score for the tire direction being "positive" or "negative" through a classification layer. B2. Marker Recognition Sub-network: The cropped sidewall sub-image ROI_S is input into this sub-network. This network is based on the YOLOv5s architecture and is optimized for the characteristic that sidewall markings (such as mounting direction arrows, DOT characters, etc.) are usually small targets. Optimization measures include: using Mosaic data augmentation to improve the model's sensitivity to small targets; enhancing the weights of shallow feature maps in the feature fusion network (Neck) to utilize their higher spatial resolution for accurate localization of small targets; and labeling the direction angles of key markers such as directional arrows during training so that the network can directly output the arrow's pointing information. This sub-network ultimately outputs the category, location, and angle information of all identified markers, as well as the directional arrows (if present).

[0033] C. Fusion decision-making and confidence assessment:

[0034] The fusion decision center within the industrial control computer 3 receives the analysis results from the two sub-networks mentioned above. The decision center performs a fusion judgment based on preset priority rules: the marker recognition result is given higher weight due to its objectivity. For example, if an "installation direction arrow" is clearly identified, its orientation is directly determined; if no arrow is identified but a "DOT" code is identified, the judgment is made according to its preset position rules; if no definite marker is identified, the judgment result P_texture from the pattern direction sub-network is adopted. Simultaneously, the system calculates the overall confidence score Score_final for this detection (e.g., a weighted average of the output confidence scores of the two sub-networks).

[0035] Finally, the industrial control computer 3 sends the judgment result (forward / reverse) and the overall confidence level to the electrical control signal system. If the result is "forward" or "reverse" and the overall confidence level is higher than the preset threshold (e.g., 0.85), the electrical control signal system controls the discharge power roller conveyor 5 to transport the tire to the qualified product flow direction; if the overall confidence level is lower than the threshold, it is judged as "uncertain", and the electrical control signal system will trigger a re-detection process (e.g., control the roller conveyor to move slightly and then trigger the detection again) or send an alarm to the upper-level MES system to request manual intervention, thereby ensuring the reliability of the final output of the system under complex working conditions.

[0036] Example 2

[0037] A tire front and back inspection method system is proposed to solve the problems of insufficient robustness and low accuracy of existing technologies that rely on manual visual inspection or existing machine vision solutions in complex industrial environments.

[0038] 1. System Composition: The entire system mainly consists of four parts: the feeding motorized roller conveyor, the vision inspection module, the discharging motorized roller conveyor, and the electrical control signal system. The feeding and discharging motorized roller conveyors are jointly responsible for the automatic conveying of tires.

[0039] The vision inspection module is the core of the system, comprising photoelectric sensors, an industrial camera, a lens, a light source, and an industrial control computer (host computer). Its workflow is as follows: When a tire is conveyed to the inspection station via the feeding roller conveyor, the photoelectric sensor detects its position and immediately sends a signal to the electronic control signal system. The electronic control signal system then sends a command to the industrial control computer to trigger the camera to acquire an image. The vision inspection algorithm analyzes the image to determine the tire's installation orientation. After inspection, the electronic control signal system sends a command to the actuator, and the tire is then conveyed to the discharge section.

[0040] The entire process is coordinated and controlled by an electronic signal system. This system uses photoelectric sensors as detection trigger units, a vision-based industrial control computer as the control core, responsible for issuing commands and triggering image acquisition, and professional vision inspection software to complete the final recognition and judgment. The three work closely together to form a complete automated detection closed loop.

[0041] 2. Working Principle: The core technical solution of this embodiment lies in a multimodal fusion online tire inspection method and system for industrial production lines. This system simultaneously acquires images of the tire tread and sidewall using multiple industrial cameras deployed at the inspection station.

[0042] At the algorithm level, a phased-multi-branch hybrid deep learning architecture is adopted, such as... Figure 2 As shown, the specific process is as follows:

[0043] Phase 1: Precise tire area positioning:

[0044] Input: Raw images with complex backgrounds captured by industrial cameras. Network and Functionality: A lightweight object detection network, an improved YOLOv5s model, is employed. The backbone uses a Focus structure with CSPNet (Cross Stage Partial Networks) to improve speed and feature reuse capabilities, while the neck uses a PANet (Path Aggregation Network) structure to fuse multi-scale features. The model's output layer is customized, with anchor box sizes and proportions pre-clustered based on a large dataset of tire images to ensure high recall of the tire itself.

[0045] Output: The network outputs the bounding box coordinates of the tire in the image. Based on this, the system crops the original image to obtain a tread sub-image (ROI_T) and a sidewall sub-image (ROI_S) containing only the tire body with greatly reduced background interference, which serve as the input for the second stage.

[0046] Second stage: Parallel extraction of multimodal features:

[0047] This stage consists of two parallel, functionally specialized subnetworks, which process the tread sub-image and lateral sub-image obtained from the first stage, respectively.

[0048] (1) Pattern direction discrimination subnetwork:

[0049] Input: Cropped tire tread image ROI_T.

[0050] Network structure:

[0051] 1. Feature Extraction Backbone: MobileNetV3-Large is used as the basic backbone network, leveraging its lightweight nature of depthwise separable convolutions for fast feature extraction. The final classification layer of the original model is removed, while retaining its feature output capability.

[0052] 2. Lightweight Self-Attention Module: Following the backbone network, a lightweight self-attention module is introduced. This module first performs channel dimensionality reduction on the input feature map using 1x1 convolutions, then reshapes it to obtain the feature sequence, calculating its query, key, and value vectors. By calculating and normalizing the similarity matrix of the query and key, an attention weight map is obtained. Finally, this weight is weighted and summed with the value. This mechanism enables the network to autonomously focus on long-range dependent features representing the macroscopic direction of patterns in the image (such as continuous V-shaped or wavy patterns) while suppressing the interference of local noise.

[0053] 3. Classification Head: The feature map enhanced by the self-attention module is fed into the global average pooling layer, flattened and connected to a fully connected layer, and finally outputs a binary classification probability distribution, i.e. (forward, backward) probability, through the Softmax function.

[0054] Functionality: This sub-network learns the inherent directional features of tire tread patterns through end-to-end training. For example, for asymmetrical tread patterns, the tilt angle and depth distribution of the grooves have a specific front-to-back relationship; for symmetrical tread patterns, the small sipes or wear indicators also possess directionality. The network achieves tread pattern direction discrimination by encoding these features and focusing through a self-attention mechanism.

[0055] (2) Label recognition subnetwork:

[0056] Input: Cropped tire sidewall image ROI_S.

[0057] Network structure:

[0058] 1. Feature Extraction and Detection: A single-stage object detection network, such as YOLOv5s or a customized version, is employed. This network simultaneously locates and classifies various deterministic features in the tire side image (such as the "DOT" character, triangular certification marks, installation direction arrows, brand logos, etc.).

[0059] 2. Customized Design: The network was optimized to address the characteristic that tire side markers are typically small targets: a. Mosaic enhancement for small targets was added during the data augmentation stage; b. In the neck network, the fusion of shallow feature maps from the backbone network was strengthened to leverage its higher spatial resolution and improve small target detection capabilities; c. For markers with clear directionality, such as "installation direction arrows," not only their location but also their direction angle were annotated in the training data, enabling the network to learn and output the arrow's direction.

[0060] Functionality: This sub-network outputs one or more detected marker information, including: the marker category (e.g., "DOT", "Arrow"), bounding box position, and the orientation angle for the "Arrow" category. This information provides a highly deterministic basis for determining whether a marker is positive or negative.

[0061] Phase Three: Integrated Decision Making and Confidence Assessment

[0062] Input: The direction probability P_texture output by the pattern direction discrimination subnetwork, and the list of tags output by the tag recognition subnetwork (including category, position, direction, etc.).

[0063] Integrated decision-making mechanism:

[0064] 1. Rule Engine: The system pre-defines a priority rule base. Tag recognition results are assigned higher priority due to their objectivity and determinism. For example:

[0065] If an "installation direction arrow" is detected, the arrow's direction is directly compared with the preset standard direction to obtain the final result.

[0066] If no arrow is detected but production markings such as "DOT" are detected, the determination is made based on its position (e.g., usually facing outwards towards the rim).

[0067] If no deterministic marker is identified, the judgment is made entirely based on the output probability P_texture of the pattern orientation discrimination subnetwork.

[0068] 2. Weighted Confidence Fusion: In more complex fusion strategies, dynamic weights can be assigned to the results of the two sub-networks. For example, when the marker recognition sub-network detects an arrow with high confidence, its weight W_mark is set to 0.9, and the weight W_texture of the pattern orientation network is set to 0.1. The final combined confidence score Score_final = W_mark * Confidence_mark + W_texture * Confidence_texture.

[0069] Output and Processing: The system outputs the final judgment result (positive / negative) and the overall confidence score Score_final. When Score_final is lower than the preset threshold (e.g., 0.85), it is judged as "uncertain". The system can automatically trigger the re-detection process (e.g., let the roller conveyor move back and forth once to re-trigger the photo) or alarm the upper-level MES system for manual intervention to ensure the reliability of the result.

[0070] At the system level, the aforementioned algorithm is tightly integrated with the electronic control signal system and the production line PLC. Inspection results are transmitted to the PLC in real time, controlling the roller conveyor to automatically sort tires to the correct workstation or rework process. All inspection data and results are uploaded to the MES system for quality traceability and production data analysis, and the entire inspection model supports remote updates and optimization via OTA (Over-The-Air).

[0071] The embodiments described above are merely illustrative of specific implementations of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A tire face detection method characterized by, The implementation is based on a hybrid deep learning architecture with stages and multiple branches, which specifically includes: The first stage is to locate the tire area, and a lightweight target detection network is used to process the original image to output the tread sub-image ROI_T and the sidewall sub-image ROI_S. The second stage is to extract multi-modal features in parallel, and a pattern direction discrimination sub-network is used to process the tread sub-image ROI_T to obtain the intrinsic directional features of the tire pattern; at the same time, a label recognition sub-network is used to process the sidewall sub-image ROI_S to obtain one or more detected label information. The third stage is to make a fusion decision and confidence assessment, and output the final tire forward and reverse discrimination result and comprehensive confidence.

2. The tire face detection method according to claim 1, characterized by, In the first stage, the target detection network is based on the YOLOv5s model, the backbone network is the Focus structure and CSPNet structure, the neck network is the PANet structure, and the size and proportion of the anchor box of the output layer are obtained by pre-clustering analysis based on a large number of tire image datasets. The output is the boundary box coordinates of the tire in the image, based on which the original image is cropped to obtain the tread sub-image ROI_T and the sidewall sub-image ROI_S containing only the main body of the tire and the background interference is eliminated.

3. The tire face detection method according to claim 1, characterized by, In the second stage, the pattern direction discrimination sub-network uses MobileNetV3 as the basic backbone network, and a lightweight self-attention module is introduced after the backbone network. The self-attention module first reduces the channel dimension of the input feature map through 1x1 convolution, then obtains the feature sequence through reconstruction operation, calculates the query Query, key Key, and value Value vectors; by calculating the similarity matrix of Query and Key and normalizing, the attention weight map is obtained, and finally the weight is weighted and summed with Value; the feature map enhanced by the self-attention module is sent to the global average pooling layer, and a fully connected layer is connected after flattening. Finally, a two-class probability distribution is output by the Softmax function, i.e. the probability of forward and reverse.

4. The tire face detection method according to claim 3, characterized by, In the second stage, the label recognition sub-network uses a single-stage target detection network to locate and classify various deterministic marks in the sidewall image; in the data enhancement stage, small targets are enhanced by Mosaic; in the neck network, the fusion of shallow feature maps from the backbone network is strengthened to utilize their higher spatial resolution to improve small target detection capability; for marks with clear directionality, not only their positions but also their direction angles are labeled in the training data annotation, so that the network can learn and output the pointing direction of the directional marks.

5. The tire face detection method according to claim 4, characterized by, In the third stage, the forward and reverse discrimination of the tire is realized by constructing a priority rule library according to the direction probability P_texture output by the pattern direction discrimination sub-network and the label list output by the label recognition sub-network.

6. The tire face detection method according to claim 5, characterized by, The priority rule library is as follows: If the directional mark is identified, the final result is obtained by directly comparing the directional mark with the preset standard direction; if the directional mark is not identified but the production identification mark is identified, the position of the mark is judged; if no certain mark is identified, the output probability P_texture of the pattern direction discrimination sub-network is completely relied on to make the judgment.

7. The tire face detection method according to claim 6, characterized by, In the third stage, the comprehensive confidence is achieved by assigning dynamic weights to the results output by the pattern direction discrimination sub-network and the mark identification sub-network; when the comprehensive confidence is lower than a preset threshold, it is determined as uncertain, and a re-detection process is automatically triggered or an alarm is sent to the upper MES system for manual intervention.

8. A tire face detection system characterized by, The visual detection module includes a photoelectric sensor and an industrial computer, when the tire is conveyed to the detection station by the feeding power roller line, the photoelectric sensor detects its in-place state and immediately sends a signal to the electric control signal system; the electric control signal system issues an instruction to the industrial computer to trigger the camera to collect images, and the images are analyzed by using the tire right-left detection method as claimed in any one of claims 1-7 disposed in the industrial computer to determine the tire installation direction; after the detection is completed, the electric control signal system will issue an instruction to the discharging power roller line to convey the tire to the discharging section.

Images