A non-contact tire deformation recognition method based on fine-tuned large visual model

Abstract

This invention discloses a non-contact tire deformation recognition method based on a fine-tuned large-scale visual model. The method includes calibrating a monocular high-speed camera, fine-tuning the parameters of the mask decoder of the large-scale visual model using a tire image dataset, generating a point coordinate cue sequence based on pixel calculation using OpenCV, inputting the cue sequence into the fine-tuned large-scale model for segmentation, further processing the pixel matrix using OpenCV to generate a point cue sequence, and post-processing using iterative geometric fitting and region growing algorithms to obtain the mechanical deformation parameters of the target sample tire. This invention can achieve accurate and rapid tire deformation recognition, reaching pixel-level precise segmentation. It solves the problems of large errors, limited measurement environments, nonlinear image distortion caused by cameras, weak generalization ability, and high training costs in existing computer vision-based vehicle tire recognition methods.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a non-contact tire deformation recognition method based on a fine-tuned large visual model. Background Technology

[0002] Effective vehicle load identification is a crucial issue that needs to be addressed to ensure safe production and transportation in China. Using machine vision-based non-contact measurement technology—which analyzes tire images to calculate vehicle load weight—provides a new method for vehicle weight identification. Due to its advantages such as low cost, non-contact operation, and ease of setup, this technology has already seen some development.

[0003] However, current non-contact measurement technologies based on machine vision also have certain shortcomings. Images based on high-speed cameras suffer from optical distortion, and image segmentation results are easily affected by lighting conditions and image quality. In addition, due to the limitations of training samples, the generalization ability of the model is weak and can only be adapted to fixed application scenarios.

[0004] With the development of pre-trained large model technology, pre-trained large models capable of practical application in image recognition have emerged. These large models exhibit excellent generalization ability, enabling rapid adaptation to specific application domains through parameter fine-tuning. Therefore, how to fine-tune the parameters of pre-trained large models and leverage the generalization capabilities of artificial intelligence to apply non-contact tire deformation recognition to more scenarios while improving recognition efficiency and accuracy has become a worthwhile research topic. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to address the shortcomings of the prior art by providing a non-contact tire deformation recognition method based on a fine-tuned large visual model. This non-contact tire deformation recognition method based on a fine-tuned large visual model can solve the problems of weak generalization ability and the influence of camera image distortion in existing computer vision-based vehicle tire recognition methods, improve recognition efficiency, reduce model training costs, and achieve pixel-level accurate segmentation.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] A non-contact tire deformation recognition method based on a fine-tuned visual large model includes the following steps.

[0008] Step 1: Camera parameter calibration: Calibrate the parameters of the camera used to acquire tire images.

[0009] Step 2: Fine-tune the large visual model, which includes the following steps:

[0010] Step 2-1: Create an application scenario dataset, which includes the following steps:

[0011] Step 2-1A: Using the camera calibrated in Step 1, acquire side images of different types of tires under different lighting conditions.

[0012] Step 2-1B: Perform distortion correction on each side image using the camera parameters calibrated in Step 1.

[0013] Step 2-1C: Label the tire and wheel hub areas for each side image after distortion correction to form an application scenario dataset.

[0014] Step 2-2, Fine-tuning: The large visual model includes an image decoder, a cue encoder, and a mask decoder; freeze the parameter weights of the image encoder and the cue encoder; based on the application scenario dataset created in Step 2-1, transfer training is performed on the parameter weights of the mask decoder to obtain the fine-tuned large tire visual model.

[0015] Step 3: Obtain the actual tire image: Using the camera calibrated in Step 1, acquire the side image of the tire to be tested; and use the camera parameters calibrated in Step 1 to perform distortion correction on the acquired side image of the tire to be tested to obtain the actual tire image.

[0016] Step 4: Generate a point cue sequence: Based on the geometric features of the tire to be detected and the image contour extraction algorithm, obtain the coordinates of the center point, N outer offset points, and N inner offset points of the tire to be detected; where N≥3; the outer offset points are located on the tire to be detected, and the inner offset points are located on the rim of the tire to be detected; the center point, the N outer offset points, and the N inner offset points together form a point cue sequence.

[0017] Step 5, Tire Segmentation: Input some or all of the point cue sequences from Step 4 into the large visual tire model obtained in Step 2, and segment the actual tire image obtained in Step 3 to obtain the actual tire mask region.

[0018] Step 6: Obtain tire deformation parameters: Post-process the tire mask area obtained in Step 5 using geometric fitting based on OpenCV to obtain the tire circumcircle and the line connecting it to the ground, thereby calculating the tire deformation parameters; the deformation of the tire to be detected is identified by the tire deformation parameters.

[0019] In step 2-1C, the tire and wheel hub areas are marked separately for each side image after distortion correction using the lableme tool.

[0020] Step 2, the method for training the mask decoder parameter weight transfer, includes the following steps:

[0021] Step 2-2A, Downsampling: Downsample each side image in the application scenario dataset created in Step 2-1;

[0022] Step 2-2B: Divide the dataset: The downsampled application scenario dataset is preferably randomly divided into training set, test set and validation set in a ratio of 7:2:1.

[0023] Step 2-2C: Generate the point cue sequence of the sample tire: Based on the geometric features of the sample tire and the image contour extraction algorithm, obtain the center point coordinates, N outer offset point coordinates, and N inner offset point coordinates of the sample tire; where N≥3; the outer offset points are located on the tire of the sample tire, and the inner offset points are located on the rim of the sample tire; the center point coordinates, N outer offset point coordinates, and N inner offset point coordinates of the sample tire together form the point cue sequence of the sample tire.

[0024] Step 2-2D, Training: Using the application scenario dataset in Step 2-2B, the true tire mask and true wheel hub mask of the sample tires obtained in Step 2-1C, and the sample tire dot cue sequence generated in Step 2-2C, the parameter weights of the mask decoder are trained to obtain the fine-tuned large tire visual model.

[0025] In step 2-2A, the cv2.pyrDown function is used to downsample each side image in the application scenario dataset created in step 2-1 twice.

[0026] In step 2-2C, cv2.findContours is used to calculate the continuous edge contour of the outermost edge of the tire. The outer contour with the largest enclosing area is selected, and cv2.boundingRect is used to construct the bounding rectangle. The coordinates (x, y) of the upper left corner of the bounding rectangle, as well as the width w and height h of the bounding rectangle, are obtained. The coordinates of the center point of the outer contour are calculated using the geometric features of the sample tire. N = 4 outer offset point coordinates and N = 4 inner offset point coordinates are calculated. The 4 outer offset points are all 90° apart on the sample tire. The 4 inner offset points are all 90° apart on the sample tire's rim.

[0027] The formula for calculating the coordinates of the center point (centerX, centerY) is:

[0028]

[0029]

[0030] The four outer offset points are the left outer offset point, right outer offset point, lower outer offset point, and upper outer offset point, with corresponding coordinates of Left_outer(x,y), Right_outer(x,y), Bottom_outer(x,y), and Top_outer(x,y), respectively; the calculation formulas are as follows:

[0031] Leftouter (x,y) =(a*centerX,centerY)

[0032] Right_outer(x,y)=(b*centerX,centerY)

[0033] Bottom_outer(x,y)=(centerX,a*centerY)

[0034] Top_outer(x,y)=(centerX,b*centerY)

[0035] Where 0.1≤a≤0.5, 1.5≤b≤2.

[0036] The four inner offset points are the left inner offset point, right inner offset point, lower inner offset point, and upper inner offset point, with corresponding coordinates of Left_inner(x,y), bottom_inner(x,y), bottom_inner(x,y), and top_inner(x,y); the calculation formulas are as follows:

[0037] Left_inner(x,y)=(c*centerX,centerY)

[0038] Rigtht_inner(x,y)=(d*centerX,centerY)

[0039] Bottom_inner(x,y)=(centerX,c*centerY)

[0040] Top_inner(x,y)=(centerX,d*centerY)

[0041] Where 0.6≤c≤1.0, 1.0≤d≤1.5.

[0042] In step 5, the actual tire masking area includes the tire masking area and the wheel hub masking area. The segmentation method for the tire masking area involves inputting the following elements in the following order: first, inputting the coordinates of the center prompt point to obtain the masking area Mask1 with the highest score; then, inputting the masking area Mask1 and the coordinates of the four outer offset points to obtain and output the masking area Mask2 with the highest score.

[0043] The method for segmenting the hub mask region involves inputting elements in the following order: First, input the coordinates of the center prompt point to obtain the mask region Mask3 with the highest score; then, input the mask region Mask1 and the coordinates of the four inner offset points to obtain and output the mask Mask4 with the highest score.

[0044] In step 6, the tire deformation parameters include length-type data and area-type data; among which, the length-type data includes the actual length l of the tire-ground contact pixel. true The actual radius of the tire's maximum pixel size is CircleR. true The actual pixel distance h from the center of the tire to the ground true Area type data includes the actual area S of the tire's maximum pixel size. 1true and the actual area of ​​tire compression ΔS true .

[0045] The method for obtaining tire deformation parameters includes the following steps:

[0046] Step 6-1: Obtain the tire circumcircle: The tire circumcircle includes the tire's outer circumcircle and the wheel rim's outer circumcircle. The method for obtaining the tire circumcircle includes the following steps.

[0047] Step 6-1A: Draw the outer contour of Mask2 and extract the outer contour points of the tire body; draw the outer contour of Mask4 and extract the outer contour points of the wheel hub.

[0048] Step 6-1B: Fit the outer contour points of the tire to obtain the outer circumcircle of the tire, the coordinates of the center point of the outer circumcircle of the tire, and the maximum pixel radius of the tire, CircleR.

[0049] Step 6-1C: Fit the outer contour points of the wheel hub to obtain the outer circle of the wheel hub and the pixel radius r of the wheel hub.

[0050] Step 6-2: Obtain the ground wiring, including the following steps.

[0051] Step 6-2A, Tire-Ground Contact Segmentation: Introduce the scaling parameter topk, and extract the topk edge points of the tire mask area in a bottom-up sorting manner; where topk is the ratio of the number of edge contour points of the tire close to the ground to the total number of pixels in the tire mask area.

[0052] Step 6-2B: Use the RANSAC algorithm to fit the straight line information of the edge points obtained in Step 6-2A to obtain the dividing line of the tire-ground contact and the intersection of the dividing line and the tire's circumcircle, and then obtain the pixel length L of the dividing line of the tire-ground contact.

[0053] Step 6-2C: Select the smallest x-coordinate among all edge points in Step 6-2A. min and the maximum x-coordinate max Project the corresponding edge points onto the segmentation line obtained in step 6-2B, and calculate the pixel length l of the tire contacting the ground.

[0054] Step 6-2D: Find the lowest point of the contour directly opposite the center of the tire's circumcircle in the tire mask area coordinates, calculate the distance from the center of the tire's circumcircle to the lowest point of the contour, and obtain the pixel distance h from the center of the tire to the ground.

[0055] Step 6-2E: Calculate the tire compression area ΔS. The calculation formula is:

[0056] ΔS=S1-S2

[0057] In the formula, the maximum pixel area S1 of the tire is calculated using the maximum pixel radius CircleR of the tire.

[0058] The area S2 of the tire mask region is calculated using cv2.contourArea for both the tire mask region and the wheel hub mask region.

[0059] Step 6-3: Introduce the tire deformation ratio factor μ.

[0060] Step 6-4: Calculate tire deformation parameters: Using the tire deformation scaling factor μ from Step 6-3, correct the tire-ground contact pixel length l, tire maximum pixel radius CircleR, tire center-to-ground pixel distance h, tire maximum pixel area S1, and tire compression area ΔS obtained in Steps 6-1 and 6-2 to obtain the tire deformation parameters. For length-type data, the original data needs to be multiplied by the tire deformation scaling factor μ during correction; for area-type data, the original data needs to be multiplied by the square of the tire deformation scaling factor μ during correction.

[0061] In step 6-3, the specific formula for calculating the tire deformation ratio factor μ is as follows:

[0062]

[0063] In the formula, r Rim This is the actual radius of the wheel hub, which can be obtained from the characters marked on the tire.

[0064] The present invention has the following beneficial effects:

[0065] 1. This invention corrects distortion in monocular camera images, effectively correcting measurement errors.

[0066] 2. The non-contact tire deformation recognition method based on a fine-tuned large-scale visual model of the present invention applies a mature large-scale visual model trained with 11 million images and 1 billion masks for segmentation. Transfer training was performed specifically on the tire dataset, and the mask decoder parameters of the large-scale visual model were fine-tuned, resulting in higher segmentation accuracy for tires and rims. This enables accurate segmentation of tire images under various lighting conditions. In practical applications, the large-scale model effectively enhances the recognition capability for tire and rim images and achieves better segmentation training results with fewer training images, significantly reducing model training costs.

[0067] 3. The non-contact tire deformation recognition method based on a fine-tuned large visual model of the present invention uses pixel operations based on OpenCV to determine the optimal input order for generating point sequences for tire segmentation and for inputting prompt sequences into the large model. This enables batch processing of tire images and rapid generation of point sequences. The present invention offers higher efficiency in prompt sequence generation and has a wider range of applications. Attached Figure Description

[0068] Figure 1 This is a flowchart of a non-contact tire deformation recognition method based on a fine-tuned visual large model according to an embodiment of the present invention.

[0069] Figure 2 This is the SAM large model structure used in this invention.

[0070] Figure 3 The coordinates of the center point in the tire image.

[0071] Figure 4 The input SAM model is masked and four offset cue points are provided.

[0072] Figure 5 The input offset cue point sequence is used to perform tire segmentation with MAsk1 to output the mask region with the highest score.

[0073] Figure 6 This is the segmented tire image.

[0074] Figure 7 The highest-scoring mask region is output by performing hub segmentation on the input center prompt point.

[0075] Figure 8 The input offset cue point sequence is used to perform hub segmentation with MAsk3 to output the mask region with the highest score.

[0076] Figure 9 This is the segmented image of the wheel hub.

[0077] Figure 10 The outline of Mask2, the tire mask area with the highest score.

[0078] Figure 11 This is the outer contour of the tire drawn using cv2.findContours.

[0079] Figure 12 This is the circumcircle of the tire based on the fitting of the tire's outer contour.

[0080] Figure 13 This refers to the outer circle of the tire and the contact line between the tire and the ground. Detailed Implementation

[0081] The present invention will now be described in further detail with reference to the accompanying drawings and specific preferred embodiments.

[0082] In the description of this invention, it should be understood that the terms "left side," "right side," "upper part," "lower part," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. "First," "second," etc., do not indicate the importance of the components, and therefore should not be construed as a limitation of this invention. The specific dimensions used in this embodiment are only for illustrating the technical solution and do not limit the scope of protection of this invention.

[0083] like Figure 1 As shown, a non-contact tire deformation recognition method based on a fine-tuned visual large model includes the following steps.

[0084] Step 1, Camera parameter calibration: The camera used to acquire tire images is calibrated. The specific calibration method preferably includes the following steps.

[0085] Step 1-1: Acquire Calibration Images: Use the camera to be calibrated to acquire no fewer than 30 calibration images (preferably 30) of the calibration board from different angles and distances. The calibration board is preferably a black and white checkerboard pattern composed of squares with sides of 2cm. The calibration board needs to appear multiple times in the corners of the camera's field of view. In this embodiment, the camera to be calibrated is preferably a Nikon D5600 SLR camera.

[0086] Step 1-2, Finding Corner Points: Using an OpenCV-based corner detection algorithm, find the black and white checkerboard corner points in each calibration image acquired in Step 1-1.

[0087] In this embodiment, the OpenCV corner detection algorithm preferably uses the cv2.findChessboardCorners function to extract corners of the image; further, the cv2.cornerSubPix function is used to extract sub-pixel corners to reduce camera calibration errors.

[0088] Steps 1-3: Calculate the camera parameter matrix: Using an OpenCV-based camera calibration algorithm, calculate the camera parameter matrix for the corner points found in Steps 1-2. The camera parameter matrix includes the camera intrinsic parameter coefficient matrix and the camera extrinsic parameter coefficient matrix. In this embodiment, it is preferable to use `cv2.calibrateCamera` to calibrate the camera, recording the camera's intrinsic and extrinsic parameter coefficient matrices; the `cv2.projectPoints` function is used to back-project the three-dimensional coordinate points in space to evaluate the calibration effect and calculate the average pixel error for each image.

[0089] Step 2: Fine-tune the large visual model, which includes the following steps.

[0090] Step 2-1: Create an application scenario dataset, which includes the following steps.

[0091] Step 2-1A: Using the camera calibrated in Step 1, collect side images of different types of sample tires under different lighting conditions. In this embodiment, it is preferable to collect 436 side images.

[0092] Step 2-1B: Perform distortion correction on each side image using the camera parameter matrix calibrated in Step 1. Specifically, input the camera's intrinsic and extrinsic parameter matrices, and use the functions cv2.initUndistortRectifyMap and cv2.remap to correct the distortion of each side image.

[0093] Step 2-1C: For each side image after distortion correction, the tire and wheel hub regions are preferably labeled using the labelme tool to form an application scenario dataset.

[0094] The preferred calibration process for the lableme tool is as follows: starting from the upper left position of the tire or rim area, the target is outlined with closed polygons in a clockwise sequence along the contour to obtain the true tire mask and the true rim mask of the sample tire.

[0095] In this embodiment, there are a total of 436 valid side images (also known as valid sample tire images) in the application scenario dataset. Therefore, after labelme tool calibration, a total of 872 tire true masks and wheel hub true masks are obtained.

[0096] Step 2-2, Fine-tuning

[0097] The visual large model, preferably the SAM visual large model, includes an image decoder, a cue encoder, and a mask decoder, as detailed below. Figure 2 As shown.

[0098] The image decoder described above runs once for each input image, generating an image embedding, which is a mapping representation of the image.

[0099] The aforementioned cue encoder is used to map and encode two types of cue information: point cue sequences and mask sequences, to generate cue embeddings. The point cue sequences and mask sequences can be selectively used and then processed with image embeddings. However, there are also cue-free methods; this case does not limit the cue method used in the invention.

[0100] The mask decoder described above is used to map image embeddings, cue embeddings, and large model network outputs to masks.

[0101] In this invention, the parameter weights of the image encoder and the cue encoder are frozen; based on the application scenario dataset created in step 2-1, the parameter weights of the mask decoder are transferred and trained to obtain a fine-tuned large-scale tire vision model.

[0102] The method for parameter weight transfer training of the above-mentioned mask decoder preferably includes the following steps.

[0103] Step 2-2A, Downsampling: Preferably, the cv2.pyrDown function is used to downsample each side image in the application scenario dataset created in Step 2-1 twice. The purpose of downsampling is twofold: firstly, to remove redundant information and improve training speed; and secondly, to enable the large visual model to be backward compatible with low-resolution images.

[0104] Step 2-2B: Divide the dataset: The downsampled application scenario dataset is preferably randomly divided into training set, test set and validation set in a ratio of 7:2:1.

[0105] Step 2-2C: Generate the point cue sequence of the sample tire: Based on the geometric features of the sample tire and the image contour extraction algorithm, obtain the center point coordinates, N outer offset point coordinates, and N inner offset point coordinates of the sample tire; where N≥3; the outer offset points are located on the tire of the sample tire, and the inner offset points are located on the rim of the sample tire; the center point coordinates, N outer offset point coordinates, and N inner offset point coordinates of the sample tire together form the point cue sequence of the sample tire.

[0106] In this embodiment, cv2.findContours is preferably used to calculate the continuous edge contour of the outermost edge of the tire. The outer contour with the largest enclosing area is selected, and cv2.boundingRect is used to construct the circumscribed rectangle. The coordinates of the upper left corner of the circumscribed rectangle (x, y), as well as the width w and height h of the circumscribed rectangle, are obtained. The coordinates of the center point of the outer contour, N outer offset point coordinates, and N inner offset point coordinates are calculated using the geometric features of the sample tire. In this embodiment, N=4 is preferred, that is, one outer offset point is taken at every 90° interval of the sample tire, for a total of 4 outer offset points; one inner offset point is taken at every 90° interval of the sample tire hub, for a total of 4 inner offset points.

[0107] In this embodiment, since there are 436 valid sample tire images, and each valid sample tire image has 9 prompts in the prompt sequence, there are a total of 3924 prompts in the 436 valid sample tire images.

[0108] The formula for calculating the coordinates (centerX, centerY) of the center point is as follows:

[0109]

[0110]

[0111] The four outer offset points mentioned above are the left outer offset point, right outer offset point, lower outer offset point, and upper outer offset point, with corresponding coordinates of Left_outer(x,y), Right_outer(x,y), Bottom_outer(x,y), and Top_outer(x,y), respectively; the calculation formulas are as follows:

[0112] Leftouter (x,y) =(a*centerX,centerY)

[0113] Rigtht_outer(x,y)=(b*centerX,centerY)

[0114] Bottom_outer(x,y)=(CenterX,a*centerY)

[0115] Top_outer(x,y)=(centerX,b*centerY)

[0116] Wherein, 0.1≤a≤0.5, and in this embodiment a=0.2 is preferred; 1.5≤b≤2, and in this embodiment b=1.8 is preferred, so that the external offset point is located within the tire region for all tire images and most tire images in all examples.

[0117] The four inner offset points mentioned above are the left inner offset point, right inner offset point, lower inner offset point, and upper inner offset point, with corresponding coordinates of Left_inner(x,y), bottom_inner(x,y), bottom_inner(x,y), and top_inner(x,y); the calculation formulas are as follows:

[0118] Left_inner(x,y)=(c*centerX,centerY)

[0119] Rigtht_inner(x,y)=(d*centerX,centerY)

[0120] Bottom_inner(x,y)=(centerX,c*centerY)

[0121] Top_inner(x,y)=(centerX,d*centerY)

[0122] Wherein, 0.6≤c≤1.0, and in this embodiment, c=0.8 is preferred; 1.0≤d≤1.5, and in this embodiment, d=1.2 is preferred, so that the inner offset point is located within the tire region for all tire images and most tire images in all examples.

[0123] Step 2-2D, Training: Using the application scenario dataset from Step 2-2B, the actual tire and rim masks of the sample tires obtained in Step 2-1C, and the sample tire dot cue sequence generated in Step 2-2C, the parameter weights of the mask decoder are trained to obtain a fine-tuned large-scale tire visual model. The learning rate is set to 0.0005, and the number of iterations is set to 300 rounds.

[0124] Steps 2-3: Evaluation of the large visual model of the tire

[0125] The aforementioned large-scale tire vision model employs a linear combination of the focal loss function and the dices loss function during image segmentation.

[0126] A. The focus loss function FL is expressed as:

[0127] FL=-α t (1-p t ) γ log(p t )

[0128] p t >0.5; γ≥0

[0129] Among them, the intermediate value p t for:

[0130]

[0131] y∈{±1}

[0132] In the formula, α t γ is the balance coefficient, p is the focus parameter, p is the probability of identifying a pixel as foreground, and y is the true label of the current pixel.

[0133] B. The block loss function DL is defined as follows:

[0134]

[0135] In the formula, p i Let g be the probability that the i-th pixel is predicted as foreground, and M be the total number of predicted pixels; i Let be the actual mask value of the i-th pixel, and be the input value.

[0136] C. The formula for calculating the total loss function TL is:

[0137] TL = θ × FL + (1 - θ) × DL

[0138] In the formula, θ is the weight coefficient of the focus loss function FL, which is a set value that can be set according to user needs.

[0139] In this embodiment, 1-TL is used as the tire detection evaluation index for the large-scale tire vision model. The large-scale tire vision model is trained for 300 rounds on the test set:

[0140] A. When four outer offset points are input as the sample tire point prompt sequence, the tire detection evaluation index can reach 0.977.

[0141] B. When the input center point and four outer offset points are used as the sample tire point prompt sequence, the tire detection score can reach 0.9961.

[0142] C. When the input center point, four outer offset points, and four inner offset points are used as the sample tire point cue sequence, the score can reach 0.7236, which is a significant improvement over the pre-trained model before transfer training.

[0143] Alternatively, a threshold for tire inspection and evaluation indicators can be set as needed. Only when the actual tire inspection and evaluation indicators of the large-scale tire visual model are not lower than the threshold requirement can subsequent steps 3 and beyond be performed. Otherwise, repeat step 2 until the indicators are not lower than the threshold requirement.

[0144] Step 3: Obtain actual tire images: Using the camera calibrated in Step 1, acquire side images of the tire to be inspected; and perform distortion correction on the acquired side images of the tire to be inspected using the camera parameters calibrated in Step 1 to obtain actual tire images; then, calibrate each actual tire image to obtain the actual tire mask and the actual rim mask.

[0145] Step 4: Generate a point prompt sequence: Based on the geometric features of the tire to be detected and the image contour extraction algorithm, obtain the coordinates of the center point, N outer offset points, and N inner offset points of the tire to be detected; where N ≥ 3; the outer offset points are located on the tire itself, and the inner offset points are located on the rim of the tire; the coordinates of the center point, N outer offset points, and N inner offset points together form the point prompt sequence of the tire to be detected. In this embodiment, N = 4 is preferred, and the coordinate calculation method for each prompt point is detailed in step 2-2C.

[0146] Step 5, Tire Segmentation: Input some or all of the point cue sequences from Step 4 into the large visual tire model obtained in Step 2, and segment the actual tire image obtained in Step 3 to obtain the tire mask region and the wheel hub mask region.

[0147] The method for segmenting the tire mask region involves inputting elements in the following order:

[0148] 1. Input as follows Figure 3 The center cue point (centerX, centerY) shown indicates the mask region Mask1 with the highest score.

[0149] 2. Input as follows Figure 4 The mask region Mask1 and the coordinates of the four outer offset points Left_outer(x,y) are shown.

[0150] Top_outer(x,y), Bottom_outer(x,y), and Top_outer(x,y) yield the following results: Figure 5 The masked region shown is Mask2, which has the highest score.

[0151] 3. Output Mask2. The tire portion segmented by Mask2 is as follows: Figure 6 As shown.

[0152] The method for segmenting the hub mask region involves inputting elements in the following order:

[0153] 1. Input the center prompt point (centerX, centerY) to get the following result: Figure 7 The masked region shown is Mask3, which has the highest score.

[0154] 2. Input the mask region Mask1 and the coordinates of the four inner offset points Left_inner(x,y), Rift_inner,

[0155] Bottom_inner and Top_inner output the mask with the highest score, Mask4, such as Figure 8 As shown.

[0156] 3. Output the wheel hub portion segmented by Mask4, as shown below. Figure 9 As shown.

[0157] Step 6: Obtain tire deformation parameters

[0158] The above tire deformation parameters include the actual length l of the tire-ground contact pixel. true The actual radius of the tire's maximum pixel size is CircleR. true The actual pixel distance h from the center of the tire to the ground true The actual area of ​​the tire's maximum pixel S 1true and the actual area of ​​tire compression ΔS true .

[0159] This invention uses OpenCV-based geometric fitting to post-process the mask region obtained in step 5 to obtain the tire's circumcircle and the line connecting it to the ground, thereby calculating the tire deformation parameters; the deformation of the tire to be detected is identified through the tire deformation parameters.

[0160] The method for obtaining the above-mentioned tire deformation parameters preferably includes the following steps.

[0161] Step 6-1: Obtain the tire circumcircle: The tire circumcircle includes the tire's outer circumcircle and the wheel rim's outer circumcircle. The method for obtaining the tire circumcircle includes the following steps.

[0162] Step 6-1A: Use the cv2.findContours function to draw the outer contour of Mask2, as shown below. Figure 10 As shown, extract the outer contour points of the tire carcass; use the cv2.findContours function to draw the outer contour of Mask4, as shown. Figure 11 As shown, extract the outer contour points of the wheel hub.

[0163] Step 6-1B: Use the cv2.minEnclosingCircle function to fit the outer contour points of the tire to obtain the outer circumcircle of the tire, the coordinates of the center point of the outer circumcircle of the tire (CircleCenterX, CircleCenterY), and the maximum pixel radius of the tire (CircleR).

[0164] Step 6-1C: Use the cv2.minEnclosingCircle function to fit the outer contour points of the wheel hub, as shown below. Figure 12 The diagram shows the circumcircle of the wheel hub and the wheel hub pixel radius r.

[0165] Step 6-2: Obtaining the Ground Line: Since the ground line between the tire bottom and the ground is usually a horizontal line segment containing two sets of information, namely the left and right boundaries of the y and x coordinates in the image, we also divide the detection of the ground line into two stages: ground line detection and ground line boundary localization. The method for obtaining the ground line includes the following steps.

[0166] Step 6-2A, Tire-Ground Contact Segmentation: Introduce a scaling parameter `topk`, and extract the top `topk` edge points of the tire mask area in a bottom-up sorting manner. Here, `topk` is the ratio of the number of tire edge contour points near the ground to the total number of pixels in the tire mask area. `topk` is a set value, with a step size of 0.01 designed for the experiment. In this embodiment, `topk` is preferably 0.5%, corresponding to 52 pixels, meaning the number of tire edge contour points near the ground is 52.

[0167] Step 6-2B: Using the RANSAC algorithm to fit straight line information to the edge points obtained in Step 6-2A, the segmentation line of the tire-ground contact is obtained, as well as the intersection point of the segmentation line and the tire's circumcircle. Then, the pixel length L of the tire-ground contact segmentation line is obtained, such as... Figure 13 As shown; where L is the length of the intersection line between the circumcircle of the tire and the ground without considering tire compression deformation.

[0168] Step 6-2C: Select the smallest x-coordinate among the 52 edge points. min and the maximum x-coordinate max Project the corresponding edge points onto the segmentation line obtained in step 6-2B, and calculate the pixel length l of the tire contacting the ground.

[0169] Step 6-2D: Find the lowest point of the contour directly opposite the center of the tire's circumcircle in the tire mask area coordinates, calculate the distance from the center of the tire's circumcircle to the lowest point of the contour, and obtain the pixel distance h from the center of the tire to the ground.

[0170] Step 6-2E: Calculate the tire compression area ΔS. The calculation formula is:

[0171] ΔS=S1-S2

[0172] The maximum pixel area S1 of the tire is calculated using the maximum pixel radius CircleR of the tire.

[0173] The tire mask area S2 mentioned above, also known as the equivalent pixel area after tire deformation, is calculated by using cv2.contourArea to calculate the tire mask area and the wheel hub mask area.

[0174] Step 6-3: Calculate the tire deformation ratio factor μ. The specific calculation formula is as follows:

[0175]

[0176] In the formula, r Rim The actual radius of the wheel hub can be determined from the markings on the tire.

[0177] Step 6-4: Calculate tire deformation parameters: Using the tire deformation scaling factor μ from Step 6-3, correct the tire-ground contact pixel length l, tire maximum pixel radius CircleR, tire center-to-ground pixel distance h, tire maximum pixel area S1, and tire compression area ΔS obtained in Steps 6-1 and 6-2 to obtain the tire deformation parameters.

[0178] The aforementioned tire-to-ground contact pixel length *l*, tire maximum pixel radius *CircleR*, and tire center-to-ground pixel distance *h* are all length-type data. During correction, the original data needs to be multiplied by the tire deformation scaling factor *μ*. Taking the tire-to-ground contact pixel length as an example, *l*... true =μ×l.

[0179] The maximum tire pixel area S1 and tire compression area ΔS mentioned above are both area-type data. When correcting them, the original data needs to be multiplied by the square of the tire deformation scaling factor μ. Taking the maximum tire pixel area as an example...

[0180] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. A non-contact tire deformation recognition method based on a fine-tuned large visual model, characterized in that: Includes the following steps: Step 1, Camera parameter calibration: Calibrate the parameters of the camera used to acquire tire images; Step 2: Fine-tune the large visual model, which includes the following steps: Step 2-1: Create an application scenario dataset, which includes the following steps: Step 2-1A: Using the camera calibrated in Step 1, acquire side images of different types of tires under different lighting conditions; Step 2-1B: Perform distortion correction on each side image using the camera parameters calibrated in Step 1; Step 2-1C: Mark the tire and wheel hub areas in each side image after distortion correction to form an application scenario dataset; Step 2-2, Fine-tuning: The large visual model includes an image decoder, a cue encoder, and a mask decoder; freeze the parameter weights of the image encoder and the cue encoder; Based on the application scenario dataset created in step 2-1, the parameter weights of the mask decoder are transferred and trained to obtain a fine-tuned large-scale tire vision model. The method for transferring and training the mask decoder parameter weights includes the following steps: Step 2-2A, Downsampling: Downsample each side image in the application scenario dataset created in Step 2-1; Step 2-2B: Split the dataset: Randomly divide the downsampled application scenario dataset into training set, test set and validation set in a ratio of 7:2:1; Step 2-2C: Generate the point cue sequence of the sample tire: Based on the geometric features of the sample tire and the image contour extraction algorithm, obtain the center point coordinates, N outer offset point coordinates, and N inner offset point coordinates of the sample tire; where N≥3; the outer offset points are located on the tire of the sample tire, and the inner offset points are located on the rim of the sample tire; the center point coordinates, N outer offset point coordinates, and N inner offset point coordinates of the sample tire together form the point cue sequence of the sample tire; Step 2-2D, Training: Using the application scenario dataset in Step 2-2B, the true tire mask and true wheel hub mask of the sample tires obtained in Step 2-1C, and the sample tire point cue sequence generated in Step 2-2C, the parameter weights of the mask decoder are trained to obtain the fine-tuned large tire visual model. Step 3: Obtain the actual tire image: Using the camera calibrated in Step 1, acquire a side image of the tire to be inspected; and perform distortion correction on the acquired side image of the tire to be inspected using the camera parameters calibrated in Step 1 to obtain the actual tire image. Step 4: Generate a point cue sequence: Based on the geometric features of the tire to be detected and the image contour extraction algorithm, obtain the coordinates of the center point, N outer offset points, and N inner offset points of the tire to be detected; where N≥3; the outer offset points are located on the tire to be detected, and the inner offset points are located on the rim of the tire to be detected; the center point, the N outer offset points, and the N inner offset points together form a point cue sequence; Step 5, Tire Segmentation: Input some or all of the point cue sequences from Step 4 into the large visual tire model obtained in Step 2, and segment the actual tire image obtained in Step 3 to obtain the actual tire mask region. Step 6: Obtain tire deformation parameters: Post-process the tire mask area obtained in Step 5 using geometric fitting based on OpenCV to obtain the tire circumcircle and the line connecting it to the ground, thereby calculating the tire deformation parameters; the deformation of the tire to be detected is identified by the tire deformation parameters. Tire deformation parameters include length-type data and area-type data; among them, the length-type data includes the actual length of the tire in contact with the ground. Maximum actual radius of the tire and the actual distance from the center of the tire to the ground Area type data includes the maximum actual tire area. and the actual area of ​​tire compression The method for obtaining tire deformation parameters includes the following steps: Step 6-1: Obtain the tire circumcircle: The tire circumcircle includes the tire's outer circumcircle and the wheel rim's outer circumcircle. The method for obtaining the tire circumcircle includes the following steps. Step 6-1A, draw Extract the outer contour points of the tire carcass; draw the outer contour. Extract the outer contour points of the wheel hub; Step 6-1B: Fit the outer contour points of the tire to obtain the outer circumcircle of the tire, the coordinates of the center point of the outer circumcircle of the tire, and the maximum pixel radius of the tire. ; Step 6-1C: Fit the outer contour points of the wheel hub to obtain the circumcircle of the wheel hub and the pixel radius of the wheel hub. ; Step 6-2: Obtain the ground wiring, including the following steps; Step 6-2A, Tire-Ground Contact Segmentation: Introducing Proportional Parameters Extract the front of the tire mask area by sorting it from bottom to top. The edge points; among them, This is the ratio of the number of edge contour points of the tire near the ground to the total number of pixels in the tire mask area; Step 6-2B: Using the RANSAC algorithm, fit the edge points obtained in Step 6-2A to the straight line information to obtain the segmentation line of the tire-ground contact and the intersection point of the segmentation line with the tire's circumcircle, and then obtain the pixel length of the tire-ground contact segmentation line. ; Step 6-2C: Select the minimum x-coordinate among all edge points in Step 6-2A. and the maximum x-coordinate Projecting the corresponding edge points onto the segmentation line obtained in step 6-2B, the pixel length of the tire-ground contact area is calculated. ; Step 6-2D: Locate the lowest point of the contour directly opposite the center of the tire's circumcircle in the tire mask area coordinates. Calculate the distance from the center of the tire's circumcircle to the lowest point of the contour to obtain the pixel distance from the tire's center to the ground. ; Step 6-2E: Calculate the tire compression area The calculation formula is: = ； In the formula, the maximum pixel area of ​​the tire is... Using the maximum pixel radius of the tire Calculated; Tire mask area The tire mask area is calculated using cv2.contourArea; Step 6-3: Introduce the tire deformation ratio factor ; Step 6-4: Calculate tire deformation parameters: Use the tire deformation scaling factor from Step 6-3. The tire-ground contact pixel length obtained in steps 6-1 and 6-2 Maximum pixel radius of tire Pixel distance from the center of the tire to the ground Maximum pixel area of ​​tire and tire compression area The original data is then corrected to obtain the tire deformation parameters. For length-type data, the original data needs to be compared with the tire deformation scaling factor during the correction process. Multiplication; for area-type data, during correction, the original data needs to be multiplied by the tire deformation scaling factor. Multiply the squares of the products.

2. The non-contact tire deformation recognition method based on a fine-tuned large visual model according to claim 1, characterized in that: In step 2-1C, the tire and wheel hub areas are marked separately for each side image after distortion correction using the lableme tool.

3. The non-contact tire deformation recognition method based on a fine-tuned visual large model according to claim 1, characterized in that: In step 2-2A, the cv2.pyrDown function is used to downsample each side image in the application scenario dataset created in step 2-1 twice.

4. The non-contact tire deformation recognition method based on a fine-tuned large visual model according to claim 1, characterized in that: In step 2-2C, cv2.findContours is used to calculate the continuous edge contour of the outermost edge of the tire. The outer contour with the largest enclosing area is selected, and cv2.boundingRect is used to construct the bounding rectangle. The coordinates of the upper left corner of the bounding rectangle are then obtained. and the width of the circumscribed rectangle and high ; The coordinates of the center point of the outer contour are calculated using the geometric features of the sample tire, along with N=4 coordinates of the outer offset points and N=4 coordinates of the inner offset points. The four external offset points are all 90° apart on the tire of the sample tire; the four internal offset points are all 90° apart on the wheel hub of the sample tire.

5. The non-contact tire deformation recognition method based on a fine-tuned large visual model according to claim 4, characterized in that: Center point coordinates ( The formula for calculating ) is: ； ； The four lateral deviation sites are the left lateral deviation site, the right lateral deviation site, the lower lateral deviation site, and the upper lateral deviation site, with corresponding coordinates as follows: , , and The calculation formulas are as follows: ； ； ； ； in, , ; The four inner offset points are the left inner offset point, the right inner offset point, the lower inner offset point, and the upper inner offset point, with corresponding coordinates as follows: , , and The calculation formulas are as follows: ； ； ； ； in, , .

6. The non-contact tire deformation recognition method based on a fine-tuned large visual model according to claim 4 or 5, characterized in that: In step 5, the actual tire masking area includes the tire masking area and the rim masking area; the segmentation method for the tire masking area involves inputting the following elements in the following order: first, input the coordinates of the center cue point to obtain the masking area with the highest score. Next, enter the mask area. Based on the coordinates of the four outer offset points, the mask region with the highest score is obtained and output. ; The method for segmenting the wheel hub mask region involves inputting elements in the following order: first, input the coordinates of the center cue point to obtain the mask region with the highest score. Next, enter the mask area. Given the coordinates of the four inner offset points, obtain and output the mask with the highest score. .

7. The non-contact tire deformation recognition method based on a fine-tuned large visual model according to claim 1, characterized in that: In step 6-3, the tire deformation ratio factor The specific calculation formula is as follows: ； In the formula, This is the actual radius of the wheel hub, which can be obtained from the characters marked on the tire.

Images