Tire damage detection system and method based on artificial intelligence, and vehicle

Abstract

The invention provides a tire damage detection system and method based on artificial intelligence, and a vehicle, and belongs to the technical field of tire detection, and the system comprises an image collection module which is used for collecting a tire image through a vehicle-mounted camera during the driving period of the vehicle; the preprocessing module is used for preprocessing the tire image; the tire damage detection module is used for inputting the preprocessed tire image into a pre-trained tire detection model to obtain a tire detection result; the display module is used for displaying the tire detection result to a user and providing a corresponding tire damage trend analysis result; and the AI question and answer module is used for receiving the tire image and the question proposed by the user, and feeding back tire-related answer information for the question based on an AI question and answer model. The method has the beneficial effects that potential faults are predicted, early warning is carried out, preventive maintenance is carried out, and the driving safety and reliability are improved; questions are answered based on an AI question and answer model, so that a driver can deeply understand the actual condition and potential risk of the vehicle tire.

Description

Technical Field

[0001] The present invention relates to the technical field of tire detection, and in particular to a tire damage detection system and method based on artificial intelligence, and a vehicle. Background Art

[0002] With the rapid development of automobile electronics and intelligence, the complexity of vehicles has increased significantly, which has put forward higher requirements for vehicle maintenance management and troubleshooting. Especially in the field of tire safety, as a key component of vehicle driving, the status of tires is directly related to the safety of drivers and the overall performance of vehicles.

[0003] Existing tire pressure detection mainly includes the tire pressure monitoring system (TPMS) on board and the tire pressure detection function in the vehicle diagnostic system, which monitors the tire pressure in real time during vehicle driving and issues an alarm when the tire pressure is abnormal to remind the driver to take timely measures.

[0004] Traditional tire pressure detection can provide a certain degree of passive detection, but this passive tire pressure detection method is time-consuming and labor-intensive, and cannot monitor the tire status in real time, posing certain safety hazards; there is currently a lack of active preventive measures, and potential tire failures cannot be discovered and resolved in advance. Usually, warnings can only be issued and corresponding measures can only be taken after obvious problems with the tire or a failure occurs, increasing safety risks and maintenance costs; especially sudden tire pressure alarms may face the dilemma of no nearby maintenance outlets, increasing the safety risk of vehicle accidents caused by tire blowouts.

[0005] In addition, the existing technology lacks effective interaction with users and has difficulty in providing detailed system operating status and fault prediction information, making it difficult for drivers to deeply understand the actual condition and potential risks of vehicle tires, further weakening the effectiveness of fault prevention and response. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a tire damage detection system and method, and a vehicle based on artificial intelligence.

[0007] The technical problem solved by the present invention can be achieved by adopting the following technical solutions:

[0008] A tire damage detection system based on artificial intelligence, comprising:

[0009] An image acquisition module, used to acquire tire images using a vehicle-mounted camera while the vehicle is traveling;

[0010] A preprocessing module, connected to the image acquisition module, for preprocessing the tire image to obtain a preprocessed tire image;

[0011] A tire damage detection module, connected to the preprocessing module, for inputting the preprocessed tire image into a pre-trained tire detection model to obtain a tire detection result;

[0012] A display module, connected to the tire damage detection module, for displaying the tire detection result to the user and providing corresponding tire damage trend analysis result;

[0013] The AI ​​question-answering module is used to receive the tire image and the tire-related questions raised by the user, and to provide tire-related answer information for the tire image and the questions based on the AI ​​question-answering model.

[0014] Preferably, the image acquisition module comprises:

[0015] The capturing unit is used to capture the tire image from the panoramic imaging interface when the vehicle enters the panoramic imaging interface including the wheel hub perspective.

[0016] Preferably, it also includes:

[0017] A panoramic switching module is used to enable the vehicle to enter a panoramic image interface including the wheel hub perspective when the vehicle meets preset conditions;

[0018] The preset conditions include:

[0019] Shifting from Park to Drive; and / or

[0020] Shift into reverse gear; and / or

[0021] The steering lever is switched to the left or right steering position.

[0022] Preferably, the preprocessing module comprises:

[0023] A white balance and exposure processing unit, configured to perform automatic white balance and automatic exposure processing on the tire image; and / or

[0024] a high dynamic range imaging processing unit, configured to perform high dynamic range imaging processing on the tire image; and / or

[0025] a denoising unit, configured to perform noise removal processing on the tire image; and / or

[0026] An image enhancement unit is used to perform image enhancement processing on the tire image.

[0027] Preferably, the tire detection model comprises:

[0028] An input layer, used for receiving the preprocessed tire image;

[0029] A convolution layer, connected to the input layer, for extracting feature information from the preprocessed tire image and mapping it to form a feature map through a convolution operation;

[0030] A pooling layer, connected to the convolutional layer, for performing pooling processing on the feature map;

[0031] A fully connected layer, connected to the pooling layer, for integrating the feature maps after pooling processing to obtain global features and outputting the conditional probability distribution of the tire image belonging to each preset category;

[0032] The output layer is connected to the fully connected layer and is used to select the category with the highest probability as the tire detection result according to the conditional probability distribution output by the fully connected layer.

[0033] Preferably, the tire detection result includes the tire damage type, damage degree, determination basis and measures to be taken;

[0034] The tire damage types include cracks, and / or cuts, and / or wear.

[0035] Preferably, the AI ​​question-answering module further includes an AI question-answering model training unit, and the AI ​​question-answering model training unit includes:

[0036] a data preparation subunit, configured to collect a pre-training data set, wherein the pre-training data set includes a driving scene, a vector description of an object level in the driving scene, a language description including a scene description and questions / answers / scoring / scoring explanations, and driving question-and-answer data, wherein the driving question-and-answer data includes open-ended questions and answer information for each driving scene;

[0037] A model pre-training subunit, configured to use the vector description and the language description as model inputs and the driving question-and-answer dataset as a model pre-prompt to perform pre-training of the AI ​​question-and-answer model;

[0038] The model evaluation and optimization subunit is used to score the answer information of the open-ended questions generated by the pre-trained AI question-answering model and optimize it according to the scoring evaluation results.

[0039] Preferably, the AI ​​question-answering model includes:

[0040] a vector encoder comprising a multi-layer perceptron layer and a criss-cross attention layer for converting an input vector in the preprocessed tire image into an input latent vector in a latent space and adding an ego feature to each learned input latent vector;

[0041] A vectorizer connected to the vector encoder, the vectorizer comprising a self-attention layer and a cross-attention layer for processing the latent space and question tokens to convert the input latent vector into an embedding that can be decoded by a large language model;

[0042] A frozen large language model with a low-rank adaptive module is connected to the vectorizer and is used to inject a trainable rank decomposition matrix into the linear layer of the pre-trained large language model to obtain the answer information according to the output of the vectorizer and the question raised by the user.

[0043] The present invention also provides a tire damage detection method based on artificial intelligence, which is applied to the above-mentioned tire damage detection system based on artificial intelligence, and the method comprises:

[0044] While the vehicle is driving, the tire image is collected using the vehicle-mounted camera;

[0045] Preprocessing the tire image to obtain a preprocessed tire image;

[0046] Inputting the preprocessed tire image into a pre-trained tire detection model to obtain a tire detection result;

[0047] Display the tire test results to the user and provide corresponding tire damage trend analysis results;

[0048] Receive the question raised by the user, and provide tire-related answer information based on the AI ​​question-answering model.

[0049] The present invention also provides a vehicle, comprising:

[0050] On-board camera for collecting tire images

[0051] An on-board computing unit, including an artificial intelligence-based tire damage detection system as described above;

[0052] A user interface is used to display the output results of the tire damage detection system.

[0053] The advantages or beneficial effects of the technical solution of the present invention are:

[0054] The present invention utilizes a vehicle-mounted camera to realize real-time acquisition of tire images, analyzes the degree of tire damage in real time based on a neural network model, predicts potential failures and issues early warnings, thereby performing preventive maintenance, improving driving safety and reliability, and reducing maintenance costs and time; at the same time, it interacts with users based on an AI question-and-answer model, and answers users' questions about tire status in the form of natural language, allowing drivers to deeply understand the actual condition and potential risks of vehicle tires, thereby enhancing user experience and the convenience of vehicle maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a structural block diagram of a tire damage detection system based on artificial intelligence in a preferred embodiment of the present invention;

[0056] Figure 2 A schematic diagram of vehicle hardware in a preferred embodiment of the present invention;

[0057] Figure 3 A schematic diagram of the field of view of a vehicle-mounted camera in a preferred embodiment of the present invention;

[0058] Figure 4 A schematic diagram of the structure of a tire detection model in a preferred embodiment of the present invention;

[0059] Figure 5 A schematic diagram of a report of a tire training set in a preferred embodiment of the present invention;

[0060] Figure 6 A schematic diagram of tire damage trend in a preferred embodiment of the present invention;

[0061] Figure 7 A schematic diagram of converting a driving scene into a vector description and a language description in a preferred embodiment of the present invention;

[0062] Figure 8 A schematic diagram of the pre-training of an AI question-answering model in a preferred embodiment of the present invention;

[0063] Fig. 9 This is a schematic diagram of the architecture of an AI question-answering model in a preferred embodiment of the present invention;

[0064] Fig.10 The figure is a flow chart of a tire damage detection method based on artificial intelligence in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0067] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0068] See also Figure 1 and Figure 2 In a preferred embodiment of the present invention, based on the above problems existing in the prior art, a tire damage detection system based on artificial intelligence is provided, comprising:

[0069] The image acquisition module 1 is used to acquire tire images using the vehicle-mounted camera 701 during vehicle driving;

[0070] The preprocessing module 2 is connected to the image acquisition module 1 and is used to preprocess the tire image to obtain a preprocessed tire image;

[0071] A tire damage detection module 3, connected to the preprocessing module 2, is used to input the preprocessed tire image into a pre-trained tire detection model 31 to obtain a tire detection result;

[0072] Display module 4, connected to tire damage detection module 3, for displaying tire detection results to users and providing corresponding tire damage trend analysis results;

[0073] The AI ​​question-answering module 5 is used to receive tire images and tire-related questions raised by users, and to feed back tire-related answer information based on the tire images and questions using an AI question-answering (AIQA) model 51.

[0074] Specifically, the present invention utilizes a vehicle-mounted camera to realize real-time acquisition of tire images, analyzes the degree of tire damage in real time based on AI deep learning technology, determines whether the tire has potential damage failure and issues an early warning, reminds the driver to check and handle it in time, and perform preventive maintenance, thereby realizing early warning of tire wear and tire wear trend analysis and prediction, thereby improving driving safety and reliability, and reducing maintenance costs and time; at the same time, based on the AI ​​question-answering model, it interacts with users and answers users' questions about tire status in the form of natural language, so that the driver can deeply understand the actual condition and potential risks of the vehicle tires, thereby enhancing the user experience and the convenience of vehicle maintenance.

[0075] In this embodiment, the tire image is captured by using the vehicle's own onboard camera 701, without adding additional hardware equipment.

[0076] To achieve high-definition image acquisition, the on-board camera 701 uses 3 million pixels, which not only has excellent resolution, but also has high-speed continuous shooting capability. Its frame rate is set to 30 frames per second, that is, it can capture 30 high-definition tire images per second. These continuously shot pictures are then transmitted to the system's on-board computing unit 702 in real time for further image processing, and finally generate clear and accurate tire status pictures or video records, providing a reliable basis for subsequent tire damage degree analysis and potential fault prediction.

[0077] As a preferred embodiment, the image acquisition module 1 includes:

[0078] The capturing unit 11 is used to capture the tire image from the panoramic image interface when the vehicle enters the panoramic image interface including the wheel hub perspective.

[0079] Specifically, in this embodiment, after the vehicle enters the panoramic image interface, the interface includes a wheel perspective, thereby capturing a tire image.

[0080] In this embodiment, the tire image may capture one or more images. The one or more tire images include images of the left front, left rear, right front or right rear tire of the vehicle to ensure that the status of each tire can be accurately monitored and analyzed.

[0081] As a preferred embodiment, the invention further comprises:

[0082] The panoramic switching module 6 is used for the vehicle to enter a panoramic image interface including a wheel hub perspective when the vehicle meets preset conditions.

[0083] The pre-conditions include:

[0084] Shifting from the parking gear (P gear) to the driving gear (D gear); and / or

[0085] Shift to reverse gear (R); and / or

[0086] The steering lever is switched to the left or right steering position.

[0087] Specifically, when the vehicle meets certain preset conditions, such as switching from the parking gear (P gear) to the drive gear (D gear), switching to the reverse gear (R gear), or the steering lever is switched to the left turn position or the right turn position, the panoramic switching module 6 is immediately activated, automatically guiding the vehicle into a panoramic image interface including the wheel hub perspective, ensuring that the system can instantly capture the real-time status of the tires every time the vehicle starts, reverses or changes lanes.

[0088] like Figure 3 A schematic diagram of the field of view of a vehicle-mounted camera is shown. The vehicle-mounted camera is deployed on the left and right rearview mirrors of the vehicle. Figure 3 In the figure, angles ɑ1 and ɑ2 represent the field of view of the vehicle camera, and their boundary lines are the vertical viewing angles of the camera; angles b1 and b2 represent the horizontal to upward viewing angles, and angles c1 and c2 represent the wheel viewing angles.

[0089] In actual operation, every time the vehicle is started and switched from P gear to D gear, the system responds immediately and automatically enters the panoramic image mode, in which the wheel hub and its surrounding area are clearly visible. Similarly, when the vehicle is shifted into R gear to prepare for reverse, or the driver sends a lane change signal through the steering lever, the system can also quickly adjust to the corresponding panoramic view to ensure that the tire image is fully captured.

[0090] As a preferred embodiment, wherein Figure 1 As shown, the preprocessing module 2 includes:

[0091] A white balance and exposure processing unit 21, used for performing automatic white balance and automatic exposure processing on the tire image; and / or

[0092] A high dynamic range imaging processing unit 22, configured to perform high dynamic range imaging processing on the tire image; and / or

[0093] A denoising unit 23, used to perform noise removal processing on the tire image; and / or

[0094] The image enhancement unit 24 is used to perform image enhancement processing on the tire image.

[0095] Specifically, in this embodiment, the captured tire image is preprocessed, including but not limited to denoising, contrast enhancement, brightness and color balance adjustment, etc., to improve the image quality.

[0096] As an example, using automatic white balance and automatic exposure adjustment can intelligently identify and adapt to different lighting conditions to reduce the adverse effects of lighting differences on image quality under different lighting conditions, ensuring the true restoration of image colors and clear display of details.

[0097] To address the problem of image detail loss caused by strong light contrast, high dynamic range (HDR) imaging technology was introduced. HDR technology captures and retains all details in the scene from the darkest to the brightest by synthesizing multiple images with different exposures, thereby significantly improving the dynamic range and layering of the image, allowing subtle damage and texture on the tire surface to be accurately presented even under strong light.

[0098] In order to address the noise points that may be introduced by the camera during the acquisition process due to environmental or equipment factors, filtering technology is used to identify and remove random noise in the image, while retaining the edge information and important features of the image, thereby improving the image clarity and signal-to-noise ratio.

[0099] For the low contrast, unclear colors or blurred areas that may appear in the image, a deblurring model is defined and created based on the Convolutional Neural Network (CNN), and training is used to learn how to map these blurred input images into clear output images. By continuously adjusting the internal parameters of the model and optimizing the deblurring algorithm, the tire image is deeply enhanced, making the image clearer and more vivid, and further enhancing the image visualization effect.

[0100] Deep learning technology is based on the architecture of artificial neural networks, especially deep neural networks that contain multiple hidden layers. It can automatically learn complex representations and features from large amounts of data by simulating the way the human brain processes information.

[0101] Convolutional Neural Network (CNN) is a deep learning model that is mainly used to process and analyze images, videos and other types of two-dimensional or three-dimensional data.

[0102] The embodiment of the present invention utilizes CNN image processing technology to extract features of tire images, such as edge detection, texture analysis, shape recognition, etc. These features will be used for subsequent damage identification.

[0103] Edges are important features of the contours of objects in an image. The Canny edge detection algorithm is used to identify and extract the contours of the tire and possible damaged edges. Canny is a multi-step algorithm that can be used to detect the edges of any input image. The specific steps are as follows:

[0104] First, a Gaussian filter is applied to smooth the image to remove noise.

[0105] Then, the derivative of the Gaussian filter is calculated to calculate the gradient of the image pixels in the x and y dimensions to obtain the gradient magnitude and direction.

[0106] Next, the non-maximum suppression technique is applied to retain the local maximum points in the gradient direction and eliminate false edge detection to refine the edge.

[0107] Next, a double threshold method is applied to determine possible (potential) boundaries and classify edge pixels into three categories: strong edge, weak edge, and non-edge.

[0108] Finally, the hysteresis threshold method is used to retain pixels above the gradient amplitude and ignore pixels below the low threshold to achieve edge tracking.

[0109] Furthermore, changes in the texture of the tire surface may indicate damage. Applying texture analysis techniques, the texture features of the image can be extracted.

[0110] Furthermore, the shape characteristics of the tire, such as deformation of the circular contour, may be a sign of damage. Applying shape recognition techniques, such as geometric shape descriptors and contour analysis, can be used to identify and extract the shape characteristics of the tire to determine whether there is a shape change caused by damage.

[0111] As a preferred embodiment, wherein Figure 4 As shown, the tire detection model 31 includes:

[0112] An input layer 310 for receiving a preprocessed tire image;

[0113] The convolution layer 311 is connected to the input layer 310 and is used to extract feature information from the preprocessed tire image and map it to form a feature map through a convolution operation;

[0114] The pooling layer 312 is connected to the convolution layer 311 and is used to perform pooling processing on the feature map;

[0115] The fully connected layer 313 is connected to the pooling layer 312, and is used to integrate the feature map after the pooling process to obtain the global feature and output the conditional probability distribution of the tire image belonging to each preset category;

[0116] The output layer 314 is connected to the fully connected layer 313 and is used to select the category with the highest probability as the tire detection result according to the conditional probability distribution output by the fully connected layer 313.

[0117] Specifically, in this embodiment, a tire detection model 31 is constructed based on a convolutional neural network CNN, and the tire detection model 31 includes an input layer 310, a convolution layer 311, a pooling layer 312, a fully connected layer 313, and an output layer 314 connected in sequence. Its working process can be divided into three parts: network model construction, network training, and network prediction. Among them:

[0118] The network model construction includes but is not limited to designing the size of the input data, data characteristics, the depth of the network, the activation function of each layer, and the step size of the convolution layer and the pooling layer. The size of the input data determines the size of the input layer of the network. If the input data is a high-resolution image, the input layer may need more nodes to process more pixel information. The depth of the network refers to the number of layers contained in the network. The deeper the depth, the more complex features the network can learn. Data characteristics include the number of color channels of the data (such as 3 channels for color images) and the size of the image (including width and height). The activation function of each layer determines how the neuron responds to the input signal. The activation function includes the Rectified Linear Unit (ReLU), the Sigmoid function, and the hyperbolic tangent function (Tanh). For example, ReLU is used as the activation function of the convolution layer and the fully connected layer to improve the training speed and generalization ability of the model. The step size in the convolution layer determines the step size of the convolution kernel moving on the input data; the step size in the pooling layer determines the step size of the pooling window moving on the feature map.

[0119] The network training part includes training samples, which are data sets used to train the network. The training data set includes a large number of labeled images and their corresponding labels. During the training process, the network model calculates the difference between the output and the true label, and then adjusts the network weights through the back propagation algorithm of the error; uses optimization algorithms such as Stochastic Gradient Descent (SGD) and its variants to update the network weights to minimize the loss function; iterates repeatedly to repeat the forward propagation and back propagation processes, and the performance of the network gradually improves after each iteration.

[0120] The training dataset should contain a large number of normal and damaged tire images to ensure that the model can accurately identify different types of damage.

[0121] For example, the dataset type is: tire image classification

[0122] Dataset format: contains jpg images

[0123] Number of images (jpg files): 9000

[0124] Number of classification categories: 2

[0125] Category name: ["Damaged", "Normal"]

[0126] Number of images per category:

[0127] Number of damaged images: 6000

[0128] Normal number of pictures: 3000

[0129] The above dataset includes 9,000 tire images in jpg format, which are divided into two categories: "damaged" and "normal". The damaged category contains 6,000 images and the normal category contains 3,000 images to ensure that the model can accurately identify different types of tire damage. This dataset is specially used for tire image classification. It uses folder classification to store images. Each category folder stores the corresponding images.

[0130] By monitoring key indicators such as training accuracy (train_accuracy), test accuracy (test_accuracy), receiver operating characteristic (ROC) curve, training loss (train_loss) and test loss (test_loss), as well as sensitivity / specificity evaluation during the training process, the performance of the model during the training and testing phases can be analyzed, such as Figure 5 As shown, these charts intuitively show the changing trends of model accuracy, recall rate, loss function and their specific performance in different categories.

[0131] Exemplarily, the following Table 1 shows tire damage training results.

[0132] Table 1 Tire damage training results

[0133] Damage Number of test samples Number of false positive samples Detection accuracy % Minor damage 100 3 97％ Moderate damage 100 2 98％ Severe damage 100 2 98％

[0134] In the network prediction part, the trained model is applied to the real-time captured tire images to identify the actual damage of the tire. The model should be able to distinguish different types of damage, such as cracks, cuts, wear, etc. In the network prediction process, the model is forward propagated layer by layer. The input layer 310 is used to receive the pre-processed tire image. The image data starts from the input layer 310 and is passed to the output layer 314 of the network layer by layer. Among them:

[0135] In the convolution layer 311 and the pooling layer 312, each layer outputs a feature map, which contains feature information of different categories. The convolution layer 311 slides a window on the tire image through a series of convolution operations to extract rich local feature information, including but not limited to feature information such as tire texture, wear marks, and crack morphology. Through the convolution operation, the input image is mapped into a series of feature maps. The pooling layer 312 is used to reduce the dimension of the feature map through pooling operations, such as maximum pooling and average pooling, so that redundant information in the feature map is effectively removed, while the key features are retained, so as to reduce the computational complexity of the model and prevent overfitting, improve the generalization ability of the model, and accelerate the training process.

[0136] In the fully connected layer 313, the feature map after pooling is further integrated and abstracted to form a global feature representation. These global features contain complete information of the tire image and can fully reflect the current state of the tire. Subsequently, the fully connected layer uses these global features to calculate the conditional probability distribution of the tire image belonging to each preset category through a series of linear transformations and nonlinear activation functions, such as the normalized exponential function (softmax). These preset categories usually include different tire damage types, such as cracks, cuts, wear, etc., and their severity.

[0137] The output layer 314 selects the category with the highest probability as the prediction result from the conditional probability distribution output by the softmax layer of the fully connected layer 313.

[0138] Exemplarily, the tire detection results are shown in Table 2 below.

[0139] Table 2 Tire test results

[0140]

[0141] After the model outputs the results, the tire test results are displayed to the driver, and the corresponding tire damage trend analysis results are provided so that tire problems can be dealt with in a timely manner. The test results and analysis results are displayed in the form of charts, graphs or text to facilitate the driver's understanding.

[0142] like Figure 6 An example of tire damage trend analysis results is shown. Figure 6 The predicted wear of the tire from the first week (i.e. the start of the prediction) to the sixteenth week is shown in a chart. Through AI analysis, the trend of the tire tread depth over time can be obtained. As time goes by, the tread depth gradually decreases and the degree of wear gradually increases. Through trend analysis, early warning of tire wear and preventive maintenance can be performed, thereby improving driving safety and reliability.

[0143] As a preferred embodiment, the tire detection result includes the tire damage type, damage degree, judgment basis and measures to be taken;

[0144] Tire damage types include cracks, and / or cuts, and / or wear.

[0145] Specifically, the tire test results include not only the type of tire damage (such as cracks, cuts, wear, etc.), but also detailed information such as the degree of damage, the basis for judgment, and the measures to be taken. This information can help them understand the tire status in a timely manner and take necessary maintenance or replacement measures to ensure driving safety, which is of great practical value to drivers.

[0146] As a preferred implementation, the AI ​​question-answering module 5 further includes an AI question-answering model training unit, and the AI ​​question-answering model training unit includes:

[0147] A data preparation subunit is used to collect a pre-training data set, which includes a driving scenario, a vector description of an object level in the driving scenario, a language description including a scenario description and questions / answers / scoring / scoring explanations, and a driving question-and-answer data, which includes open-ended questions and answer information for each driving scenario;

[0148] A model pre-training subunit, for pre-training an AI question-answering model 51 by using the vector description and the language description as model input and the driving question-answering dataset as a model pre-prompt 52;

[0149] The model evaluation and optimization subunit is used to score the answer information of open-ended questions generated by the pre-trained AI question-answering model and optimize it according to the scoring evaluation results.

[0150] As a preferred implementation, the AI ​​question-answering model 51 includes:

[0151] A vector encoder 511, the vector encoder includes a multi-layer perceptron layer and a cross-attention layer, which is used to convert the input vector in the preprocessed tire image into an input latent vector in a latent space, and add the self-feature to each learned input latent vector;

[0152] A vectorizer 512, connected to the vector encoder 511, the vectorizer includes a self-attention layer and a cross-attention layer, which is used to process the latent space and question tokens, and convert the input latent vector into an embedding that can be decoded by the large language model;

[0153] The frozen large language model 513 with a low rank adaptation (LoRA) module 5130 is connected to a vectorizer to inject a trainable rank decomposition matrix into the linear layer of the pre-trained large language model to obtain answer information based on the output of the vectorizer and the question raised by the user.

[0154] Specifically, in the model preparation stage, the driving simulator-Carla simulator is used to generate diverse and realistic driving scenarios, covering different road conditions, traffic conditions and weather conditions; the structured language generator is used to convert the object-level vector representation into a language description containing scenario description Scenario and questions / answers / scoring / scoring explanations, such as Figure 7 An example of the language description corresponding to the scenario of changing lanes from the vehicle to the left turn lane is shown; at the same time, a large language model (LLM) such as Zhipu Qingyan is used to generate a driving question and answer (Driving QA) dataset, which contains open-ended questions and answers for each scenario. The questions cover aspects such as scene understanding, action reasoning, and decision making.

[0155] The model training phase includes model pre-training and Driving QA data fine-tuning. During the model pre-training process, the vector description and language description generated by the structured language generator are used for pre-training, such as Figure 8 The schematic diagram of the AI ​​question answering model pre-training process is shown. Through contrastive learning, the vector space is mapped to the embedding space of the LLM to achieve semantic alignment between the vector and the language. In the process of Driving QA data fine-tuning, Driving QA data fine-tuning is performed based on the three key components of the AIQA model, namely, the vector encoder 511, the vector former 512, and the frozen LLM with a low-rank adaptive module.

[0156] like Fig. 9The schematic diagram of the architecture of the AI ​​question-answering model is shown, showing the pre-training process. The three key components of the architecture are used to perform driving reasoning (DrivingAP) and fine-tune the Driving QA data. The input is four input vectors, including input vector 1 (such as Vehicle tokens), input vector 2 (such as Tire tokens), input vector 3 (such as Routetokens), and input vector 4 (such as Other tokens). The four input vectors pass through the multi-layer perceptron (MLP) layer 5110 of the vector encoder 511, and then are processed by the cross-attention layer 5111 to move the four input vectors to a latent space, and add self-features 5112 (such as lalents) to each learned input latent vector to emphasize the self-state. The vector former 512 includes a self-attention layer 5120 and a cross-attention layer 5121, which processes the latent space and question tokens 5122 (queries) and converts the latent vector into an embedding that can be decoded by the LLM. The vectorizer 512 and the pre-prompt 52 are embedded as input to the LLM. Finally, the trainable rank factorization matrix (LoRA) is injected into the linear layer of the pre-trained LLM with adapter for efficient parameter fine-tuning.

[0157] Finally, the trained AI question-answering model 51 is evaluated and optimized. Specifically, questions are asked to the trained AI question-answering model 51, and the answers to open-ended questions generated by the AI ​​question-answering model 51 are scored using LLM models such as Zhipu Qingyan to evaluate its ability to understand driving scenarios and answer questions; then, based on the evaluation results, the model architecture, training strategy, and data processing method are optimized to improve the performance and robustness of the model.

[0158] Exemplarily, an AIQA conversational question-answering example is shown below.

[0159] For tire condition monitoring:

[0160] Question: "Do my tires need to be replaced?"

[0161] Answer: The system analyzes the tire image to determine the degree of tire wear, whether there are cracks, bulges, etc., and recommends whether the tire needs to be replaced.

[0162] Warning for tire abnormality:

[0163] Question: "Why do my tires squeaking?"

[0164] Answer: The system analyzes tire image and sound information to determine whether there is any abnormality in the tire, such as low tire pressure, tread damage, etc., and issues a warning.

[0165] Question: "My tires look fine, so why are the tire pressures showing as low?"

[0166] Answer: The system combines image information and tire pressure monitoring data to analyze that the possible cause is tire leakage and the current air pressure is 1.85 Bar.

[0167] Recommendations for tire use:

[0168] Question: "What tires should I choose?"

[0169] Answer: The system recommends the appropriate tire type and specifications based on vehicle type, driving habits, road conditions and other information.

[0170] Tips for driving safety:

[0171] Question: "Why is my vehicle driving erratically?"

[0172] Answer: The system analyzes tire conditions and vehicle driving data to determine whether unstable driving is caused by tire problems and gives corresponding driving suggestions.

[0173] Popularization of tire knowledge:

[0174] Question: "What is the purpose of the tread on a tire?"

[0175] Answer: The system explains the functions of tire patterns, such as drainage, grip, etc.

[0176] The system of the present invention integrates a large language model to achieve in-depth understanding and prediction of complex traffic scenarios. The model supports advanced scene understanding capabilities, can process natural language queries, and provide scene explanations.

[0177] The present invention also provides a tire damage detection method based on artificial intelligence, which is applied to the tire damage detection system based on artificial intelligence as described above. Fig.10 As shown, the method includes:

[0178] S100, collecting tire images using a vehicle-mounted camera during vehicle driving;

[0179] S200, preprocessing the tire image to obtain a preprocessed tire image;

[0180] S300, inputting the preprocessed tire image into a pre-trained tire detection model 31 to obtain a tire detection result;

[0181] S400, showing the tire test results to the user and providing corresponding tire damage trend analysis results;

[0182] S500, receiving a tire image and a tire-related question raised by a user, and feeding back tire-related answer information based on the tire image and the question based on the AI ​​question-answering model 51.

[0183] Specifically, in view of the problem that the existing technology can only passively detect the current status of the tire and can only issue an alarm when there is an obvious problem with the tire, and has no data analysis and learning capabilities, the present invention uses deep learning technology to predict potential tire failures and take measures before the problem occurs, thereby improving driving safety and reducing potential risks. At the same time, the model can automatically analyze the real-time status and historical data of the tire, predict the future tire wear trend, predict the tire's service life and potential failures, and provide early warnings, so that the driver can replace the tire in advance according to the wear prediction trend to achieve preventive maintenance.

[0184] Moreover, the AI ​​question-answering model 51 can make the scene explainable. For example, combined with the VLM visual large language model, it can explain why it believes that a certain area is damaged and how it reaches this conclusion based on image features and text descriptions, thereby achieving more advanced human-computer interaction.

[0185] The present invention also provides a vehicle, such as Figure 2 As shown, including:

[0186] On-board camera 701, used to collect tire images

[0187] The vehicle-mounted computing unit 702 includes the tire damage detection system based on artificial intelligence as described above;

[0188] The user interface 703 is used to display the output results of the tire damage detection system.

[0189] Specifically, the onboard sensor 701 (Onboard Sensors) includes two onboard surround-view cameras. The left camera and the right camera of the onboard surround-view camera capture two tire edges respectively. Each camera has 3 million pixels and a resolution of 2560x1440.

[0190] The onboard computing unit 702 is the core processing component of the system, including CPU, GPU, memory, storage devices, etc. It has sufficient computing power to run AI algorithms, conduct real-time data analysis and fault prediction. The onboard computing unit 702 also has certain vibration and impact resistance capabilities to adapt to the movement and environment of the vehicle.

[0191] The user interface 703 is intuitive and easy to operate, including an on-board display screen for displaying key information such as the system's analysis results, fault warnings, and maintenance recommendations to the user. The driver can interact with the system through the interface to view tire status, fault warnings, and maintenance recommendations. The system provides the driver with a fault warning based on the analysis results, which may include the type of fault, possible causes, and recommended countermeasures. The system can also provide the driver with maintenance recommendations based on the predicted maintenance needs, including recommended maintenance cycles, maintenance content, and maintenance priorities.

[0192] The present invention is applicable to all types of vehicles equipped with intelligent driving systems, and can monitor the status of the four tires of the vehicle in real time during normal operation of the vehicle, predict potential failures and take action. The following are common application scenarios: such as daily commuting, long-distance travel, bad weather and other common application scenarios.

[0193] The present invention is also applicable to vehicles in other fields, for example:

[0194] In agricultural machinery, the condition of tires is crucial to the efficiency of operation and the protection of crops. The AI-based tire damage detection system in the embodiment of the present invention can monitor tire wear and air pressure in real time to ensure that the machinery can work stably and efficiently when operating in the field.

[0195] In the field of industrial vehicles, industrial vehicles, such as forklifts, trucks, etc., need to frequently move and carry heavy objects in factory and warehouse environments. The state of the tires directly affects the safety and production efficiency of operations. The AI-based tire damage detection system in the embodiment of the present invention can monitor the tire status in real time, predict tire failures, avoid unexpected downtime and maintenance costs caused by tire problems; and respond quickly in emergency situations to improve the reliability and safety of industrial vehicles.

[0196] Military vehicles travel in complex and ever-changing environments, and the status of tires is directly related to the completion of missions and the safety of personnel. The AI-based tire damage detection system in the embodiment of the present invention can monitor the tire status in real time and respond quickly in emergency situations, thereby improving the reliability and efficiency of military operations.

[0197] Public transportation, such as buses and subways, needs to ensure the safety of passengers and the punctuality of transportation. The condition of tires is crucial to the stability and braking performance of the vehicle. The AI-based tire damage detection system in the embodiment of the present invention can predict tire failures in advance and reduce delays and accidents caused by tire problems.

[0198] In the field of drones and robots, the status of tires (or wheels) is crucial to the movement and execution of tasks of the equipment when the drones and robots are performing tasks. The AI-based tire damage detection system in the embodiment of the present invention can predict tire failures and ensure that the equipment can move stably and reliably when performing tasks.

[0199] Specific application cases: As an important part of urban transportation, city buses undertake a large number of passenger transportation tasks. The safety condition of tires directly affects the driving stability of buses and the safety of passengers. The traditional method of passively checking tires is time-consuming and labor-intensive, and it is impossible to monitor the status of tires in real time, which poses certain safety hazards. The AI-based tire damage detection system of the present invention can be installed on each bus. The system monitors the wear of tires in real time through the surround-view camera installed on the vehicle's exterior rearview mirror; through AI deep learning technology, it analyzes and processes real-time monitored tire photos to predict the degree of tire wear and potential failures; when the system predicts that there is a potential failure in the tire, that is, before the problem occurs, an alarm is issued through the on-board display and buzzer, etc., to remind the driver to check and deal with it in time. You can also use voice / text input to launch AIQA knowledge questions and answers, and the system will give a full explanation.

[0200] The AI ​​tire damage detection system of the present invention can ensure the tire safety of urban buses, improve the reliability and safety of vehicles, reduce traffic accidents caused by tire failures, improve the comfort and satisfaction of drivers and passengers, and provide a safer and more reliable driving experience; promote the development of intelligent transportation systems, and provide strong support for urban traffic management. At the same time, the system can also reduce maintenance costs and reduce delays and accidents caused by tire problems.

[0201] The advantages or beneficial effects of adopting the above technical solution are: the present invention utilizes the on-board camera to realize real-time acquisition of tire images, analyzes the degree of tire damage in real time based on the neural network model, predicts potential failures and issues early warnings, thereby performing preventive maintenance, improving driving safety and reliability, and reducing maintenance costs and time; at the same time, based on the AI ​​question-answering model, it interacts with users and answers users' questions about tire status in the form of natural language, so that drivers can deeply understand the actual condition and potential risks of vehicle tires, thereby enhancing user experience and the convenience of vehicle maintenance.

[0202] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the contents of this specification and illustrations should be included in the protection scope of the present invention.

Claims

1. A tire damage detection system based on artificial intelligence, characterized in that: include: An image acquisition module, used to acquire tire images using a vehicle-mounted camera while the vehicle is traveling; A preprocessing module, connected to the image acquisition module, for preprocessing the tire image to obtain a preprocessed tire image; A tire damage detection module, connected to the preprocessing module, for inputting the preprocessed tire image into a pre-trained tire detection model to obtain a tire detection result; A display module, connected to the tire damage detection module, for displaying the tire detection result to the user and providing corresponding tire damage trend analysis result; The AI ​​question-answering module is used to receive the tire image and the tire-related questions raised by the user, and to provide tire-related answer information for the tire image and the questions based on the AI ​​question-answering model.

2. The tire damage detection system based on artificial intelligence according to claim 1, characterized in that: The image acquisition module comprises: The capturing unit is used to capture the tire image from the panoramic imaging interface when the vehicle enters the panoramic imaging interface including the wheel hub perspective.

3. The tire damage detection system based on artificial intelligence according to claim 2, characterized in that: It also includes a panoramic switching module, which is used for the vehicle to enter a panoramic image interface including a wheel hub perspective when the vehicle meets preset conditions; The preset conditions include: Shifting from Park to Drive; and / or Shift into reverse gear; and / or The steering lever is switched to the left or right steering position.

4. The tire damage detection system based on artificial intelligence according to claim 1, characterized in that: The preprocessing module comprises: A white balance and exposure processing unit, configured to perform automatic white balance and automatic exposure processing on the tire image; and / or a high dynamic range imaging processing unit, configured to perform high dynamic range imaging processing on the tire image; and / or a denoising unit, configured to perform noise removal processing on the tire image; and / or An image enhancement unit is used to perform image enhancement processing on the tire image.

5. The tire damage detection system based on artificial intelligence according to claim 1, characterized in that: The tire detection model includes: An input layer, used for receiving the preprocessed tire image; A convolution layer, connected to the input layer, for extracting feature information from the preprocessed tire image and mapping it to form a feature map through a convolution operation; A pooling layer, connected to the convolutional layer, for performing pooling processing on the feature map; A fully connected layer, connected to the pooling layer, for integrating the feature maps after pooling processing to obtain global features and outputting the conditional probability distribution of the tire image belonging to each preset category; The output layer is connected to the fully connected layer and is used to select the category with the highest probability as the tire detection result according to the conditional probability distribution of the fully connected layer.

6. The tire damage detection system based on artificial intelligence according to claim 1, characterized in that: The tire test results include tire damage type, damage degree, judgment basis and measures to be taken; The tire damage types include cracks, and / or cuts, and / or wear.

7. The tire damage detection system based on artificial intelligence according to claim 1, characterized in that: The AI ​​question-answering module further includes an AI question-answering model training unit, and the AI ​​question-answering model training unit includes: a data preparation subunit, configured to collect a pre-training data set, wherein the pre-training data set includes a driving scene, a vector description of an object level in the driving scene, a language description including a scene description and questions / answers / scoring / scoring explanations, and driving question-and-answer data, wherein the driving question-and-answer data includes open-ended questions and answer information for each driving scene; A model pre-training subunit, configured to use the vector description and the language description as model inputs and the driving question-and-answer dataset as a model pre-prompt to perform pre-training of the AI ​​question-and-answer model; The model evaluation and optimization subunit is used to score the answer information of the open-ended questions generated by the pre-trained AI question-answering model and optimize it according to the scoring evaluation results.

8. The tire damage detection system based on artificial intelligence according to claim 7, characterized in that: The AI ​​question-answering model includes: a vector encoder comprising a multi-layer perceptron layer and a criss-cross attention layer for converting an input vector in the preprocessed tire image into an input latent vector in a latent space and adding an ego feature to each learned input latent vector; A vectorizer connected to the vector encoder, the vectorizer comprising a self-attention layer and a cross-attention layer for processing the latent space and question tokens to convert the input latent vector into an embedding that can be decoded by a large language model; A frozen large language model with a low-rank adaptive module is connected to the vectorizer and is used to inject a trainable rank decomposition matrix into the linear layer of the pre-trained large language model to obtain the answer information according to the output of the vectorizer and the question raised by the user.

9. A tire damage detection method based on artificial intelligence, characterized in that: Applied to the tire damage detection system based on artificial intelligence as claimed in any one of claims 1 to 8, the method comprising: While the vehicle is driving, the tire image is collected using the vehicle-mounted camera; Preprocessing the tire image to obtain a preprocessed tire image; Inputting the preprocessed tire image into a pre-trained tire detection model to obtain a tire detection result; Display the tire test results to the user and provide corresponding tire damage trend analysis results; The tire image and the tire-related question raised by the user are received, and tire-related answer information is fed back for the tire image and the question based on the AI ​​question-answering model.

10. A vehicle, characterized in that: include: On-board camera for collecting tire images An on-board computing unit, comprising the tire damage detection system based on artificial intelligence according to any one of claims 1 to 8; A user interface is used to display the output results of the tire damage detection system.

Images