Artificial intelligence-based tire damage detection method and related devices

Abstract

The application provides a tire damage detection method and device based on artificial intelligence, an electronic device and a storage medium. The tire damage detection method based on artificial intelligence comprises the following steps: collecting a vehicle image with a component label as a component training set, training a semantic segmentation network based on the component training set to obtain a vehicle component segmentation network; collecting a real-time vehicle image of a target vehicle, inputting the real-time vehicle image into the vehicle component segmentation network to obtain a real-time tire area image of the target vehicle; building a damage detection first network; collecting a tire area image with a damage label as a damage training set, training the damage detection first network based on the damage training set and a preset loss function to obtain a damage detection second network; inputting the real-time tire area image into the damage detection second network to obtain a tire damage detection result, wherein the tire damage detection result comprises area information and damage types of all damage areas in the real-time tire area image. The application can improve the accuracy of tire damage detection.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a tire damage detection method, device, electronic device and storage medium based on artificial intelligence. Background Technology

[0002] During vehicle operation, damage is inevitable due to factors such as weather, road conditions, and driver skill. The assessment of the damaged parts and extent of the damage directly impacts the determination of the subsequent vehicle repair plan and the confirmation of the amount of financial compensation to the parties involved in the accident.

[0003] Currently, existing semantic segmentation networks or target segmentation networks are typically used to process images of damaged vehicles to achieve intelligent damage assessment. However, different vehicle parts have significant differences in features in the images, and this method cannot be refined to specific vehicle parts. In addition, the structure and surface features of vehicle tires are quite complex, resulting in low accuracy in vehicle tire damage detection. Summary of the Invention

[0004] In view of the above, it is necessary to propose an artificial intelligence-based tire damage detection method and related equipment to solve the technical problem of how to improve the accuracy of vehicle tire damage detection. The related equipment includes an artificial intelligence-based tire damage detection device, electronic equipment, and storage medium.

[0005] This application provides an artificial intelligence-based tire damage detection method, the method comprising:

[0006] Vehicle images with component labels are collected as a component training set, and a semantic segmentation network is trained based on the component training set to obtain a vehicle component segmentation network. The component labels include the type of vehicle component for each pixel in the vehicle image.

[0007] Real-time vehicle images of the target vehicle are acquired, and the real-time vehicle images are input into the vehicle component segmentation network to obtain real-time tire region images of the target vehicle.

[0008] Establish a primary network for damage detection;

[0009] Images of tire regions with damage labels are collected as a damage training set, and the damage detection first network is trained based on the damage training set and the preset loss function to obtain a damage detection second network. The damage labels include the regional information and damage type of all damaged regions in the tire region images.

[0010] The real-time tire area image is input into the second damage detection network to obtain the tire damage detection result of the target vehicle.

[0011] In some embodiments, acquiring real-time vehicle images of the target vehicle and inputting the real-time vehicle images into the vehicle component segmentation network to obtain real-time tire region images of the target vehicle includes:

[0012] Real-time vehicle images of the target vehicle are acquired, and the real-time vehicle images are input into the vehicle component segmentation network to obtain a component segmentation map of the real-time vehicle images. The component segmentation map of the real-time vehicle images includes the type of vehicle component to which each pixel in the real-time vehicle images belongs.

[0013] In the component segmentation image of the real-time vehicle image, the pixel value of the vehicle component type of tire component is set to 1, and the pixel value of other areas is set to 0 to obtain the mask image of the tire component. The tire component includes the left front tire, the left rear tire, the right front tire, and the right rear tire.

[0014] The mask image of the tire component is multiplied with the real-time vehicle image to obtain a real-time tire region image of the target vehicle.

[0015] In some embodiments, the damage detection first network is a target detection network, and constructing the damage detection first network includes:

[0016] The input of the first damage detection network is a tire region image, and the output is the damage detection result of the tire region image. The damage detection result includes the region information and damage type of all damaged regions in the tire region image. The region information is the width and height of the rectangle containing the damaged region and the coordinates of the center point. The damage types include missing, perforated missing, cracked, severe damage, and bulge.

[0017] The first network for damage detection includes an encoder and multiple predictors that share parameters;

[0018] The encoder continuously extracts features from the input tire region image to obtain downsampled feature maps, and the number of downsampled feature maps is one or more.

[0019] All downsampled feature maps are input into a predictor with multiple shared parameters to obtain a sub-result for each downsampled feature map, wherein the predictor and the downsampled feature map are used together. Figure 1 In a one-to-one correspondence, the sub-results of the downsampled feature map include the regional information and damage type of all damaged regions in the downsampled feature map;

[0020] The sub-results of all downsampled feature maps are used as the damage detection results of the tire region image.

[0021] In some embodiments, training the first damage detection network based on the damage training set and the preset loss function to obtain the second damage detection network includes:

[0022] A first preset loss function is constructed based on the damage training set and the damage detection first network;

[0023] A preset loss function is constructed based on the first preset loss function and the target detection loss function. The target detection loss function is related to the network structure of the first damage detection network, and the preset loss function satisfies the following relationship:

[0024] Loss = Loss1 + Loss2

[0025] Wherein, Loss1 is the first preset loss function, Loss2 is the target detection loss function, and Loss is the preset loss function;

[0026] The damage detection first network is trained based on the damage training set and the preset loss function to obtain the damage detection second network.

[0027] In some embodiments, constructing a first preset loss function based on the damage training set and the damage detection first network includes:

[0028] A tire region image is randomly selected from the damage training set as the target image, and the damage label of the target image is used as the target damage label. The target damage label includes the region information and damage type of all damaged regions in the target image.

[0029] The target image is input into the first damage detection network to obtain a downsampled feature map of the target image, wherein the downsampled feature map is one or more.

[0030] The target damage label is mapped onto all downsampled feature maps based on the size of the downsampled feature map, resulting in a damage label for each downsampled feature map. The damage label of the downsampled feature map includes the region information and damage type of all damaged regions in the target image in the downsampled feature map.

[0031] Based on the damage labels of the downsampled feature maps, a feature difference matrix is ​​constructed for each downsampled feature map. All feature difference matrices are M-row, M-column square matrices, where M is the number of different damage types in the target image. The value in the i-th row and j-th column of the feature difference matrix of the downsampled feature map represents the feature difference degree between damage type i and damage type j in that downsampled feature map. The feature difference degree satisfies the following relationship:

[0032]

[0033] in, This represents the number of all pixels in the region information of damage type i in the u-th downsampled feature map. This represents the k-th region information of damage type i in the u-th downsampled feature map. i The pixel value of each pixel. This represents the number of all pixels in the region information of damage type j in the u-th downsampled feature map. This represents the k-th region information of damage type j in the u-th downsampled feature map. j The pixel value of each pixel. This represents the feature difference between damage type i and damage type j in the u-th downsampled feature map;

[0034] A first preset loss function is constructed based on the feature difference matrix of all downsampled feature maps and the standard difference matrix of the target image. In the standard difference matrix, the values ​​on the diagonal are 0, and the values ​​at other positions are preset thresholds for the feature difference degree between different damage types in the target image.

[0035] In some embodiments, the step of mapping the target damage label onto all downsampled feature maps based on the size of the downsampled feature map to obtain a damage label for each downsampled feature map, wherein the damage label of the downsampled feature map includes the region information and damage type of all damaged regions in the target image in the downsampled feature map, including:

[0036] The region information and damage type of a damage area are randomly selected from the target damage labels as the target damage information;

[0037] The target damage information is mapped onto all downsampled feature maps based on the mapping model and the size of the downsampled feature maps to obtain the mapping result of the target damage information. The mapping model satisfies the following relationship:

[0038]

[0039] Where, x * ,y * ,w * ,h * The region information representing the target damage information is a rectangular bounding box containing the damaged area, x * ,y * w represents the coordinates of the center point of the region information. * ,h * Z represents the width and height dimensions of the region information. * W represents the damage type of the target damage information. * H * W represents the width and height dimensions of the target image. u H u Let be the width and height dimensions of the u-th downsampled feature map. This indicates that the parameter X is rounded up, (x u ,y u ,w u ,h u ,z u The target damage information is mapped onto the u-th downsampled feature map;

[0040] By traversing the damage information of all damage regions in the target damage label, a mapping result for each damage information is obtained. The damage information includes the region information of the damage region and the damage type.

[0041] All mapping results on the same downsampled feature map are used as the damage labels of the downsampled feature map. The damage labels of the downsampled feature map include the region information and damage type of all damaged regions in the target image in the downsampled feature map.

[0042] In some embodiments, the first preset loss function satisfies the following relationship:

[0043]

[0044] Where n is the number of downsampled feature maps, C u C is the feature difference matrix of the u-th downsampled feature map; * Let C be the standard difference matrix of the target image. u -C * ||0 represents the L0 norm, used to calculate C. u -C * The number of all non-zero values ​​in the table, where Loss1 is the first preset loss function.

[0045] This application embodiment also provides a tire damage detection device based on artificial intelligence, the device comprising:

[0046] The first training unit is used to collect vehicle images with component labels as a component training set, and to train a semantic segmentation network based on the component training set to obtain a vehicle component segmentation network. The component labels include the type of vehicle component for each pixel in the vehicle image.

[0047] The segmentation unit is used to acquire real-time vehicle images of the target vehicle and input the real-time vehicle images into the vehicle component segmentation network to obtain real-time tire region images of the target vehicle.

[0048] A building block is used to build the first network for damage detection.

[0049] The second training unit is used to collect tire region images with damage labels as a damage training set, and to train the first damage detection network based on the damage training set and the preset loss function to obtain the second damage detection network. The damage labels include the region information and damage type of all damaged regions in the tire region image.

[0050] The damage detection unit is used to input the real-time tire area image into the second damage detection network to obtain the tire damage detection result of the target vehicle.

[0051] This application embodiment also provides an electronic device, the electronic device comprising:

[0052] Memory, storing at least one instruction;

[0053] The processor executes the instructions stored in the memory to implement the artificial intelligence-based tire damage detection method.

[0054] This application also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the artificial intelligence-based tire damage detection method.

[0055] In summary, this application utilizes a vehicle component segmentation network to extract tire region images from real-time images of the target vehicle, achieving component-level localization of the tire. Furthermore, the tire region images are input into a second damage detection network to obtain tire damage detection results. The second damage detection network can learn the distinguishing features of different types of damage in the tire region images, thereby improving the accuracy of tire damage detection. Attached Figure Description

[0056] Figure 1 This is a flowchart of a preferred embodiment of the artificial intelligence-based tire damage detection method involved in this application.

[0057] Figure 2 This is a schematic diagram of the structure of the first damage detection network involved in this application.

[0058] Figure 3 This is a functional block diagram of a preferred embodiment of the artificial intelligence-based tire damage detection device involved in this application.

[0059] Figure 4 This is a schematic diagram of the electronic device of a preferred embodiment of the artificial intelligence-based tire damage detection method involved in this application. Detailed Implementation

[0060] To better understand the purpose, features, and advantages of this application, a detailed description of the application is provided below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of this application can be combined with each other. Numerous specific details are set forth in the following description to provide a thorough understanding of this application; the described embodiments are only a part of the embodiments of this application, and not all of them.

[0061] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0063] This application provides an artificial intelligence-based tire damage detection method, which can be applied to one or more electronic devices. An electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0064] Electronic devices can be any electronic product that allows human-computer interaction with a customer, such as personal computers, tablets, smartphones, personal digital assistants (PDAs), game consoles, interactive network television (IPTV), smart wearable devices, etc.

[0065] Electronic devices may also include network devices and / or client devices. The network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0066] The networks in which electronic devices are located include, but are not limited to, the Internet, wide area networks, metropolitan area networks, local area networks, and virtual private networks (VPNs).

[0067] like Figure 1 The diagram shown is a flowchart of a preferred embodiment of the artificial intelligence-based tire damage detection method of this application. Depending on different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0068] S10, collect vehicle images with component labels as a component training set, and train a semantic segmentation network based on the component training set to obtain a vehicle component segmentation network. The component labels include the type of vehicle component for each pixel in the vehicle image.

[0069] In an optional embodiment, a large number of vehicle images are acquired, and component labels are obtained for each vehicle image. The component label is an image of the same size as the vehicle image. The pixel value in the component label represents a preset label for the type of vehicle component at each pixel. The preset label is an integer from 1 to N, where N represents the number of all vehicle component types, including tire components. Each vehicle component type corresponds one-to-one with its preset label. All vehicle images and their component labels are stored as a component training set. The tire components include the left front tire, left rear tire, right front tire, and right rear tire.

[0070] In this optional embodiment, training the semantic segmentation network based on the component training set to obtain the vehicle component segmentation network includes:

[0071] A semantic segmentation network is constructed, which includes an encoder and a decoder;

[0072] The semantic segmentation network is trained based on the component training set to obtain a vehicle component segmentation network. The input of the vehicle component segmentation network is a vehicle image, and the output is a component segmentation map of the vehicle image. The component segmentation map includes the type of vehicle component to which each pixel in the vehicle image belongs. The vehicle components include a variety of vehicle components, including tire components. The tire components include the left front tire, left rear tire, right front tire, and right rear tire.

[0073] In this optional embodiment, a semantic segmentation network is constructed, which consists of an encoder and a decoder connected in series. The input to the semantic segmentation network is a vehicle image. The encoder continuously downsamples the vehicle image to obtain a feature map, which is then input into the decoder. The decoder continuously upsamples the feature map to obtain a component segmentation map. The component segmentation map includes the type of vehicle component to which each pixel in the input vehicle image belongs. The vehicle components include various vehicle components, including tire components. The tire components include the left front tire, left rear tire, right front tire, and right rear tire. The encoder and decoder can adopt the structures of existing semantic segmentation networks such as Deeplab V3+, UNet, and FCN, and this application is not limited to these structures.

[0074] In this optional embodiment, to ensure the accuracy of the component segmentation map output by the semantic segmentation network, the semantic segmentation network needs to be trained based on the component training set. Vehicle images are randomly selected from the component training set as training images; the training images are input into the semantic segmentation network to obtain component segmentation maps, and the cross-entropy loss function is calculated based on the component segmentation maps and the input component labels; after obtaining the value of the cross-entropy loss function, the parameters of the semantic segmentation network are updated using gradient descent; vehicle images are continuously selected from the component training set to update the parameters of the semantic segmentation network; when the value of the cross-entropy loss function no longer changes, training is stopped to obtain the vehicle component segmentation network, which can obtain accurate component segmentation maps.

[0075] Thus, by obtaining a vehicle component segmentation network and inputting a vehicle image into the vehicle component segmentation network, an accurate component segmentation map can be obtained. The component segmentation map can reflect the type of vehicle component to which each pixel in the vehicle image belongs, thereby achieving precise pixel-level classification in the vehicle image.

[0076] S11, acquire real-time vehicle images of the target vehicle, and input the real-time vehicle images into the vehicle component segmentation network to obtain real-time tire region images of the target vehicle.

[0077] In an optional embodiment, acquiring real-time vehicle images of the target vehicle and inputting the real-time vehicle images into the vehicle component segmentation network to obtain real-time tire region images of the target vehicle includes:

[0078] Real-time vehicle images of the target vehicle are acquired, and the real-time vehicle images are input into the vehicle component segmentation network to obtain a component segmentation map of the real-time vehicle images. The component segmentation map of the real-time vehicle images includes the type of vehicle component to which each pixel in the real-time vehicle images belongs.

[0079] In the component segmentation image of the real-time vehicle image, the pixel value of the vehicle component type of tire component is set to 1, and the pixel value of other areas is set to 0 to obtain the mask image of the tire component. The tire component includes the left front tire, the left rear tire, the right front tire, and the right rear tire.

[0080] The mask image of the tire component is multiplied with the real-time vehicle image to obtain a real-time tire region image of the target vehicle.

[0081] Thus, by using a vehicle component segmentation network to achieve component-level localization, real-time tire region images can be obtained from real-time vehicle images of the target vehicle, providing a data foundation for subsequent tire damage detection.

[0082] S12, build the first network for damage detection.

[0083] In an optional embodiment, the damage detection first network is a target detection network, and the construction of the damage detection first network includes:

[0084] The input of the first damage detection network is a tire region image, and the output is the damage detection result of the tire region image. The damage detection result includes the region information and damage type of all damaged regions in the tire region image. The region information is the width and height dimensions and center point coordinates of the rectangle containing the damaged region. The damage types include missing, perforated missing, cracked, severe damage, and bulge.

[0085] The first network for damage detection includes an encoder and multiple predictors that share parameters;

[0086] The encoder continuously extracts features from the input tire region image to obtain downsampled feature maps, and the number of downsampled feature maps is one or more.

[0087] All downsampled feature maps are input into a predictor with multiple shared parameters to obtain a sub-result for each downsampled feature map, wherein the predictor and the downsampled feature map are used together. Figure 1 In a one-to-one correspondence, the sub-results of the downsampled feature map include the regional information and damage type of all damaged regions in the downsampled feature map;

[0088] The sub-results of all downsampled feature maps are used as the damage detection results of the tire region image.

[0089] The different downsampled feature maps have different sizes to ensure that damage of different sizes can be detected in the tire region image. The number and size of the downsampled feature maps are determined by the encoder structure. Taking a damage detection first network with one encoder and n predictors sharing parameters as an example, the structural diagram of the damage detection first network is shown below. Figure 2 As shown.

[0090] It should be noted that the encoder and predictor can adopt any structure of existing object detection networks such as RetinaNet, RefineDet, and CenterNet, and this application does not impose any restrictions.

[0091] For example, assuming the damage detection first network adopts the structure of the RetinaNet target detection network, the encoder obtains 5 downsampled feature maps, and the sizes of the 5 downsampled feature maps are 1 / 8, 1 / 16, 1 / 32, 1 / 64, and 1 / 128 of the size of the input tire region image, respectively. The downsampled feature maps of different sizes can ensure the detection of damage of different sizes in the input tire region image. If the size of the input tire region image is 640×640, the sizes of the 5 downsampled feature maps are 80×80, 40×40, 20×20, 10×10, and 5×5, respectively. The 5 downsampled feature maps are input into 5 predictors with shared parameters to obtain sub-results of downsampled feature maps of different sizes. All sub-results of downsampled feature maps are used as the damage detection result of the input tire region image.

[0092] Thus, the construction of the first damage detection network is completed, providing a network foundation for the subsequent acquisition of the second damage detection network.

[0093] S13, collect tire region images with damage labels as damage training sets, and train the first damage detection network based on the damage training set and the preset loss function to obtain the second damage detection network. The damage labels include the region information and damage type of all damaged regions in the tire region images.

[0094] In an optional embodiment, to ensure the accuracy of the damage detection results output by the first damage detection network, the first damage detection network needs to be trained. Before training the first damage detection network, a large number of tire region images with damage labels are collected as a damage training set. The tire region images can be obtained using the vehicle component segmentation network. The damage labels include the region information and damage type of all damaged regions in the tire region images. The region information is the coordinates of the center point and the width and height dimensions of the rectangle containing the damaged region.

[0095] In an optional embodiment, training the first damage detection network based on the damage training set and the preset loss function to obtain the second damage detection network includes:

[0096] A first preset loss function is constructed based on the damage training set and the damage detection first network;

[0097] A preset loss function is constructed based on the first preset loss function and the target detection loss function, wherein the target detection loss function is related to the network structure of the first damage detection network.

[0098] The damage detection first network is trained based on the damage training set and the preset loss function to obtain the damage detection second network.

[0099] In this optional embodiment, since the surface features of the tire are complex and different types of damage are easily confused, in order to constrain the first damage detection network to learn the distinguishing features between different types of damage and ensure the accuracy of tire damage detection, this application constructs a first preset loss function based on the damage training set and the first damage detection network.

[0100] In an optional embodiment, constructing a first preset loss function based on the damage training set and the damage detection first network includes:

[0101] A tire region image is randomly selected from the damage training set as the target image, and the damage label of the target image is used as the target damage label. The target damage label includes the region information and damage type of all damaged regions in the target image.

[0102] The target image is input into the first damage detection network to obtain a downsampled feature map of the target image, wherein the downsampled feature map is one or more.

[0103] The target damage label is mapped onto all downsampled feature maps based on the size of the downsampled feature map, resulting in a damage label for each downsampled feature map. The damage label of the downsampled feature map includes the region information and damage type of all damaged regions in the target image in the downsampled feature map.

[0104] Based on the damage labels of the downsampled feature maps, a feature difference matrix is ​​constructed for each downsampled feature map. All feature difference matrices are M-row, M-column square matrices, where M is the number of different damage types in the target image. The value in the i-th row and j-th column of the feature difference matrix of the downsampled feature map represents the feature difference degree between damage type i and damage type j in that downsampled feature map. The feature difference degree satisfies the following relationship:

[0105]

[0106] in, This represents the number of all pixels in the region information of damage type i in the u-th downsampled feature map. This represents the k-th region information of damage type i in the u-th downsampled feature map. i The pixel value of each pixel. This represents the number of all pixels in the region information of damage type j in the u-th downsampled feature map. This represents the k-th region information of damage type j in the u-th downsampled feature map. j The pixel value of each pixel. This represents the feature difference between damage type i and damage type j in the u-th downsampled feature map;

[0107] A first preset loss function is constructed based on the feature difference matrix of all downsampled feature maps and the standard difference matrix of the target image. In the standard difference matrix, the values ​​on the diagonal are 0, and the values ​​at other positions are preset thresholds for the feature difference degree between different damage types in the target image.

[0108] In this optional embodiment, the step of mapping the target damage label onto all downsampled feature maps based on the size of the downsampled feature map to obtain a damage label for each downsampled feature map, wherein the damage label of the downsampled feature map includes the region information and damage type of all damaged regions in the target image in the downsampled feature map, including:

[0109] The region information and damage type of a damage area are randomly selected from the target damage labels as the target damage information;

[0110] The target damage information is mapped onto all downsampled feature maps based on the mapping model and the size of the downsampled feature maps to obtain the mapping result of the target damage information. The mapping model satisfies the following relationship:

[0111]

[0112] Where, x * ,y * ,w * ,h * The region information representing the target damage information is a rectangular bounding box containing the damaged area, x * ,y * w represents the coordinates of the center point of the region information. * ,h * Z represents the width and height dimensions of the region information. * W represents the damage type of the target damage information. * H * W represents the width and height dimensions of the target image. u H u Let be the width and height dimensions of the u-th downsampled feature map. This indicates that the parameter X is rounded up, (x u ,y u ,w u ,h u,z u The target damage information is mapped onto the u-th downsampled feature map;

[0113] By traversing the damage information of all damage regions in the target damage label, a mapping result for each damage information is obtained. The damage information includes the region information of the damage region and the damage type.

[0114] All mapping results on the same downsampled feature map are used as the damage labels of the downsampled feature map. The damage labels of the downsampled feature map include the region information and damage type of all damaged regions in the target image in the downsampled feature map.

[0115] In this optional embodiment, the first preset loss function satisfies the following relationship:

[0116]

[0117] Where n is the number of downsampled feature maps, C u C is the feature difference matrix of the u-th downsampled feature map; * Let C be the standard difference matrix of the target image. u -C * ||0 represents the L0 norm, used to calculate C. u -C * The number of all non-zero values ​​in the matrix, where Loss1 is the first preset loss function. The standard difference matrix C of the target image is... * It is an M x M square matrix, where M is the number of different damage types in the target image, in the standard difference matrix C * In the image, the diagonal values ​​are 0, and the values ​​at other positions are preset thresholds for the feature differences between different damage types in the target image. The preset threshold is 5. The first preset loss function is used to constrain the feature differences between different damage types in all downsampled feature maps to be no less than the preset threshold, thereby ensuring the accuracy of damage detection.

[0118] In this optional embodiment, a preset loss function is constructed based on the first preset loss function and the target detection loss function, and the preset loss function satisfies the following relationship:

[0119] Loss = Loss1 + Loss2

[0120] Wherein, Loss1 is the first preset loss function, Loss2 is the target detection loss function, and the target detection loss function is related to the network structure of the first damage detection network. Loss is the preset loss function. For example, assuming that the first damage detection network adopts the structure of the target detection network RetinaNet, then the target detection loss function is the loss function of the target detection network RetinaNet.

[0121] In this optional embodiment, the damage detection first network is trained based on the damage training set and the preset loss function to obtain the damage detection second network. Tire region images are continuously selected from the damage training set and input into the damage detection first network to calculate the value of the preset loss function. The parameters of the damage detection first network are continuously updated using the gradient descent method. When the value of the preset loss function no longer changes, training is stopped to obtain the damage detection second network. The damage detection second network can learn the distinguishing features of different damage types in the tire region image and obtain accurate damage detection results.

[0122] Thus, by using the damage training set and a preset loss function, the first damage detection network is trained to obtain the second damage detection network. The second damage detection network can learn the distinguishing features of different damage types in the tire area image and obtain accurate damage detection results.

[0123] S14, input the real-time tire area image into the second damage detection network to obtain the tire damage detection result of the target vehicle.

[0124] In an optional embodiment, the real-time tire region image is input into the second damage detection network to obtain the tire damage detection result of the target vehicle. The tire damage detection result includes the region information and damage type of all damaged areas in the real-time tire region image.

[0125] Thus, by using the second damage detection network, the tire damage detection results of the target vehicle are obtained, improving the accuracy and speed of tire damage detection.

[0126] As can be seen from the above technical solutions, this application utilizes a vehicle component segmentation network to extract tire region images from real-time images of the target vehicle, thereby achieving component-level localization of the tire. Furthermore, the tire region images are input into a second damage detection network to obtain tire damage detection results. The second damage detection network can learn the distinguishing features of different types of damage in the tire region images, thereby improving the accuracy of tire damage detection.

[0127] Please see Figure 3 , Figure 3This is a functional block diagram of a preferred embodiment of the artificial intelligence-based tire damage detection device of this application. The artificial intelligence-based tire damage detection device 11 includes a first training unit 110, a segmentation unit 111, a construction unit 112, a second training unit 113, and a damage detection unit 114. The module / unit referred to in this application refers to a series of computer-readable instruction segments that can be executed by the processor 13 and perform a fixed function, and are stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0128] In an optional embodiment, the first training unit 110 is used to collect vehicle images with component labels as a component training set, and to train a semantic segmentation network based on the component training set to obtain a vehicle component segmentation network. The component labels include the type of vehicle component for each pixel in the vehicle image.

[0129] In an optional embodiment, a large number of vehicle images are acquired, and component labels are obtained for each vehicle image. The component label is an image of the same size as the vehicle image. The pixel value in the component label represents a preset label for the type of vehicle component at each pixel. The preset label is an integer from 1 to N, where N represents the number of all vehicle component types, including tire components. Each vehicle component type corresponds one-to-one with its preset label. All vehicle images and their component labels are stored as a component training set. The tire components include the left front tire, left rear tire, right front tire, and right rear tire.

[0130] In this optional embodiment, training the semantic segmentation network based on the component training set to obtain the vehicle component segmentation network includes:

[0131] A semantic segmentation network is constructed, which includes an encoder and a decoder;

[0132] The semantic segmentation network is trained based on the component training set to obtain a vehicle component segmentation network. The input of the vehicle component segmentation network is a vehicle image, and the output is a component segmentation map of the vehicle image. The component segmentation map includes the type of vehicle component to which each pixel in the vehicle image belongs. The vehicle components include a variety of vehicle components, including tire components. The tire components include the left front tire, left rear tire, right front tire, and right rear tire.

[0133] In this optional embodiment, a semantic segmentation network is constructed, which consists of an encoder and a decoder connected in series. The input to the semantic segmentation network is a vehicle image. The encoder continuously downsamples the vehicle image to obtain a feature map, which is then input into the decoder. The decoder continuously upsamples the feature map to obtain a component segmentation map. The component segmentation map includes the type of vehicle component to which each pixel in the input vehicle image belongs. The vehicle components include various vehicle components, including tire components. The tire components include the left front tire, left rear tire, right front tire, and right rear tire. The encoder and decoder can adopt the structures of existing semantic segmentation networks such as Deeplab V3+, UNet, and FCN, and this application is not limited to these structures.

[0134] In this optional embodiment, to ensure the accuracy of the component segmentation map output by the semantic segmentation network, the semantic segmentation network needs to be trained based on the component training set. Vehicle images are randomly selected from the component training set as training images; the training images are input into the semantic segmentation network to obtain component segmentation maps, and the cross-entropy loss function is calculated based on the component segmentation maps and the input component labels; after obtaining the value of the cross-entropy loss function, the parameters of the semantic segmentation network are updated using gradient descent; vehicle images are continuously selected from the component training set to update the parameters of the semantic segmentation network; when the value of the cross-entropy loss function no longer changes, training is stopped to obtain the vehicle component segmentation network, which can obtain accurate component segmentation maps.

[0135] In an optional embodiment, the segmentation unit 111 is used to acquire real-time vehicle images of the target vehicle and input the real-time vehicle images into the vehicle component segmentation network to obtain real-time tire region images of the target vehicle.

[0136] In an optional embodiment, acquiring real-time vehicle images of the target vehicle and inputting the real-time vehicle images into the vehicle component segmentation network to obtain real-time tire region images of the target vehicle includes:

[0137] Real-time vehicle images of the target vehicle are acquired, and the real-time vehicle images are input into the vehicle component segmentation network to obtain a component segmentation map of the real-time vehicle images. The component segmentation map of the real-time vehicle images includes the type of vehicle component to which each pixel in the real-time vehicle images belongs.

[0138] In the component segmentation image of the real-time vehicle image, the pixel value of the vehicle component type of tire component is set to 1, and the pixel value of other areas is set to 0 to obtain the mask image of the tire component. The tire component includes the left front tire, the left rear tire, the right front tire, and the right rear tire.

[0139] The mask image of the tire component is multiplied with the real-time vehicle image to obtain a real-time tire region image of the target vehicle.

[0140] In an optional embodiment, the construction unit 112 is used to construct a first damage detection network.

[0141] In an optional embodiment, the damage detection first network is a target detection network, and the construction of the damage detection first network includes:

[0142] The input of the first damage detection network is a tire region image, and the output is the damage detection result of the tire region image. The damage detection result includes the region information and damage type of all damaged regions in the tire region image. The region information is the width and height dimensions and center point coordinates of the rectangle containing the damaged region. The damage types include missing, perforated missing, cracked, severe damage, and bulge.

[0143] The first network for damage detection includes an encoder and multiple predictors that share parameters;

[0144] The encoder continuously extracts features from the input tire region image to obtain downsampled feature maps, and the number of downsampled feature maps is one or more.

[0145] All downsampled feature maps are input into a predictor with multiple shared parameters to obtain a sub-result for each downsampled feature map, wherein the predictor and the downsampled feature map are used together. Figure 1 In a one-to-one correspondence, the sub-results of the downsampled feature map include the regional information and damage type of all damaged regions in the downsampled feature map;

[0146] The sub-results of all downsampled feature maps are used as the damage detection results of the tire region image.

[0147] The different downsampled feature maps have different sizes to ensure that damage of different sizes can be detected in the tire region image. The number and size of the downsampled feature maps are determined by the encoder structure. Taking a damage detection first network with one encoder and n predictors sharing parameters as an example, the structural diagram of the damage detection first network is shown below. Figure 2 As shown.

[0148] It should be noted that the encoder and predictor can adopt any structure of existing object detection networks such as RetinaNet, RefineDet, and CenterNet, and this application does not impose any restrictions.

[0149] For example, assuming the damage detection first network adopts the structure of the RetinaNet target detection network, the encoder obtains 5 downsampled feature maps, and the sizes of the 5 downsampled feature maps are 1 / 8, 1 / 16, 1 / 32, 1 / 64, and 1 / 128 of the size of the input tire region image, respectively. The downsampled feature maps of different sizes can ensure the detection of damage of different sizes in the input tire region image. If the size of the input tire region image is 640×640, the sizes of the 5 downsampled feature maps are 80×80, 40×40, 20×20, 10×10, and 5×5, respectively. The 5 downsampled feature maps are input into 5 predictors with shared parameters to obtain sub-results of downsampled feature maps of different sizes. All sub-results of downsampled feature maps are used as the damage detection result of the input tire region image.

[0150] In an optional embodiment, the second training unit 113 is used to collect tire region images with damage labels as a damage training set, and train the damage detection first network based on the damage training set and the preset loss function to obtain a damage detection second network. The damage labels include the region information and damage type of all damaged regions in the tire region image.

[0151] In an optional embodiment, to ensure the accuracy of the damage detection results output by the first damage detection network, the first damage detection network needs to be trained. Before training the first damage detection network, a large number of tire region images with damage labels are collected as a damage training set. The tire region images can be obtained using the vehicle component segmentation network. The damage labels include the region information and damage type of all damaged regions in the tire region images. The region information is the coordinates of the center point and the width and height dimensions of the rectangle containing the damaged region.

[0152] In an optional embodiment, training the first damage detection network based on the damage training set and the preset loss function to obtain the second damage detection network includes:

[0153] A first preset loss function is constructed based on the damage training set and the damage detection first network;

[0154] A preset loss function is constructed based on the first preset loss function and the target detection loss function, wherein the target detection loss function is related to the network structure of the first damage detection network.

[0155] The damage detection first network is trained based on the damage training set and the preset loss function to obtain the damage detection second network.

[0156] In this optional embodiment, since the surface features of the tire are complex and different types of damage are easily confused, in order to constrain the first damage detection network to learn the distinguishing features between different types of damage and ensure the accuracy of tire damage detection, this application constructs a first preset loss function based on the damage training set and the first damage detection network.

[0157] In an optional embodiment, constructing a first preset loss function based on the damage training set and the damage detection first network includes:

[0158] A tire region image is randomly selected from the damage training set as the target image, and the damage label of the target image is used as the target damage label. The target damage label includes the region information and damage type of all damaged regions in the target image.

[0159] The target image is input into the first damage detection network to obtain a downsampled feature map of the target image, wherein the downsampled feature map is one or more.

[0160] The target damage label is mapped onto all downsampled feature maps based on the size of the downsampled feature map, resulting in a damage label for each downsampled feature map. The damage label of the downsampled feature map includes the region information and damage type of all damaged regions in the target image in the downsampled feature map.

[0161] Based on the damage labels of the downsampled feature maps, a feature difference matrix is ​​constructed for each downsampled feature map. All feature difference matrices are M-row, M-column square matrices, where M is the number of different damage types in the target image. The value in the i-th row and j-th column of the feature difference matrix of the downsampled feature map represents the feature difference degree between damage type i and damage type j in that downsampled feature map. The feature difference degree satisfies the following relationship:

[0162]

[0163] in, This represents the number of all pixels in the region information of damage type i in the u-th downsampled feature map. This represents the k-th region information of damage type i in the u-th downsampled feature map. i The pixel value of each pixel. This represents the number of all pixels in the region information of damage type j in the u-th downsampled feature map. This represents the k-th region information of damage type j in the u-th downsampled feature map. j The pixel value of each pixel. This represents the feature difference between damage type i and damage type j in the u-th downsampled feature map;

[0164] A first preset loss function is constructed based on the feature difference matrix of all downsampled feature maps and the standard difference matrix of the target image. In the standard difference matrix, the values ​​on the diagonal are 0, and the values ​​at other positions are preset thresholds for the feature difference degree between different damage types in the target image.

[0165] In this optional embodiment, the step of mapping the target damage label onto all downsampled feature maps based on the size of the downsampled feature map to obtain a damage label for each downsampled feature map, wherein the damage label of the downsampled feature map includes the region information and damage type of all damaged regions in the target image in the downsampled feature map, including:

[0166] The region information and damage type of a damage area are randomly selected from the target damage labels as the target damage information;

[0167] The target damage information is mapped onto all downsampled feature maps based on the mapping model and the size of the downsampled feature maps to obtain the mapping result of the target damage information. The mapping model satisfies the following relationship:

[0168]

[0169] Where, x * ,y * ,w * ,h * The region information representing the target damage information is a rectangular bounding box containing the damaged area, x * ,y * w represents the coordinates of the center point of the region information. * ,h * Z represents the width and height dimensions of the region information. * W represents the damage type of the target damage information. * H * W represents the width and height dimensions of the target image. u H u Let be the width and height dimensions of the u-th downsampled feature map. This indicates that the parameter X is rounded up, (x u ,y u ,w u ,h u ,z u The target damage information is mapped onto the u-th downsampled feature map;

[0170] By traversing the damage information of all damage regions in the target damage label, a mapping result for each damage information is obtained. The damage information includes the region information of the damage region and the damage type.

[0171] All mapping results on the same downsampled feature map are used as the damage labels of the downsampled feature map. The damage labels of the downsampled feature map include the region information and damage type of all damaged regions in the target image in the downsampled feature map.

[0172] In this optional embodiment, the first preset loss function satisfies the following relationship:

[0173]

[0174] Where n is the number of downsampled feature maps, C u C is the feature difference matrix of the u-th downsampled feature map; * Let C be the standard difference matrix of the target image. u -C * ||0 represents the L0 norm, used to calculate C. u -C * The number of all non-zero values ​​in the matrix, where Loss1 is the first preset loss function. The standard difference matrix C of the target image is... * It is an M x M square matrix, where M is the number of different damage types in the target image, in the standard difference matrix C * In the image, the diagonal values ​​are 0, and the values ​​at other positions are preset thresholds for the feature differences between different damage types in the target image. The preset threshold is 5. The first preset loss function is used to constrain the feature differences between different damage types in all downsampled feature maps to be no less than the preset threshold, thereby ensuring the accuracy of damage detection.

[0175] In this optional embodiment, a preset loss function is constructed based on the first preset loss function and the target detection loss function, and the preset loss function satisfies the following relationship:

[0176] Loss = Loss1 + Loss2

[0177] Wherein, Loss1 is the first preset loss function, Loss2 is the target detection loss function, and the target detection loss function is related to the network structure of the first damage detection network. Loss is the preset loss function. For example, assuming that the first damage detection network adopts the structure of the target detection network RetinaNet, then the target detection loss function is the loss function of the target detection network RetinaNet.

[0178] In this optional embodiment, the damage detection first network is trained based on the damage training set and the preset loss function to obtain the damage detection second network. Tire region images are continuously selected from the damage training set and input into the damage detection first network to calculate the value of the preset loss function. The parameters of the damage detection first network are continuously updated using the gradient descent method. When the value of the preset loss function no longer changes, training is stopped to obtain the damage detection second network. The damage detection second network can learn the distinguishing features of different damage types in the tire region image and obtain accurate damage detection results.

[0179] In an optional embodiment, the damage detection unit 114 is used to input the real-time tire area image into the second damage detection network to obtain the tire damage detection result of the target vehicle.

[0180] In an optional embodiment, the real-time tire region image is input into the second damage detection network to obtain the tire damage detection result of the target vehicle. The tire damage detection result includes the region information and damage type of all damaged areas in the real-time tire region image.

[0181] As can be seen from the above technical solutions, this application utilizes a vehicle component segmentation network to extract tire region images from real-time images of the target vehicle, thereby achieving component-level localization of the tire. Furthermore, the tire region images are input into a second damage detection network to obtain tire damage detection results. The second damage detection network can learn the distinguishing features of different types of damage in the tire region images, thereby improving the accuracy of tire damage detection.

[0182] Please see Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 1 includes a memory 12 and a processor 13. The memory 12 is used to store computer-readable instructions, and the processor 13 executes the computer-readable instructions stored in the memory to implement the artificial intelligence-based tire damage detection method described in any of the above embodiments.

[0183] In an alternative embodiment, the electronic device 1 further includes a bus and a computer program stored in the memory 12 and executable on the processor 13, such as an AI-based tire damage detection program.

[0184] Figure 4 Only electronic device 1 with memory 12 and processor 13 is shown. It will be understood by those skilled in the art that... Figure 4 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0185] Combination Figure 1 The memory 12 in the electronic device 1 stores a plurality of computer-readable instructions to implement an artificial intelligence-based tire damage detection method, and the processor 13 can execute the plurality of instructions to achieve:

[0186] Vehicle images with component labels are collected as a component training set, and a semantic segmentation network is trained based on the component training set to obtain a vehicle component segmentation network. The component labels include the type of vehicle component for each pixel in the vehicle image.

[0187] Real-time vehicle images of the target vehicle are acquired, and the real-time vehicle images are input into the vehicle component segmentation network to obtain real-time tire region images of the target vehicle.

[0188] Establish a primary network for damage detection;

[0189] Images of tire regions with damage labels are collected as a damage training set, and the damage detection first network is trained based on the damage training set and the preset loss function to obtain a damage detection second network. The damage labels include the regional information and damage type of all damaged regions in the tire region images.

[0190] The real-time tire area image is input into the second damage detection network to obtain the tire damage detection result of the target vehicle.

[0191] Specifically, the processor 13's implementation method for the above instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0192] Those skilled in the art will understand that the schematic diagram is merely an example of electronic device 1 and does not constitute a limitation on electronic device 1. Electronic device 1 can be a bus-type structure or a star-type structure. Electronic device 1 may also include more or fewer other hardware or software than shown in the diagram, or different component arrangements. For example, electronic device 1 may also include input / output devices, network access devices, etc.

[0193] It should be noted that electronic device 1 is only an example. Other existing or future electronic products that are suitable for this application should also be included within the scope of protection of this application and are incorporated herein by reference.

[0194] The memory 12 includes at least one type of readable storage medium, which can be non-volatile or volatile. The readable storage medium includes flash memory, portable hard drives, multimedia cards, card-type memory (e.g., SD or DX memory), magnetic storage, magnetic disks, optical disks, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 1, such as a portable hard drive of the electronic device 1. In other embodiments, the memory 12 can also be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 1. The memory 12 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of an AI-based tire damage detection program, but also to temporarily store data that has been output or will be output.

[0195] In some embodiments, the processor 13 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 13 is the control unit of the electronic device 1, connecting various components of the electronic device 1 via various interfaces and lines. It executes programs or modules stored in the memory 12 (e.g., executing an AI-based tire damage detection program) and calls data stored in the memory 12 to perform various functions and process data in the electronic device 1.

[0196] The processor 13 executes the operating system of the electronic device 1 and various installed applications. The processor 13 executes these applications to implement the steps in the various embodiments of the artificial intelligence-based tire damage detection method described above, for example... Figure 1 The steps are shown.

[0197] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete this application. The one or more modules / units may be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into a first training unit 110, a segmentation unit 111, a construction unit 112, a second training unit 113, and a damage detection unit 114.

[0198] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, computer equipment, or network device, etc.) or processor to execute portions of the artificial intelligence-based tire damage detection method described in the various embodiments of this application.

[0199] If the modules / units integrated in electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware devices. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above.

[0200] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, and other memory.

[0201] Furthermore, the computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of blockchain nodes, etc.

[0202] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0203] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, in... Figure 4 The symbol is represented by only one arrow, but this does not indicate that there is only one bus or one type of bus. The bus is configured to enable communication between the memory 12 and at least one processor 13, etc.

[0204] This application also provides a computer-readable storage medium (not shown) storing computer-readable instructions, which are executed by a processor in an electronic device to implement the artificial intelligence-based tire damage detection method described in any of the above embodiments.

[0205] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0206] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0207] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0208] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in the specification may also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0209] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A tire damage detection method based on artificial intelligence, characterized in that, The method includes: Vehicle images with component labels are collected as a component training set, and a semantic segmentation network is trained based on the component training set to obtain a vehicle component segmentation network. The component labels include the type of vehicle component for each pixel in the vehicle image. Real-time vehicle images of the target vehicle are acquired, and the real-time vehicle images are input into the vehicle component segmentation network to obtain real-time tire region images of the target vehicle. Establish a primary network for damage detection; Images of tire regions with damage labels are collected as a damage training set. A second damage detection network is then trained based on the damage training set and a preset loss function. The damage labels include regional information and damage types of all damaged regions in the tire region images. Constructing a first preset loss function based on the damage training set and the first damage detection network includes: randomly selecting a tire region image from the damage training set as a target image; using the damage label of the target image as a target damage label, which includes regional information and damage types of all damaged regions in the target image; inputting the target image into the first damage detection network to obtain a downsampled feature map of the target image, where the downsampled feature map can be one or more; and adjusting the loss function according to the size of the downsampled feature map. The target damage label is mapped onto all downsampled feature maps to obtain a damage label for each downsampled feature map. The damage label of each downsampled feature map includes the region information and damage type of all damaged areas in the target image in that downsampled feature map. A feature difference matrix is ​​constructed for each downsampled feature map based on the damage labels of the downsampled feature maps. A first preset loss function is constructed based on the feature difference matrices of all downsampled feature maps and the standard difference matrix of the target image. In the standard difference matrix, the values ​​on the diagonal are 0, and the values ​​in other positions are preset thresholds for the feature difference degree between different damage types in the target image. The preset loss function is constructed based on the first preset loss function and the target detection loss function. The target detection loss function is related to the network structure of the damage detection first network. The real-time tire area image is input into the second damage detection network to obtain the tire damage detection result of the target vehicle.

2. The tire damage detection method based on artificial intelligence as described in claim 1, characterized in that, The process of acquiring real-time vehicle images of the target vehicle and inputting these images into the vehicle component segmentation network to obtain real-time tire region images of the target vehicle includes: Real-time vehicle images of the target vehicle are acquired, and the real-time vehicle images are input into the vehicle component segmentation network to obtain a component segmentation map of the real-time vehicle images. The component segmentation map of the real-time vehicle images includes the type of vehicle component to which each pixel in the real-time vehicle images belongs. In the component segmentation image of the real-time vehicle image, the pixel value of the vehicle component type of tire component is set to 1, and the pixel value of other areas is set to 0 to obtain the mask image of the tire component. The tire component includes the left front tire, the left rear tire, the right front tire, and the right rear tire. The mask image of the tire component is multiplied with the real-time vehicle image to obtain a real-time tire region image of the target vehicle.

3. The tire damage detection method based on artificial intelligence as described in claim 1, characterized in that, The damage detection first network is a target detection network, and the construction of the damage detection first network includes: The input of the first damage detection network is a tire region image, and the output is the damage detection result of the tire region image. The damage detection result includes the region information and damage type of all damaged regions in the tire region image. The region information is the width and height of the rectangle containing the damaged region and the coordinates of the center point. The damage types include missing, perforated missing, cracked, severe damage, and bulge. The first network for damage detection includes an encoder and multiple predictors that share parameters; The encoder continuously extracts features from the input tire region image to obtain downsampled feature maps, and the number of downsampled feature maps is one or more. All downsampled feature maps are input into multiple predictors that share parameters to obtain sub-results for each downsampled feature map, wherein each predictor corresponds one-to-one with the downsampled feature map, and the sub-results of the downsampled feature map include the regional information and damage type of all damaged areas in the downsampled feature map; The sub-results of all downsampled feature maps are used as the damage detection results of the tire region image.

4. The tire damage detection method based on artificial intelligence as described in claim 1, characterized in that, The preset loss function satisfies the following relationship: in, For the first preset loss function, Let be the target detection loss function. This is the preset loss function.

5. The tire damage detection method based on artificial intelligence as described in claim 4, characterized in that, All feature difference matrices are M x M square matrices, where M is the number of different damage types in the target image. The value in the i-th row and j-th column of the feature difference matrix of the downsampled feature map represents the feature difference degree between damage type i and damage type j in the downsampled feature map, and the feature difference degree satisfies the following relationship: in, This represents the number of all pixels in the region information of damage type i in the u-th downsampled feature map. This represents the region information of damage type i in the u-th downsampled feature map. The pixel value of each pixel. This represents the number of all pixels in the region information of damage type j in the u-th downsampled feature map. This represents the region information of damage type j in the u-th downsampled feature map. The pixel value of each pixel. This represents the feature difference between damage type i and damage type j in the u-th downsampled feature map.

6. The tire damage detection method based on artificial intelligence as described in claim 5, characterized in that, The target damage label is mapped onto all downsampled feature maps based on the size of the downsampled feature map, resulting in a damage label for each downsampled feature map. The damage label of each downsampled feature map includes the region information and damage type of all damaged areas in the target image within that downsampled feature map, including: The region information and damage type of a damage area are randomly selected from the target damage labels as the target damage information; The target damage information is mapped onto all downsampled feature maps based on the mapping model and the size of the downsampled feature maps to obtain the mapping result of the target damage information. The mapping model satisfies the following relationship: in, The region information represents the target damage information, and the region information is a rectangular box containing the damaged area. The coordinates of the center point of the region information. The width and height dimensions of the region information. This indicates the type of damage in the target damage information. The width and height dimensions of the target image. Let be the width and height dimensions of the u-th downsampled feature map. This indicates that the parameter X is rounded up. The mapping result of the target damage information onto the u-th downsampled feature map; By traversing the damage information of all damage regions in the target damage label, a mapping result for each damage information is obtained. The damage information includes the region information of the damage region and the damage type. All mapping results on the same downsampled feature map are used as the damage labels of the downsampled feature map. The damage labels of the downsampled feature map include the region information and damage type of all damaged regions in the target image in the downsampled feature map.

7. The tire damage detection method based on artificial intelligence as described in claim 5, characterized in that, The first preset loss function satisfies the following relationship: Where n is the number of downsampled feature maps, Let be the feature difference matrix of the u-th downsampled feature map; The standard difference matrix of the target image is... The L0 norm is used to calculate... The number of all non-zero values ​​in the array. This is the first preset loss function.

8. A tire damage detection device based on artificial intelligence, characterized in that, The device includes: The first training unit is used to collect vehicle images with component labels as a component training set, and to train a semantic segmentation network based on the component training set to obtain a vehicle component segmentation network. The component labels include the type of vehicle component for each pixel in the vehicle image. The segmentation unit is used to acquire real-time vehicle images of the target vehicle and input the real-time vehicle images into the vehicle component segmentation network to obtain real-time tire region images of the target vehicle. A building block is used to build the first network for damage detection. The second training unit is used to collect tire region images with damage labels as a damage training set, and to train the first damage detection network based on the damage training set and a preset loss function to obtain a second damage detection network. The damage labels include the region information and damage type of all damaged regions in the tire region image. The construction of the first preset loss function based on the damage training set and the first damage detection network includes: randomly selecting a tire region image from the damage training set as a target image, and using the damage label of the target image as a target damage label, the target damage label including the region information and damage type of all damaged regions in the target image; inputting the target image into the first damage detection network to obtain a downsampled feature map of the target image, wherein the downsampled feature map is one or more; and then, based on the downsampled feature map... The size of the target damage label is mapped onto all downsampled feature maps to obtain a damage label for each downsampled feature map. The damage label of each downsampled feature map includes the region information and damage type of all damaged regions in the target image in that downsampled feature map. A feature difference matrix is ​​constructed for each downsampled feature map based on the damage labels of the downsampled feature maps. A first preset loss function is constructed based on the feature difference matrices of all downsampled feature maps and the standard difference matrix of the target image. In the standard difference matrix, the values ​​on the diagonal are 0, and the values ​​in other positions are preset thresholds for the feature difference degree between different damage types in the target image. The preset loss function is constructed based on the first preset loss function and the target detection loss function. The target detection loss function is related to the network structure of the damage detection first network. The damage detection unit is used to input the real-time tire area image into the second damage detection network to obtain the tire damage detection result of the target vehicle.

9. An electronic device, characterized in that, The electronic device includes: Memory, which stores computer-readable instructions; and The processor executes computer-readable instructions stored in the memory to implement the artificial intelligence-based tire damage detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the artificial intelligence-based tire damage detection method as described in any one of claims 1 to 7.

Images