Vehicle damage detection method, device, equipment and storage medium

Abstract

The present application relates to artificial intelligence, and provides a vehicle damage detection method, device, equipment and storage medium. The method extracts vehicle damage feature maps from vehicle damage images based on multiple feature network layers, performs cavity convolution processing on the vehicle damage feature maps to obtain convolution feature maps, performs convolution attention processing on the convolution feature maps and the vehicle damage feature maps to obtain attention features and global features, fuses the attention features to obtain local features, fuses the local features and the global features to obtain target features, performs vehicle damage detection on the target features to obtain detection results and result confidence, and generates vehicle damage results accurately according to the detection results and the result confidence. In addition, the present application also relates to blockchain technology, and the vehicle damage results can be stored in the blockchain.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, equipment and storage medium for vehicle damage detection. Background Technology

[0002] With the development of artificial intelligence, automated vehicle damage claims systems have emerged. In these automated systems, when a user uploads images of vehicle damage, the system assesses the damage based on these images and then provides the owner with an appropriate compensation amount according to certain rules. However, limitations such as the actual shooting scene, shooting angle, lighting conditions, the integrity of damaged parts, and occlusion in the images prevent the automated claims system from accurately assessing the extent of the damage to the vehicle. Summary of the Invention

[0003] In view of the above, it is necessary to provide a vehicle damage detection method, apparatus, equipment, and storage medium that can solve the technical problem of how to improve the accuracy of vehicle damage detection.

[0004] On one hand, the present invention proposes a vehicle damage detection method, the vehicle damage detection method comprising:

[0005] Based on multiple feature network layers in the pre-trained feature extraction model, vehicle damage feature maps of each feature network layer are extracted from the acquired vehicle damage images.

[0006] Each vehicle damage feature map is subjected to dilated convolution processing to obtain multiple convolutional feature maps for each vehicle damage feature map;

[0007] Convolutional attention processing is performed on each convolutional feature map to obtain attention features, and convolutional attention processing is performed on each vehicle damage feature map to obtain global features;

[0008] By fusing multiple attention features corresponding to each vehicle damage feature map, local features of each vehicle damage feature map are obtained;

[0009] The local features and the global features are subjected to feature projection fusion processing to obtain the target features of each vehicle damage feature map;

[0010] Based on the pre-trained vehicle damage detection model, vehicle damage is detected on the target features to obtain the detection result of each vehicle damage feature map and the result confidence level corresponding to the detection result;

[0011] The vehicle damage result is generated from the vehicle damage image based on multiple detection results and the confidence level of the results.

[0012] According to a preferred embodiment of the present invention, the step of performing convolutional attention processing on each convolutional feature map to obtain attention features includes:

[0013] Max pooling is performed on the convolutional feature map to obtain the first pooled feature, and average pooling is performed on the convolutional feature map to obtain the second pooled feature.

[0014] The first pooling feature and the second pooling feature are subjected to shared fully connected processing to obtain the first fully connected feature;

[0015] The convolutional feature map is subjected to channel interaction processing to obtain interactive features;

[0016] The interaction features are processed using a fully connected method to obtain a second fully connected feature.

[0017] The first fully connected feature and the second fully connected feature are fused to obtain the fused feature;

[0018] Spatial attention processing is performed on the first fully connected feature, the second fully connected feature, and the fused feature to obtain the attention feature.

[0019] According to a preferred embodiment of the present invention, the step of performing channel interaction processing on the convolutional feature map to obtain interactive features includes:

[0020] The convolutional feature map is grouped and convolved to obtain multiple initial features;

[0021] By concatenating the multiple initial features, a concatenated feature is obtained;

[0022] The splicing features are transposed to obtain transposed features;

[0023] Based on the feature dimension of any initial feature among the multiple initial features, multiple output features are sequentially extracted from the transposed features;

[0024] The interaction features are obtained by concatenating the multiple output features.

[0025] According to a preferred embodiment of the present invention, the spatial attention processing performed on the first fully connected feature, the second fully connected feature, and the fused feature to obtain the attention feature includes:

[0026] The input features are obtained by multiplying the first fully connected feature, the second fully connected feature, and the fused feature.

[0027] The input features are subjected to average pooling to obtain the third pooling feature, and the input features are subjected to max pooling to obtain the fourth pooling feature.

[0028] The third pooling feature and the fourth pooling feature are subjected to convolution and activation processing to obtain the attention feature.

[0029] According to a preferred embodiment of the present invention, the step of performing feature projection fusion processing on the local features and the global features to obtain the target features of each vehicle damage feature map includes:

[0030] Calculate the product feature of the local feature and the global feature;

[0031] The ratio of the product feature to the magnitude of the local feature and the global feature is calculated to obtain the included angle feature;

[0032] Based on the dimension of the global feature, the local feature and the included angle feature are scaled to obtain a first transformed feature corresponding to the local feature and a second transformed feature corresponding to the included angle feature;

[0033] The target feature is obtained by fusing the global feature, the first transformation feature, and the second transformation feature.

[0034] According to a preferred embodiment of the present invention, the vehicle damage detection model includes a feature pyramid network and a prediction network. The vehicle damage detection is performed on the target features based on the pre-trained vehicle damage detection model to obtain the detection result for each vehicle damage feature map and the corresponding confidence level of the detection result, including:

[0035] The target features are upsampled based on the feature pyramid network to obtain upsampled features.

[0036] The upsampled features are detected based on the candidate box detection layer in the prediction network to obtain candidate box features;

[0037] Based on the mapping matrix in the prediction network, the candidate box features are activated and normalized to obtain a probability vector.

[0038] The category corresponding to the element in the probability vector whose value is greater than a preset probability threshold is determined as the detection result, and this element is determined as the confidence level of the result.

[0039] According to a preferred embodiment of the present invention, the plurality of feature network layers include a first network layer, a second network layer, and a third network layer, which are sequentially connected in the feature extraction model. The first network layer includes a convolutional layer with multiple branches. Extracting the vehicle damage feature map of each feature network layer from the acquired vehicle damage image based on the plurality of feature network layers in the pre-trained feature extraction model includes:

[0040] The vehicle damage image is convolved based on the convolutional layer of each branch to obtain the convolutional features of each branch.

[0041] The multiple convolutional features are weighted and fused to obtain the vehicle damage feature map output by the first network layer;

[0042] The vehicle damage feature map output by the first network layer is convolved based on the second network layer to obtain the vehicle damage feature map output by the second network layer.

[0043] The vehicle damage feature map output by the second network layer is obtained by convolution processing based on the third network layer.

[0044] On the other hand, the present invention also proposes a vehicle damage detection device, the vehicle damage detection device comprising:

[0045] The extraction unit is used to extract the vehicle damage feature map of each feature network layer from the acquired vehicle damage image based on multiple feature network layers in the pre-trained feature extraction model.

[0046] The convolutional unit is used to perform dilated convolution on each vehicle damage feature map to obtain multiple convolutional feature maps for each vehicle damage feature map;

[0047] The attention unit is used to perform convolutional attention processing on each convolutional feature map to obtain attention features, and to perform convolutional attention processing on each vehicle damage feature map to obtain global features;

[0048] The fusion unit is used to fuse multiple attention features corresponding to each vehicle damage feature map to obtain the local features of each vehicle damage feature map;

[0049] The fusion unit is also used to perform feature projection fusion processing on the local features and the global features to obtain the target features of each vehicle damage feature map;

[0050] The detection unit is used to perform vehicle damage detection on the target features based on a pre-trained vehicle damage detection model, and obtain the detection result of each vehicle damage feature map and the result confidence level corresponding to the detection result.

[0051] The generation unit is used to generate the vehicle damage result of the vehicle damage image based on multiple detection results and the confidence level of the results.

[0052] On the other hand, the present invention also proposes an electronic device, the electronic device comprising:

[0053] Memory, which stores computer-readable instructions; and

[0054] The processor executes computer-readable instructions stored in the memory to implement the vehicle damage detection method.

[0055] On the other hand, the present invention also proposes a computer-readable storage medium storing computer-readable instructions, which are executed by a processor in an electronic device to implement the vehicle damage detection method.

[0056] As can be seen from the above technical solutions, this application can obtain convolutional feature maps with different receptive fields by performing convolutional processing on the vehicle damage feature map with different porosity. Then, by performing convolutional attention processing and fusion processing on each convolutional feature map, the representation ability of the local features can be enhanced. Thus, the target features of the local features and the global features are fused as the final features for vehicle damage detection, which can effectively improve the feature extraction capability of the vehicle damage image, thereby improving the accuracy of the detection results and the generation of the result confidence. Furthermore, by combining the detection results and result confidence of the vehicle damage feature maps output by the multiple feature network layers, the vehicle damage results can be detected from different feature dimensions, thereby further improving the detection accuracy of the vehicle damage results. Attached Figure Description

[0057] Figure 1 This is a flowchart of a preferred embodiment of the vehicle damage detection method of the present invention.

[0058] Figure 2 This is a schematic diagram of the network framework for attention processing in the vehicle damage detection method of the present invention.

[0059] Figure 3 This is a schematic diagram of the network framework for generating detection results in the vehicle damage detection method of the present invention.

[0060] Figure 4 This is a functional block diagram of a preferred embodiment of the vehicle damage detection device of the present invention.

[0061] Figure 5 This is a schematic diagram of the structure of an electronic device that implements a preferred embodiment of the vehicle damage detection method of the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0063] like Figure 1 The diagram shown is a flowchart of a preferred embodiment of the vehicle damage detection method of the present invention. The order of the steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements.

[0064] The vehicle damage detection method described above can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0065] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0066] The vehicle damage detection method is applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored computer-readable instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0067] The electronic device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0068] The electronic devices may include network devices and / or user devices. The network devices include, but are not limited to, single network electronic devices, groups of multiple network electronic devices, or cloud computing-based systems consisting of a large number of hosts or network electronic devices.

[0069] The network in which the electronic device is located includes, but is not limited to: the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0070] 101. Based on multiple feature network layers in a pre-trained feature extraction model, vehicle damage feature maps of each feature network layer are extracted from the acquired vehicle damage images.

[0071] In at least one embodiment of the present invention, the plurality of feature network layers include a first network layer, a second network layer and a third network layer, wherein the first network layer, the second network layer and the third network layer are sequentially connected in the feature extraction model, and the first network layer includes a convolutional layer with multiple branches.

[0072] The second network layer may include a convolutional network, and the third network layer may include a convolutional network.

[0073] The vehicle damage image can be an image uploaded by the claimant to the auto insurance claims system. When the electronic device detects that an image has been uploaded to the auto insurance claims system, it retrieves the image from the auto insurance claims system as the vehicle damage image.

[0074] The feature dimensions of the multiple vehicle damage feature maps are different.

[0075] In at least one embodiment of the present invention, the electronic device extracts the vehicle damage feature map of each feature network layer from the acquired vehicle damage image based on multiple feature network layers in a pre-trained feature extraction model, including:

[0076] The vehicle damage image is convolved based on the convolutional layer of each branch to obtain the convolutional features of each branch.

[0077] The multiple convolutional features are weighted and fused to obtain the vehicle damage feature map output by the first network layer;

[0078] The vehicle damage feature map output by the first network layer is convolved based on the second network layer to obtain the vehicle damage feature map output by the second network layer.

[0079] The vehicle damage feature map output by the second network layer is obtained by convolution processing based on the third network layer.

[0080] For example, the convolutional features of branch A are: The convolutional features of branch B are The convolutional features of branch C are If the branch weight ratio of branch A, branch B, and branch C is 3:5:2, then the multiple convolutional features are weighted and fused to obtain the vehicle damage feature map output by the first network layer.

[0081] By simultaneously performing convolution processing on the vehicle damage image using multiple branch convolutional layers, the generation efficiency of the convolutional features can be improved, thereby increasing the generation efficiency of the vehicle damage feature map. Furthermore, by weighting and fusing multiple convolutional features, the generation accuracy of the vehicle damage feature map can be improved.

[0082] 102. Perform dilated convolution on each vehicle damage feature map to obtain multiple convolutional feature maps for each vehicle damage feature map.

[0083] In at least one embodiment of the present invention, the multiple convolutional feature maps of each vehicle damage feature map are obtained by the electronic device performing convolution processing on the vehicle damage feature map using multiple convolutional layers with different dilation rates.

[0084] In at least one embodiment of the present invention, the electronic device performs dilated convolution processing on each vehicle damage feature map to obtain multiple convolutional feature maps for each vehicle damage feature map, including:

[0085] Based on multiple different dilation rates, dilated convolution processing is performed on each vehicle damage feature map to obtain multiple convolutional feature maps of each vehicle damage feature map on the multiple different dilation rates.

[0086] The multiple different void ratios may include, but are not limited to, 2, 4, 8, etc.

[0087] By using the multiple different hole rates, convolutional feature maps with different receptive fields can be obtained.

[0088] 103. Perform convolutional attention processing on each convolutional feature map to obtain attention features, and perform convolutional attention processing on each vehicle damage feature map to obtain global features.

[0089] In at least one embodiment of the present invention, the attention feature refers to the convolutional feature map processed as follows: Figure 2 The features generated after the attention processing network framework shown, wherein the global features refer to the vehicle damage feature map after processing as follows: Figure 2 The features generated after the attention processing network framework shown.

[0090] exist Figure 2 The network framework for attention processing shown includes max pooling network, average pooling network, channel interaction network, fully connected network 1 and fully connected network 2, fusion network and spatial attention network.

[0091] In at least one embodiment of the present invention, the electronic device performs convolutional attention processing on each convolutional feature map to obtain attention features including:

[0092] Max pooling is performed on the convolutional feature map to obtain the first pooled feature, and average pooling is performed on the convolutional feature map to obtain the second pooled feature.

[0093] The first pooling feature and the second pooling feature are subjected to shared fully connected processing to obtain the first fully connected feature;

[0094] The convolutional feature map is subjected to channel interaction processing to obtain interactive features;

[0095] The interaction features are processed using a fully connected method to obtain a second fully connected feature.

[0096] The first fully connected feature and the second fully connected feature are fused to obtain the fused feature;

[0097] Spatial attention processing is performed on the first fully connected feature, the second fully connected feature, and the fused feature to obtain the attention feature.

[0098] exist Figure 2 In the middle, when Figure 2 When the Feature map in the diagram represents the convolutional feature map, the feature output by the max pooling network is the first pooling feature, the feature output by the average pooling network is the second pooling feature, the feature output by the fully connected network 1 is the first fully connected feature, the feature output by the channel interaction network is the interaction feature, the feature output by the fully connected network 2 is the second fully connected feature, the feature output by the fusion network is the fusion feature, and the Refined feature output by the spatial attention network represents the attention feature.

[0099] Through the above implementation, average pooling and max pooling can be considered when compressing the spatial dimension. Therefore, while considering each pixel in the convolutional feature map, the pixel with the largest response during gradient backpropagation in the convolutional feature map can also be considered, thereby improving the representation effect of the first fully connected feature. By performing channel interaction processing on the convolutional feature map, the purpose of channel information interaction can be achieved, improving the representation ability of the interaction feature, thereby doubly improving the representation ability of the attention feature.

[0100] Specifically, the electronic device performs shared fully connected processing on the first pooling feature and the second pooling feature to obtain the first fully connected feature, which includes:

[0101] The first fully connected feature is obtained by averaging the first pooling feature and the second pooling feature after performing fully connected processing on each feature.

[0102] Specifically, the electronic device performs channel interaction processing on the convolutional feature map to obtain interactive features, including:

[0103] The convolutional feature map is grouped and convolved to obtain multiple initial features;

[0104] By concatenating the multiple initial features, a concatenated feature is obtained;

[0105] The splicing features are transposed to obtain transposed features;

[0106] Based on the feature dimension of any initial feature among the multiple initial features, multiple output features are sequentially extracted from the transposed features;

[0107] The interaction features are obtained by concatenating the multiple output features.

[0108] For example, the transpose feature is: If the feature dimension of any initial feature is 4, then after extraction, the multiple output features are obtained as: [1 7 8 2], [5 10 3 9], [11 4 6 12].

[0109] By performing grouped convolution processing on the convolutional feature maps, the generation efficiency of the interaction features can be improved while ensuring the accuracy of the interaction features.

[0110] Specifically, the electronic device performs spatial attention processing on the first fully connected feature, the second fully connected feature, and the fused feature to obtain the attention feature, which includes:

[0111] The input features are obtained by multiplying the first fully connected feature, the second fully connected feature, and the fused feature.

[0112] The input features are subjected to average pooling to obtain the third pooling feature, and the input features are subjected to max pooling to obtain the fourth pooling feature.

[0113] The third pooling feature and the fourth pooling feature are subjected to convolution and activation processing to obtain the attention feature.

[0114] By performing convolution and activation processing on the third and fourth pooling features, nonlinear factors can be introduced to improve the adaptability of the attention features.

[0115] In other embodiments, the generation method of the global features is similar to that of the attention features, and will not be described in detail here.

[0116] 104. Fuse multiple attention features corresponding to each vehicle damage feature map to obtain the local features of each vehicle damage feature map.

[0117] In at least one embodiment of the present invention, the local feature refers to the feature information corresponding to the defective area in the vehicle damage image.

[0118] In at least one embodiment of the present invention, the electronic device performs scale unification processing on the plurality of attention features, and further, the electronic device performs weighted and fused processing on the scale-unified features to obtain the local features.

[0119] By applying scale-unified processing to the multiple attention features, feature fusion can be facilitated, thereby improving the directness of generating the local features.

[0120] 105. Perform feature projection fusion processing on the local features and the global features to obtain the target features of each vehicle damage feature map.

[0121] In at least one embodiment of the present invention, the target feature refers to the final feature used for vehicle damage detection.

[0122] In at least one embodiment of the present invention, the electronic device performs feature projection fusion processing on the local features and the global features to obtain the target features of each vehicle damage feature map, including:

[0123] Calculate the product feature of the local feature and the global feature;

[0124] The ratio of the product feature to the magnitude of the local feature and the global feature is calculated to obtain the included angle feature;

[0125] Based on the dimension of the global feature, the local feature and the included angle feature are scaled to obtain a first transformed feature corresponding to the local feature and a second transformed feature corresponding to the included angle feature;

[0126] The target feature is obtained by fusing the global feature, the first transformation feature, and the second transformation feature.

[0127] The above implementation method can further improve the ability of the target features to represent defective areas in the vehicle damage image.

[0128] 106. Based on the pre-trained vehicle damage detection model, vehicle damage detection is performed on the target features to obtain the detection results of each vehicle damage feature map and the result confidence level corresponding to the detection results.

[0129] In at least one embodiment of the present invention, the vehicle damage detection model includes a feature pyramid network and a prediction network. The feature pyramid network is used to perform dimensionality reduction on the features. The prediction network is used to predict the damage result corresponding to the vehicle damage image.

[0130] The detection result refers to the result obtained after performing vehicle damage detection on each vehicle damage feature map. The detection result may include the damage type of the damaged accessory, for example, the detection result is: fender scratch. The result confidence level refers to the prediction probability of the vehicle damage detection network for the detection result.

[0131] like Figure 3 As shown, Figure 3 This is a schematic diagram of the network framework for generating detection results in the vehicle damage detection method of the present invention. Figure 3 In this process, each prediction network outputs the corresponding detection result and the result confidence level for each vehicle damage feature map.

[0132] In at least one embodiment of the present invention, the electronic device performs vehicle damage detection on the target features based on a pre-trained vehicle damage detection model, and obtains the detection result of each vehicle damage feature map and the result confidence level corresponding to the detection result, including:

[0133] The target features are upsampled based on the feature pyramid network to obtain upsampled features.

[0134] The upsampled features are detected based on the candidate box detection layer in the prediction network to obtain candidate box features;

[0135] Based on the mapping matrix in the prediction network, the candidate box features are activated and normalized to obtain a probability vector.

[0136] The category corresponding to the element in the probability vector whose value is greater than a preset probability threshold is determined as the detection result, and this element is determined as the confidence level of the result.

[0137] The mapping matrix and the preset probability threshold are hyperparameters in the vehicle damage detection model.

[0138] The feature pyramid network can further reduce the dimensionality of the upsampled features, thereby facilitating the detection of the upsampled features by the candidate box detection layer and improving the generation efficiency of the candidate box features. By filtering the detection results through the preset probability threshold, the filtering efficiency of the detection results can be further improved.

[0139] 107. Generate the vehicle damage result of the vehicle damage image based on multiple detection results and the confidence level of the results.

[0140] It should be emphasized that, to further ensure the privacy and security of the above vehicle damage results, the results can also be stored in a blockchain node.

[0141] In at least one embodiment of the present invention, the vehicle damage result refers to the detection result with the highest confidence level.

[0142] In at least one embodiment of the present invention, the electronic device generates the vehicle damage result of the vehicle damage image based on a plurality of detection results and the confidence level of the results, including:

[0143] If all the test results are the same, then the test result is determined as the vehicle damage result; or

[0144] If the multiple detection results are not all the same, the vehicle damage result is selected from the multiple detection results based on the confidence level of the result.

[0145] For example, the multiple detection results include: detection result A is fender scratch, detection result B is wheel arch scratch, and detection result C is fender scratch. The confidence level of the result corresponding to detection result A is 0.45, the confidence level of the result corresponding to detection result B is 0.51, and the confidence level of the result corresponding to detection result C is 0.50. Therefore, after calculation, the confidence level of the result "fender scratch" is 0.45 + 0.5 = 0.95, and the confidence level of the result "wheel arch scratch" is 0.51. Thus, the vehicle damage result is "fender scratch".

[0146] Through the above implementation method, when the multiple detection results are not all the same, the vehicle damage result can be accurately and quickly selected based on the result confidence level.

[0147] As can be seen from the above technical solutions, this application can obtain convolutional feature maps with different receptive fields by performing convolutional processing on the vehicle damage feature map with different porosity. Then, by performing convolutional attention processing and fusion processing on each convolutional feature map, the representation ability of the local features can be enhanced. Thus, the target features of the local features and the global features are fused as the final features for vehicle damage detection, which can effectively improve the feature extraction capability of the vehicle damage image, thereby improving the accuracy of the detection results and the generation of the result confidence. Furthermore, by combining the detection results and result confidence of the vehicle damage feature maps output by the multiple feature network layers, the vehicle damage results can be detected from different feature dimensions, thereby further improving the detection accuracy of the vehicle damage results.

[0148] like Figure 4The diagram shown is a functional block diagram of a preferred embodiment of the vehicle damage detection device of the present invention. The vehicle damage detection device 11 includes an extraction unit 110, a convolution unit 111, an attention unit 112, a fusion unit 113, a detection unit 114, and a generation unit 115. The module / unit referred to in this invention refers to a series of computer-readable instruction segments that can be acquired by the processor 13 and perform a fixed function, and are stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0149] Extraction unit 110 is used to extract the vehicle damage feature map of each feature network layer from the acquired vehicle damage image based on multiple feature network layers in the pre-trained feature extraction model.

[0150] Convolutional unit 111 is used to perform dilated convolution processing on each vehicle damage feature map to obtain multiple convolutional feature maps for each vehicle damage feature map;

[0151] Attention unit 112 is used to perform convolutional attention processing on each convolutional feature map to obtain attention features, and to perform convolutional attention processing on each vehicle damage feature map to obtain global features;

[0152] The fusion unit 113 is used to fuse multiple attention features corresponding to each vehicle damage feature map to obtain local features of each vehicle damage feature map;

[0153] The fusion unit 113 is also used to perform feature projection fusion processing on the local features and the global features to obtain the target features of each vehicle damage feature map;

[0154] The detection unit 114 is used to perform vehicle damage detection on the target features based on a pre-trained vehicle damage detection model, and obtain the detection result of each vehicle damage feature map and the result confidence level corresponding to the detection result.

[0155] The generation unit 115 is used to generate the vehicle damage result of the vehicle damage image based on multiple detection results and the confidence level of the results.

[0156] In at least one embodiment of the present invention, the attention unit 112 is further configured to perform max pooling on the convolutional feature map to obtain a first pooled feature, and perform average pooling on the convolutional feature map to obtain a second pooled feature.

[0157] The first pooling feature and the second pooling feature are subjected to shared fully connected processing to obtain the first fully connected feature;

[0158] The convolutional feature map is subjected to channel interaction processing to obtain interactive features;

[0159] The interaction features are processed using a fully connected method to obtain a second fully connected feature.

[0160] The first fully connected feature and the second fully connected feature are fused to obtain the fused feature;

[0161] Spatial attention processing is performed on the first fully connected feature, the second fully connected feature, and the fused feature to obtain the attention feature.

[0162] In at least one embodiment of the present invention, the attention unit 112 is further configured to perform grouped convolution processing on the convolutional feature map to obtain multiple initial features;

[0163] By concatenating the multiple initial features, a concatenated feature is obtained;

[0164] The splicing features are transposed to obtain transposed features;

[0165] Based on the feature dimension of any initial feature among the multiple initial features, multiple output features are sequentially extracted from the transposed features;

[0166] The interaction features are obtained by concatenating the multiple output features.

[0167] In at least one embodiment of the present invention, the attention unit 112 is further configured to calculate the product of the first fully connected feature, the second fully connected feature and the fused feature to obtain the input feature;

[0168] The input features are subjected to average pooling to obtain the third pooling feature, and the input features are subjected to max pooling to obtain the fourth pooling feature.

[0169] The third pooling feature and the fourth pooling feature are subjected to convolution and activation processing to obtain the attention feature.

[0170] In at least one embodiment of the present invention, the fusion unit 113 is further configured to calculate the product feature of the local feature and the global feature;

[0171] The ratio of the product feature to the magnitude of the local feature and the global feature is calculated to obtain the included angle feature;

[0172] Based on the dimension of the global feature, the local feature and the included angle feature are scaled to obtain a first transformed feature corresponding to the local feature and a second transformed feature corresponding to the included angle feature;

[0173] The target feature is obtained by fusing the global feature, the first transformation feature, and the second transformation feature.

[0174] In at least one embodiment of the present invention, the vehicle damage detection model includes a feature pyramid network and a prediction network, and the detection unit 114 is further configured to perform upsampling processing on the target features based on the feature pyramid network to obtain upsampled features;

[0175] The upsampled features are detected based on the candidate box detection layer in the prediction network to obtain candidate box features;

[0176] Based on the mapping matrix in the prediction network, the candidate box features are activated and normalized to obtain a probability vector.

[0177] The category corresponding to the element in the probability vector whose value is greater than a preset probability threshold is determined as the detection result, and this element is determined as the confidence level of the result.

[0178] In at least one embodiment of the present invention, the plurality of feature network layers include a first network layer, a second network layer and a third network layer, the first network layer, the second network layer and the third network layer are connected in sequence in the feature extraction model, the first network layer includes a convolutional layer of a plurality of branches, and the extraction unit 110 is further configured to perform convolution processing on the vehicle damage image based on the convolutional layer of each branch to obtain the convolutional features of each branch.

[0179] The multiple convolutional features are weighted and fused to obtain the vehicle damage feature map output by the first network layer;

[0180] The vehicle damage feature map output by the first network layer is convolved based on the second network layer to obtain the vehicle damage feature map output by the second network layer.

[0181] The vehicle damage feature map output by the second network layer is obtained by convolution processing based on the third network layer.

[0182] As can be seen from the above technical solutions, this application can obtain convolutional feature maps with different receptive fields by performing convolutional processing on the vehicle damage feature map with different porosity. Then, by performing convolutional attention processing and fusion processing on each convolutional feature map, the representation ability of the local features can be enhanced. Thus, the target features of the local features and the global features are fused as the final features for vehicle damage detection, which can effectively improve the feature extraction capability of the vehicle damage image, thereby improving the accuracy of the detection results and the generation of the result confidence. Furthermore, by combining the detection results and result confidence of the vehicle damage feature maps output by the multiple feature network layers, the vehicle damage results can be detected from different feature dimensions, thereby further improving the detection accuracy of the vehicle damage results.

[0183] like Figure 5 The diagram shown is a schematic diagram of the electronic device for implementing the vehicle damage detection method of the present invention.

[0184] In one embodiment of the present invention, the electronic device 1 includes, but is not limited to, a memory 12, a processor 13, and computer-readable instructions, such as a vehicle damage detection program, stored in the memory 12 and executable on the processor 13.

[0185] Those skilled in the art will understand that the schematic diagram is merely an example of electronic device 1 and does not constitute a limitation on electronic device 1. It may include more or fewer components than shown in the diagram, or combine certain components, or different components. For example, electronic device 1 may also include input / output devices, network access devices, buses, etc.

[0186] The processor 13 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 13 is the computing core and control center of the electronic device 1, connecting various parts of the electronic device 1 through various interfaces and lines, and executing the operating system of the electronic device 1, as well as various installed application programs and program code.

[0187] For example, the computer-readable instructions can be segmented into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present invention. The one or more modules / units can be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer-readable instructions in the electronic device 1. For example, the computer-readable instructions can be segmented into an extraction unit 110, a convolution unit 111, an attention unit 112, a fusion unit 113, a detection unit 114, and a generation unit 115.

[0188] The memory 12 can be used to store the computer-readable instructions and / or modules. The processor 13 implements various functions of the electronic device 1 by running or executing the computer-readable instructions and / or modules stored in the memory 12 and calling the data stored in the memory 12. The memory 12 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. The memory 12 may include non-volatile and volatile memory, such as: hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other storage devices.

[0189] The memory 12 can be the external memory and / or internal memory of the electronic device 1. Furthermore, the memory 12 can be a physical memory, such as a memory module, a TF card (Trans-flash Card), etc.

[0190] If the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by instructing related hardware through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when executed by a processor, the computer-readable instructions can implement the steps of the various method embodiments described above.

[0191] The computer-readable instructions include computer-readable instruction code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer-readable instruction code, recording medium, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), and random access memory (RAM).

[0192] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed vehicle damage detection, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0193] Combination Figure 1 The memory 12 in the electronic device 1 stores computer-readable instructions to implement a vehicle damage detection method, and the processor 13 can execute the computer-readable instructions to implement:

[0194] Based on multiple feature network layers in the pre-trained feature extraction model, vehicle damage feature maps of each feature network layer are extracted from the acquired vehicle damage images.

[0195] Each vehicle damage feature map is subjected to dilated convolution processing to obtain multiple convolutional feature maps for each vehicle damage feature map;

[0196] Convolutional attention processing is performed on each convolutional feature map to obtain attention features, and convolutional attention processing is performed on each vehicle damage feature map to obtain global features;

[0197] By fusing multiple attention features corresponding to each vehicle damage feature map, local features of each vehicle damage feature map are obtained;

[0198] The local features and the global features are subjected to feature projection fusion processing to obtain the target features of each vehicle damage feature map;

[0199] Based on the pre-trained vehicle damage detection model, vehicle damage is detected on the target features to obtain the detection result of each vehicle damage feature map and the result confidence level corresponding to the detection result;

[0200] The vehicle damage result is generated from the vehicle damage image based on multiple detection results and the confidence level of the results.

[0201] Specifically, the specific implementation method of the processor 13 for the above-mentioned computer-readable instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0202] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0203] The computer-readable storage medium stores computer-readable instructions, which, when executed by the processor 13, are used to perform the following steps:

[0204] Based on multiple feature network layers in the pre-trained feature extraction model, vehicle damage feature maps of each feature network layer are extracted from the acquired vehicle damage images.

[0205] Each vehicle damage feature map is subjected to dilated convolution processing to obtain multiple convolutional feature maps for each vehicle damage feature map;

[0206] Convolutional attention processing is performed on each convolutional feature map to obtain attention features, and convolutional attention processing is performed on each vehicle damage feature map to obtain global features;

[0207] By fusing multiple attention features corresponding to each vehicle damage feature map, local features of each vehicle damage feature map are obtained;

[0208] The local features and the global features are subjected to feature projection fusion processing to obtain the target features of each vehicle damage feature map;

[0209] Based on the pre-trained vehicle damage detection model, vehicle damage is detected on the target features to obtain the detection result of each vehicle damage feature map and the result confidence level corresponding to the detection result;

[0210] The vehicle damage result is generated from the vehicle damage image based on multiple detection results and the confidence level of the results.

[0211] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0212] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0213] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0214] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices described may also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0215] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for detecting vehicle damage, characterized in that, The vehicle damage detection method includes: Based on multiple feature network layers in the pre-trained feature extraction model, vehicle damage feature maps of each feature network layer are extracted from the acquired vehicle damage images. Each vehicle damage feature map is subjected to dilated convolution processing to obtain multiple convolutional feature maps for each vehicle damage feature map. The multiple convolutional feature maps for each vehicle damage feature map are obtained by performing convolution processing on each vehicle damage feature map using multiple convolutional layers with different dilation rates. Convolutional attention processing is performed on each convolutional feature map to obtain attention features, and convolutional attention processing is performed on each vehicle damage feature map to obtain global features; By fusing multiple attention features corresponding to each vehicle damage feature map, local features of each vehicle damage feature map are obtained; The local features and the global features are subjected to feature projection fusion processing to obtain the target features of each vehicle damage feature map; Based on the pre-trained vehicle damage detection model, vehicle damage is detected on the target features to obtain the detection result of each vehicle damage feature map and the result confidence level corresponding to the detection result; The vehicle damage result is generated from the vehicle damage image based on multiple detection results and the confidence level of the results. The vehicle damage result refers to the detection result with the highest confidence level.

2. The vehicle damage detection method as described in claim 1, characterized in that, The convolutional attention processing performed on each convolutional feature map yields attention features including: Max pooling is performed on the convolutional feature map to obtain the first pooled feature, and average pooling is performed on the convolutional feature map to obtain the second pooled feature. The first pooling feature and the second pooling feature are subjected to shared fully connected processing to obtain the first fully connected feature; The convolutional feature map is subjected to channel interaction processing to obtain interactive features; The interaction features are processed using a fully connected method to obtain a second fully connected feature. The first fully connected feature and the second fully connected feature are fused to obtain the fused feature; Spatial attention processing is performed on the first fully connected feature, the second fully connected feature, and the fused feature to obtain the attention feature.

3. The vehicle damage detection method as described in claim 2, characterized in that, The channel interaction processing of the convolutional feature map to obtain interactive features includes: The convolutional feature map is grouped and convolved to obtain multiple initial features; By concatenating the multiple initial features, a concatenated feature is obtained; The splicing features are transposed to obtain transposed features; Based on the feature dimension of any initial feature among the multiple initial features, multiple output features are sequentially extracted from the transposed features; The interaction features are obtained by concatenating the multiple output features.

4. The vehicle damage detection method as described in claim 2, characterized in that, The spatial attention processing performed on the first fully connected feature, the second fully connected feature, and the fused feature to obtain the attention feature includes: The input features are obtained by multiplying the first fully connected feature, the second fully connected feature, and the fused feature. The input features are subjected to average pooling to obtain the third pooling feature, and the input features are subjected to max pooling to obtain the fourth pooling feature. The third pooling feature and the fourth pooling feature are subjected to convolution and activation processing to obtain the attention feature.

5. The vehicle damage detection method as described in claim 1, characterized in that, The feature projection fusion process performed on the local features and the global features to obtain the target features of each vehicle damage feature map includes: Calculate the product feature of the local feature and the global feature; The ratio of the product feature to the magnitude of the local feature and the global feature is calculated to obtain the included angle feature; Based on the dimension of the global feature, the local feature and the included angle feature are scaled to obtain a first transformed feature corresponding to the local feature and a second transformed feature corresponding to the included angle feature; The target feature is obtained by fusing the global feature, the first transformation feature, and the second transformation feature.

6. The vehicle damage detection method as described in claim 1, characterized in that, The vehicle damage detection model includes a feature pyramid network and a prediction network. The pre-trained vehicle damage detection model performs vehicle damage detection on the target features, obtaining the detection result for each vehicle damage feature map and the corresponding confidence level of the detection result, including: The target features are upsampled based on the feature pyramid network to obtain upsampled features. The upsampled features are detected based on the candidate box detection layer in the prediction network to obtain candidate box features; Based on the mapping matrix in the prediction network, the candidate box features are activated and normalized to obtain a probability vector. The category corresponding to the element in the probability vector whose value is greater than a preset probability threshold is determined as the detection result, and this element is determined as the confidence level of the result.

7. The vehicle damage detection method as described in claim 1, characterized in that, The plurality of feature network layers include a first network layer, a second network layer, and a third network layer, which are sequentially connected in the feature extraction model. The first network layer includes a convolutional layer with multiple branches. Extracting the vehicle damage feature map of each feature network layer from the acquired vehicle damage image based on the plurality of feature network layers in the pre-trained feature extraction model includes: The vehicle damage image is convolved based on the convolutional layer of each branch to obtain the convolutional features of each branch. The multiple convolutional features are weighted and fused to obtain the vehicle damage feature map output by the first network layer; The vehicle damage feature map output by the first network layer is convolved based on the second network layer to obtain the vehicle damage feature map output by the second network layer. The vehicle damage feature map output by the second network layer is obtained by convolution processing based on the third network layer.

8. A vehicle damage detection device, characterized in that, The vehicle damage detection device includes: The extraction unit is used to extract the vehicle damage feature map of each feature network layer from the acquired vehicle damage image based on multiple feature network layers in the pre-trained feature extraction model. A convolutional unit is used to perform dilated convolution processing on each vehicle damage feature map to obtain multiple convolutional feature maps for each vehicle damage feature map. The multiple convolutional feature maps for each vehicle damage feature map are obtained by performing convolution processing on each vehicle damage feature map using multiple convolutional layers with different dilation rates. The attention unit is used to perform convolutional attention processing on each convolutional feature map to obtain attention features, and to perform convolutional attention processing on each vehicle damage feature map to obtain global features; The fusion unit is used to fuse multiple attention features corresponding to each vehicle damage feature map to obtain the local features of each vehicle damage feature map; The fusion unit is also used to perform feature projection fusion processing on the local features and the global features to obtain the target features of each vehicle damage feature map; The detection unit is used to perform vehicle damage detection on the target features based on a pre-trained vehicle damage detection model, and obtain the detection result of each vehicle damage feature map and the result confidence level corresponding to the detection result. The generation unit is used to generate a vehicle damage result of the vehicle damage image based on multiple detection results and the confidence level of the results, wherein the vehicle damage result refers to the detection result with the highest confidence level.

9. An electronic device, characterized in that, The electronic device includes: Memory, which stores computer-readable instructions; and The processor executes computer-readable instructions stored in the memory to implement the vehicle damage detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which are executed by a processor in an electronic device to implement the vehicle damage detection method as described in any one of claims 1 to 7.

Images