Vehicle damage detection method, device, equipment and storage medium

Abstract

The present application relates to artificial intelligence, and provides a vehicle damage detection method, device, equipment and storage medium. The method performs amplification processing on the obtained sample image to obtain a training image, performs feature extraction on the training image based on a feature extraction layer to obtain image features, performs fusion processing on the image features based on a feature fusion layer to obtain fused features, inputs the fused features into a network output layer to obtain prediction information, generates a network loss value according to the prediction information and a label result of the training image, adjusts parameters of a preset vehicle damage detection network based on the network loss value to obtain a vehicle damage detection model, and detects a to-be-recognized image according to the vehicle damage detection model and a semantic segmentation model, thereby improving the generation accuracy and efficiency of the detection result. In addition, the present application also relates to blockchain technology, and the detection result can be stored in a blockchain.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a vehicle damage detection method, apparatus, equipment, and storage medium. Background Technology

[0002] In routine auto insurance claims processing, in addition to inspecting the main exterior components of the vehicle, it is also necessary to examine and assess the damage to accessories on the exterior components and internal parts of the car to achieve accurate claims settlement. Exterior accessories are numerous and can be broadly categorized into four main types: bumper accessories, fender accessories, door accessories, and hood and trunk lid accessories. Each main category can be further divided into several subcategories. For example, bumper accessories include emblems, bright bumper strips, bumper trim, parking sensors, and tow hook covers. Each subcategory may also contain location information, such as front bumper emblems, hood emblems, and rear bumper trim.

[0003] In the traditional claims process, surveyors need to conduct on-site inspections at locations such as the vehicle's repair shop. They then manually identify damaged parts of the vehicle accessories, combining their personal claims experience with the company's claims rules. This process heavily relies on subjective judgment, and due to factors such as inconsistent understanding of claims rules and insufficient observation experience among users, the manually identified results are prone to discrepancies with the actual damage. Furthermore, the manual identification process is labor-intensive, inefficient, and consumes significant human and time resources. Summary of the Invention

[0004] In view of the above, it is necessary to provide a vehicle damage detection method, apparatus, equipment, and storage medium that can solve the technical problem of how to improve the accuracy and efficiency of vehicle damage detection.

[0005] On one hand, the present invention proposes a vehicle damage detection method, the vehicle damage detection method comprising:

[0006] The acquired sample images are amplified to obtain training images;

[0007] A preset vehicle damage detection network is obtained, which includes a feature extraction layer, a feature fusion layer, and a network output layer.

[0008] Based on the feature extraction layer, feature extraction is performed on the training image to obtain image features;

[0009] The image features are fused based on the feature fusion layer to obtain fused features;

[0010] The fused features are input into the network output layer to obtain prediction information;

[0011] The network loss value of the preset vehicle damage detection network is generated based on the prediction information and the annotation results of the training images;

[0012] Based on the network loss value, the parameters of the preset vehicle damage detection network are adjusted to obtain the vehicle damage detection model;

[0013] The vehicle damage detection model and the pre-trained semantic segmentation model are used to detect the image to be identified, and the detection results are obtained.

[0014] According to a preferred embodiment of the present invention, the step of amplifying the acquired sample image to obtain the training image includes:

[0015] Obtain the image categories of multiple sample images;

[0016] Count the number of categories in each image category for the multiple sample images;

[0017] Based on the number of categories, identify the target category from multiple image categories;

[0018] Based on a preset strategy, the sample images corresponding to the target category are amplified to obtain the training images.

[0019] According to a preferred embodiment of the present invention, identifying the target category from the plurality of image categories based on the number of categories includes:

[0020] Calculate the standard deviation and mean of the number of items in each category;

[0021] A target value is generated based on the standard deviation of the quantity and the mean of the quantity.

[0022] The image category corresponding to the number of categories that is less than the target value is determined as the target category.

[0023] According to a preferred embodiment of the present invention, the feature extraction layer includes a convolutional layer with multiple branches, and the feature extraction of the training image based on the feature extraction layer to obtain image features includes:

[0024] The training image is convolved based on the convolutional layer of each branch to obtain the convolutional features of each branch.

[0025] The image features are obtained by performing a weighted sum operation on multiple convolutional features.

[0026] According to a preferred embodiment of the present invention, the step of fusing the image features based on the feature fusion layer to obtain fused features includes:

[0027] The image features are subjected to multi-layer convolution processing to obtain initial features;

[0028] Each initial feature is pooled to obtain pooled features;

[0029] Max pooling is performed on the high-level features in the pooling features to obtain the first feature;

[0030] The lower-level features in the pooling features are upsampled to obtain the second feature;

[0031] The first feature and the second feature are summed to obtain the fused feature.

[0032] According to a preferred embodiment of the present invention, the prediction information includes a prediction block diagram, the damage attachments corresponding to the prediction block diagram, and the predicted probabilities of the damage attachments; the annotation result includes an annotation block diagram and the annotation type corresponding to the annotation block diagram; and the formula for calculating the network loss value is:

[0033] y = a × FL(p) t )+b×R;

[0034] FL(p t )=-(1-p t ) γ log(p t );

[0035]

[0036] Where y represents the network loss value, FL(p) t ) represents the first loss value of the preset vehicle damage detection network, R represents the second loss value of the preset vehicle damage detection network, a > 0, b > 0, γ > 0, p t This represents the predicted probability corresponding to the loss attachment when the loss attachment and the annotation type are the same, k1 represents the center position of the prediction diagram, k2 represents the center position of the annotation diagram, and d represents the predicted probability. 2 (k1, k2) represents the squared distance between the center position of the predicted bounding box and the center position of the labeled bounding box, and L represents the diagonal length of the minimum outer rectangle of the predicted bounding box and the labeled bounding box.

[0037] According to a preferred embodiment of the present invention, the step of detecting the image to be recognized based on the vehicle damage detection model and the pre-trained semantic segmentation model to obtain the detection result includes:

[0038] The vehicle damage detection model is used to detect the image to be identified, resulting in a damage map, the detection category corresponding to the damage map, and the detection probability corresponding to the detection category.

[0039] Based on the detection probability, a target category is selected from the detection categories;

[0040] The damage diagram is input into the semantic segmentation model for semantic segmentation processing to obtain the semantic category;

[0041] The detection result is generated based on the target category and the semantic category.

[0042] On the other hand, the present invention also proposes a vehicle damage detection device, the vehicle damage detection device comprising:

[0043] The amplification unit is used to amplify the acquired sample images to obtain training images;

[0044] An acquisition unit is used to acquire a preset vehicle damage detection network, the preset vehicle damage detection network including a feature extraction layer, a feature fusion layer and a network output layer;

[0045] The extraction unit is used to extract features from the training image based on the feature extraction layer to obtain image features;

[0046] The fusion unit is used to fuse the image features based on the feature fusion layer to obtain fused features;

[0047] An input unit is used to input the fused features into the network output layer to obtain prediction information;

[0048] A generation unit is used to generate the network loss value of the preset vehicle damage detection network based on the prediction information and the annotation results of the training images;

[0049] An adjustment unit is used to adjust the parameters of the preset vehicle damage detection network based on the network loss value to obtain a vehicle damage detection model.

[0050] The detection unit is used to detect the image to be identified based on the vehicle damage detection model and the pre-trained semantic segmentation model, and obtain the detection result.

[0051] On the other hand, the present invention also proposes an electronic device, the electronic device comprising:

[0052] Memory, which stores computer-readable instructions; and

[0053] The processor executes computer-readable instructions stored in the memory to implement the vehicle damage detection method.

[0054] On the other hand, the present invention also proposes a computer-readable storage medium storing computer-readable instructions, which are executed by a processor in an electronic device to implement the vehicle damage detection method.

[0055] As can be seen from the above technical solutions, this application, by amplifying the sample images, can avoid the inability to reasonably adjust the parameters of the preset vehicle damage detection network due to severe imbalances between data categories. The feature extraction layer extracts features from the training images; since it employs a multi-branch grouped convolutional approach, the extraction accuracy of image features is improved. The feature fusion layer fuses the image features; by combining lateral connections and top-down feature fusion, it effectively utilizes the resolution of low-level features and the semantic information of high-level features, improving the representational ability of the fused features on the training images, thereby enhancing the detection capability of the vehicle damage detection model. This application, combining the vehicle damage detection model and the semantic segmentation model to detect the image to be identified, can accurately generate detection results including damaged attachments and their locations. Furthermore, since manual identification and analysis are unnecessary, the generation efficiency of the detection results is improved. Attached Figure Description

[0056] Figure 1 This is a flowchart of a preferred embodiment of the vehicle damage detection method of the present invention.

[0057] Figure 2 This is a schematic diagram of the feature extraction layer in the vehicle damage detection method of the present invention.

[0058] Figure 3 This is a functional block diagram of a preferred embodiment of the vehicle damage detection device of the present invention.

[0059] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the vehicle damage detection method of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0061] like Figure 1 The diagram shown is a flowchart of a preferred embodiment of the vehicle damage detection method of the present invention. The order of the steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements.

[0062] The vehicle damage detection method described above can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0063] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0064] The vehicle damage detection method is applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored computer-readable instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0065] The electronic device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0066] The electronic devices may include network devices and / or user devices. The network devices include, but are not limited to, single network electronic devices, groups of multiple network electronic devices, or cloud computing-based systems consisting of a large number of hosts or network electronic devices.

[0067] The network in which the electronic device is located includes, but is not limited to: the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0068] 101. The acquired sample images are amplified to obtain training images.

[0069] In at least one embodiment of the present invention, the sample image refers to an image used to train the vehicle damage detection model. The sample image can be obtained from a configured vehicle maintenance database.

[0070] The training image refers to the image obtained by amplifying the sample images corresponding to image categories with fewer than a preset threshold number of categories.

[0071] In at least one embodiment of the present invention, the electronic device performs amplification processing on the acquired sample image to obtain a training image, including:

[0072] Obtain the image categories of multiple sample images;

[0073] Count the number of categories in each image category for the multiple sample images;

[0074] Based on the number of categories, identify the target category from multiple image categories;

[0075] Based on a preset strategy, the sample images corresponding to the target category are amplified to obtain the training images.

[0076] The target category refers to an image category whose number of categories is less than a preset threshold.

[0077] The preset strategies include Gaussian blur, linear contrast, Gaussian noise, brightness variation, radial transformation, random cropping, photometric geometric distortion, random flipping, and other strategies.

[0078] By using the number of categories, the target category can be reasonably identified, and then the sample images corresponding to the target category can be augmented. This avoids the problem of class imbalance in the sample images corresponding to the target category, which would prevent the model from being trained accurately, thereby effectively improving the robustness of the model.

[0079] Specifically, the electronic device identifies the target category from the multiple image categories based on the number of categories, including:

[0080] Calculate the standard deviation and mean of the number of items in each category;

[0081] A target value is generated based on the standard deviation of the quantity and the mean of the quantity.

[0082] The image category corresponding to the number of categories that is less than the target value is determined as the target category.

[0083] The target value can be the difference between the standard deviation of the quantity and the mean of the quantity, or it can be the difference between the standard deviation of the quantity and a preset multiple of the mean of the quantity. The preset multiple can be set according to actual needs. For example, the preset multiple can be 3.

[0084] By combining the standard deviation of the quantity and the mean of the quantity, the target value can be reasonably determined. Then, by comparing the target value with the number of categories, the target category can be accurately identified.

[0085] 102. Obtain a preset vehicle damage detection network, which includes a feature extraction layer, a feature fusion layer, and a network output layer.

[0086] In at least one embodiment of the present invention, the network parameters in the preset vehicle damage detection network are preset parameter values.

[0087] The feature extraction layer includes convolutional layers with multiple branches. For example... Figure 2 As shown, Figure 2 This is a schematic diagram of the feature extraction layer in the vehicle damage detection method of the present invention. Figure 2 The diagram shows a convolutional layer with 32 branches, each branch consisting of 3 convolutional layers.

[0088] The feature fusion layer includes multiple convolutional layers and pooling layers, etc.

[0089] The network output layer includes a candidate box detection layer and a category prediction layer.

[0090] 103. Based on the feature extraction layer, feature extraction is performed on the training image to obtain image features.

[0091] In at least one embodiment of the present invention, the image feature refers to the feature obtained by convolutional processing of the training image by the convolutional layers of multiple branches in the feature extraction layer.

[0092] In at least one embodiment of the present invention, the electronic device performs feature extraction on the training image based on the feature extraction layer to obtain image features including:

[0093] The training image is convolved based on the convolutional layer of each branch to obtain the convolutional features of each branch.

[0094] The image features are obtained by performing a weighted sum operation on multiple convolutional features.

[0095] For example, the convolutional features of branch A are: The convolutional features of branch B are The convolutional features of branch C are If the weight ratio of branch A, branch B, and branch C is 3:5:2, then a weighted sum operation is performed on the multiple convolutional features to obtain the image features as follows:

[0096] By simultaneously performing convolution processing on the training image using multiple branch convolutional layers, the generation efficiency of the convolutional features can be improved, thereby increasing the generation efficiency of the image features. Furthermore, by performing a weighted sum operation on multiple convolutional features, the generation accuracy of the image features can be improved.

[0097] 104. The image features are fused based on the feature fusion layer to obtain fused features.

[0098] In at least one embodiment of the present invention, the fusion feature refers to the feature obtained after fusing the image features.

[0099] In at least one embodiment of the present invention, the electronic device performs fusion processing on the image features based on the feature fusion layer to obtain fused features including:

[0100] The image features are subjected to multi-layer convolution processing to obtain initial features;

[0101] Each initial feature is pooled to obtain pooled features;

[0102] Max pooling is performed on the high-level features in the pooling features to obtain the first feature;

[0103] The lower-level features in the pooling features are upsampled to obtain the second feature;

[0104] The first feature and the second feature are summed to obtain the fused feature.

[0105] The high-level features refer to pooled features with fewer than a preset dimension threshold, while the low-level features refer to pooled features with more than or equal to the preset dimension threshold. The preset dimension threshold can be set according to actual needs.

[0106] By performing feature summation on the first feature obtained from the high-level features and the second feature obtained from the low-level features, the resolution of the low-level features and the semantic information of the high-level features can be effectively utilized, thereby improving the representational ability of the fused features on the training image.

[0107] 105. The fused features are input into the network output layer to obtain prediction information.

[0108] In at least one embodiment of the present invention, the prediction information includes a prediction block diagram, the damage attachments corresponding to the prediction block diagram, and the prediction probability of the damage attachments.

[0109] In at least one embodiment of the present invention, the electronic device inputs the fused features into the network output layer to obtain prediction information including:

[0110] Based on the candidate box detection layer in the network output layer, candidate boxes are detected on the fused features to obtain the predicted bounding map;

[0111] The fused features are activated and normalized based on the category prediction layer in the network output layer to obtain a probability vector.

[0112] The category corresponding to the element with the largest value in the probability vector is determined as the damage attachment, and this element is determined as the predicted probability corresponding to the damage attachment.

[0113] 106. Generate the network loss value of the preset vehicle damage detection network based on the prediction information and the annotation results of the training images.

[0114] In at least one embodiment of the present invention, the annotation result includes an annotation block diagram and the annotation type corresponding to the annotation block diagram.

[0115] In at least one embodiment of the present invention, the formula for calculating the network loss value is:

[0116] y = a × FL(p) t )+b×R;

[0117] FL(p t )=-(1-p t ) γ log(p t );

[0118]

[0119] Where y represents the network loss value, FL(p) t ) represents the first loss value of the preset vehicle damage detection network, R represents the second loss value of the preset vehicle damage detection network, a > 0, b > 0, γ > 0, p t This represents the predicted probability corresponding to the loss attachment when the loss attachment and the annotation type are the same, k1 represents the center position of the prediction diagram, k2 represents the center position of the annotation diagram, and d represents the predicted probability. 2 (k1, k2) represents the squared distance between the center position of the predicted bounding box and the center position of the labeled bounding box, and L represents the diagonal length of the minimum outer rectangle of the predicted bounding box and the labeled bounding box.

[0120] By calculating the first loss value and the second loss value in different ways, and then combining the first loss value and the second loss value to quantify the network loss value, the accuracy of model adjustment can be improved.

[0121] 107. Based on the network loss value, the parameters of the preset vehicle damage detection network are adjusted to obtain the vehicle damage detection model.

[0122] In at least one embodiment of the present invention, the vehicle damage detection model refers to the preset vehicle damage detection network whose parameters are adjusted until the network loss value no longer decreases.

[0123] In at least one embodiment of the present invention, the electronic device adjusts the parameters of the preset vehicle damage detection network including: the number of branches and the number of layers in each convolutional layer in the feature extraction layer, the number of convolutional layers, the size of the convolutional kernel, and the number of pooling layers in the feature fusion layer, etc.

[0124] 108. The image to be identified is detected according to the vehicle damage detection model and the pre-trained semantic segmentation model to obtain the detection result.

[0125] It should be emphasized that, to further ensure the privacy and security of the above test results, the test results can also be stored in a blockchain node.

[0126] In at least one embodiment of the present invention, the detection result includes the name of the damaged accessory and the location information of the accessory. For example, the detection result may be that the right rear wheel arch is damaged, that is, the name of the damaged accessory is wheel arch, and the location information of the accessory is the right rear wheel arch.

[0127] In at least one embodiment of the present invention, the electronic device detects the image to be recognized based on the vehicle damage detection model and the pre-trained semantic segmentation model, and obtains the detection result including:

[0128] The vehicle damage detection model is used to detect the image to be identified, resulting in a damage map, the detection category corresponding to the damage map, and the detection probability corresponding to the detection category.

[0129] Based on the detection probability, a target category is selected from the detection categories;

[0130] The damage diagram is input into the semantic segmentation model for semantic segmentation processing to obtain the semantic category;

[0131] The detection result is generated based on the target category and the semantic category.

[0132] The target category refers to the detection category with the highest detection probability, for example, the target category is a brow.

[0133] The semantic category refers to the orientation information identified by the semantic segmentation model from the damage diagram. For example, the semantic category is the right rear fender.

[0134] The vehicle damage detection model can accurately identify the damaged parts of the image to be identified, and the semantic segmentation model can accurately segment the location information of the damaged parts, thereby improving the comprehensiveness of the detection results.

[0135] Specifically, the way the vehicle damage detection model detects the image to be identified is similar to the way the preset vehicle damage detection network predicts the training image, and this application will not elaborate further on this.

[0136] Specifically, the electronic device inputs the damage diagram into the semantic segmentation model for semantic segmentation processing, and obtains semantic categories including:

[0137] The damage block diagram is vectorized based on the pixel information of the damage block diagram to obtain a block diagram matrix;

[0138] The block diagram matrix is ​​compared with the pre-stored attachment pixel values ​​in the semantic segmentation model to obtain multiple region locations corresponding to the attachment pixel values;

[0139] The semantic category is identified based on the configuration attachments corresponding to the multiple region locations and the attachment pixel values.

[0140] As can be seen from the above technical solutions, this application, by amplifying the sample images, can avoid the inability to reasonably adjust the parameters of the preset vehicle damage detection network due to severe imbalances between data categories. The feature extraction layer extracts features from the training images; since it employs a multi-branch grouped convolutional approach, the extraction accuracy of image features is improved. The feature fusion layer fuses the image features; by combining lateral connections and top-down feature fusion, it effectively utilizes the resolution of low-level features and the semantic information of high-level features, improving the representational ability of the fused features on the training images, thereby enhancing the detection capability of the vehicle damage detection model. This application, combining the vehicle damage detection model and the semantic segmentation model to detect the image to be identified, can accurately generate detection results including damaged attachments and their locations. Furthermore, since manual identification and analysis are unnecessary, the generation efficiency of the detection results is improved.

[0141] like Figure 3The diagram shown is a functional block diagram of a preferred embodiment of the vehicle damage detection device of the present invention. The vehicle damage detection device 11 includes an amplification unit 110, an acquisition unit 111, an extraction unit 112, a fusion unit 113, an input unit 114, a generation unit 115, an adjustment unit 116, and a detection unit 117. The module / unit referred to in this invention refers to a series of computer-readable instruction segments that can be acquired by the processor 13 and perform a fixed function, and are stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0142] The amplification unit 110 is used to amplify the acquired sample image to obtain a training image;

[0143] Acquisition unit 111 is used to acquire a preset vehicle damage detection network, the preset vehicle damage detection network including a feature extraction layer, a feature fusion layer and a network output layer;

[0144] Extraction unit 112 is used to extract features from the training image based on the feature extraction layer to obtain image features;

[0145] The fusion unit 113 is used to perform fusion processing on the image features based on the feature fusion layer to obtain fused features;

[0146] Input unit 114 is used to input the fused features into the network output layer to obtain prediction information;

[0147] The generation unit 115 is used to generate the network loss value of the preset vehicle damage detection network based on the prediction information and the annotation results of the training image;

[0148] Adjustment unit 116 is used to adjust the parameters of the preset vehicle damage detection network based on the network loss value to obtain a vehicle damage detection model;

[0149] The detection unit 117 is used to detect the image to be identified based on the vehicle damage detection model and the pre-trained semantic segmentation model, and obtain the detection result.

[0150] In at least one embodiment of the present invention, the amplification unit 110 is further configured to acquire image categories of a plurality of the sample images;

[0151] Count the number of categories in each image category for the multiple sample images;

[0152] Based on the number of categories, identify the target category from multiple image categories;

[0153] Based on a preset strategy, the sample images corresponding to the target category are amplified to obtain the training images.

[0154] In at least one embodiment of the present invention, the amplification unit 110 is further configured to calculate the standard deviation and mean of the number of categories;

[0155] A target value is generated based on the standard deviation of the quantity and the mean of the quantity.

[0156] The image category corresponding to the number of categories that is less than the target value is determined as the target category.

[0157] In at least one embodiment of the present invention, the extraction unit 112 is further configured to perform convolution processing on the training image based on the convolutional layer of each branch to obtain the convolutional features of each branch.

[0158] The image features are obtained by performing a weighted sum operation on multiple convolutional features.

[0159] In at least one embodiment of the present invention, the fusion unit 113 is further configured to perform multi-layer convolution processing on the image features to obtain initial features;

[0160] Each initial feature is pooled to obtain pooled features;

[0161] Max pooling is performed on the high-level features in the pooling features to obtain the first feature;

[0162] The lower-level features in the pooling features are upsampled to obtain the second feature;

[0163] The first feature and the second feature are summed to obtain the fused feature.

[0164] In at least one embodiment of the present invention, the prediction information includes a prediction block diagram, damage attachments corresponding to the prediction block diagram, and the predicted probabilities of the damage attachments; the annotation result includes an annotation block diagram and the annotation type corresponding to the annotation block diagram; and the formula for calculating the network loss value is:

[0165] y = a × FL(p) t )+b×R;

[0166] FL(p t )=-(1-p t ) γ log(p t );

[0167]

[0168] Where y represents the network loss value, FL(p) t) represents the first loss value of the preset vehicle damage detection network, R represents the second loss value of the preset vehicle damage detection network, a > 0, b > 0, γ > 0, p t This represents the predicted probability corresponding to the loss attachment when the loss attachment and the annotation type are the same, k1 represents the center position of the prediction diagram, k2 represents the center position of the annotation diagram, and d represents the predicted probability. 2 (k1, k2) represents the squared distance between the center position of the predicted bounding box and the center position of the labeled bounding box, and L represents the diagonal length of the minimum outer rectangle of the predicted bounding box and the labeled bounding box.

[0169] In at least one embodiment of the present invention, the detection unit 117 is further configured to detect the image to be identified based on the vehicle damage detection model to obtain a damage block diagram, a detection category corresponding to the damage block diagram, and a detection probability corresponding to the detection category.

[0170] Based on the detection probability, a target category is selected from the detection categories;

[0171] The damage diagram is input into the semantic segmentation model for semantic segmentation processing to obtain the semantic category;

[0172] The detection result is generated based on the target category and the semantic category.

[0173] As can be seen from the above technical solutions, this application, by amplifying the sample images, can avoid the inability to reasonably adjust the parameters of the preset vehicle damage detection network due to severe imbalances between data categories. The feature extraction layer extracts features from the training images; since it employs a multi-branch grouped convolutional approach, the extraction accuracy of image features is improved. The feature fusion layer fuses the image features; by combining lateral connections and top-down feature fusion, it effectively utilizes the resolution of low-level features and the semantic information of high-level features, improving the representational ability of the fused features on the training images, thereby enhancing the detection capability of the vehicle damage detection model. This application, combining the vehicle damage detection model and the semantic segmentation model to detect the image to be identified, can accurately generate detection results including damaged attachments and their locations. Furthermore, since manual identification and analysis are unnecessary, the generation efficiency of the detection results is improved.

[0174] like Figure 4 The diagram shown is a schematic diagram of the electronic device for implementing the vehicle damage detection method of the present invention.

[0175] In one embodiment of the present invention, the electronic device 1 includes, but is not limited to, a memory 12, a processor 13, and computer-readable instructions, such as a vehicle damage detection program, stored in the memory 12 and executable on the processor 13.

[0176] Those skilled in the art will understand that the schematic diagram is merely an example of electronic device 1 and does not constitute a limitation on electronic device 1. It may include more or fewer components than shown in the diagram, or combine certain components, or different components. For example, electronic device 1 may also include input / output devices, network access devices, buses, etc.

[0177] The processor 13 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 13 is the computing core and control center of the electronic device 1, connecting various parts of the electronic device 1 through various interfaces and lines, and executing the operating system of the electronic device 1, as well as various installed application programs and program code.

[0178] For example, the computer-readable instructions can be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present invention. The one or more modules / units can be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer-readable instructions in the electronic device 1. For example, the computer-readable instructions can be divided into an amplification unit 110, an acquisition unit 111, an extraction unit 112, a fusion unit 113, an input unit 114, a generation unit 115, an adjustment unit 116, and a detection unit 117.

[0179] The memory 12 can be used to store the computer-readable instructions and / or modules. The processor 13 implements various functions of the electronic device 1 by running or executing the computer-readable instructions and / or modules stored in the memory 12 and calling the data stored in the memory 12. The memory 12 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. The memory 12 may include non-volatile and volatile memory, such as: hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other storage devices.

[0180] The memory 12 can be the external memory and / or internal memory of the electronic device 1. Furthermore, the memory 12 can be a physical memory, such as a memory module, a TF card (Trans-flash Card), etc.

[0181] If the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by instructing related hardware through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when executed by a processor, the computer-readable instructions can implement the steps of the various method embodiments described above.

[0182] The computer-readable instructions include computer-readable instruction code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer-readable instruction code, recording medium, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), and random access memory (RAM).

[0183] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed vehicle damage detection, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0184] Combination Figure 1 The memory 12 in the electronic device 1 stores computer-readable instructions to implement a vehicle damage detection method, and the processor 13 can execute the computer-readable instructions to achieve the following:

[0185] The acquired sample images are amplified to obtain training images;

[0186] A preset vehicle damage detection network is obtained, which includes a feature extraction layer, a feature fusion layer, and a network output layer.

[0187] Based on the feature extraction layer, feature extraction is performed on the training image to obtain image features;

[0188] The image features are fused based on the feature fusion layer to obtain fused features;

[0189] The fused features are input into the network output layer to obtain prediction information;

[0190] The network loss value of the preset vehicle damage detection network is generated based on the prediction information and the annotation results of the training images;

[0191] Based on the network loss value, the parameters of the preset vehicle damage detection network are adjusted to obtain the vehicle damage detection model;

[0192] The vehicle damage detection model and the pre-trained semantic segmentation model are used to detect the image to be identified, and the detection results are obtained.

[0193] Specifically, the specific implementation method of the processor 13 of the above-mentioned computer-readable instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0194] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0195] The computer-readable storage medium stores computer-readable instructions, which, when executed by the processor 13, are used to perform the following steps:

[0196] The acquired sample images are amplified to obtain training images;

[0197] A preset vehicle damage detection network is obtained, which includes a feature extraction layer, a feature fusion layer, and a network output layer.

[0198] Based on the feature extraction layer, feature extraction is performed on the training image to obtain image features;

[0199] The image features are fused based on the feature fusion layer to obtain fused features;

[0200] The fused features are input into the network output layer to obtain prediction information;

[0201] The network loss value of the preset vehicle damage detection network is generated based on the prediction information and the annotation results of the training images;

[0202] Based on the network loss value, the parameters of the preset vehicle damage detection network are adjusted to obtain the vehicle damage detection model;

[0203] The vehicle damage detection model and the pre-trained semantic segmentation model are used to detect the image to be identified, and the detection results are obtained.

[0204] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0205] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0206] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0207] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices described may also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A method for detecting vehicle damage, characterized in that, The vehicle damage detection method includes: The acquired sample images are amplified to obtain training images; A preset vehicle damage detection network is obtained, which includes a feature extraction layer, a feature fusion layer, and a network output layer. Based on the feature extraction layer, feature extraction is performed on the training image to obtain image features; The image features are fused based on the feature fusion layer to obtain fused features, including: performing multi-layer convolution on the image features to obtain initial features; performing pooling on each initial feature to obtain pooled features; performing max pooling on the high-level features of the pooled features to obtain first features; performing upsampling on the low-level features of the pooled features to obtain second features; and performing feature summation on the first feature and the second feature to obtain the fused features. The fused features are input into the network output layer to obtain prediction information, which includes a prediction block diagram, the damage attachments corresponding to the prediction block diagram, and the prediction probability of the damage attachments. The network loss value of the preset vehicle damage detection network is generated based on the predicted information and the annotation results of the training images. The annotation results include an annotation box and the annotation type corresponding to the annotation box. The calculation formula for the network loss value is as follows: ； ； ； in, This represents the network loss value. This represents the first loss value of the preset vehicle damage detection network. This represents the second loss value of the preset vehicle damage detection network. , , , This indicates the predicted probability corresponding to the loss attachment when the loss attachment and the annotation type are the same. This indicates the center position of the prediction diagram. This indicates the center position of the labeled diagram. This represents the squared distance between the center position of the predicted bounding box and the center position of the labeled bounding box. This represents the diagonal length of the minimum outer rectangle of the predicted bounding box and the labeled bounding box; Based on the network loss value, the parameters of the preset vehicle damage detection network are adjusted to obtain the vehicle damage detection model; The vehicle damage detection model and the pre-trained semantic segmentation model are used to detect the image to be identified, and the detection results are obtained.

2. The vehicle damage detection method as described in claim 1, characterized in that, The step of amplifying the acquired sample images to obtain training images includes: Obtain the image categories of multiple sample images; Count the number of categories in each image category for the multiple sample images; Based on the number of categories, identify the target category from multiple image categories; Based on a preset strategy, the sample images corresponding to the target category are amplified to obtain the training images.

3. The vehicle damage detection method as described in claim 2, characterized in that, The step of identifying the target category from multiple image categories based on the number of categories includes: Calculate the standard deviation and mean of the number of items in each category; A target value is generated based on the standard deviation of the quantity and the mean of the quantity. The image category corresponding to the number of categories that is less than the target value is determined as the target category.

4. The vehicle damage detection method as described in claim 1, characterized in that, The feature extraction layer includes a convolutional layer with multiple branches. The feature extraction of the training image based on the feature extraction layer yields image features including: The training image is convolved based on the convolutional layer of each branch to obtain the convolutional features of each branch. The image features are obtained by performing a weighted sum operation on multiple convolutional features.

5. The vehicle damage detection method as described in claim 1, characterized in that, The detection of the image to be identified based on the vehicle damage detection model and the pre-trained semantic segmentation model yields the following detection results: The vehicle damage detection model is used to detect the image to be identified, resulting in a damage map, the detection category corresponding to the damage map, and the detection probability corresponding to the detection category. Based on the detection probability, a target category is selected from the detection categories; The damage diagram is input into the semantic segmentation model for semantic segmentation processing to obtain the semantic category; The detection result is generated based on the target category and the semantic category.

6. A vehicle damage detection device, characterized in that, The vehicle damage detection device includes: The amplification unit is used to amplify the acquired sample images to obtain training images; An acquisition unit is used to acquire a preset vehicle damage detection network, the preset vehicle damage detection network including a feature extraction layer, a feature fusion layer and a network output layer; The extraction unit is used to extract features from the training image based on the feature extraction layer to obtain image features; The fusion unit is used to perform fusion processing on the image features based on the feature fusion layer to obtain fused features, including: performing multi-layer convolution processing on the image features to obtain initial features; performing pooling processing on each initial feature to obtain pooled features; performing max pooling processing on the high-level features in the pooled features to obtain first features; performing upsampling processing on the low-level features in the pooled features to obtain second features; and performing feature summation operation on the first feature and the second feature to obtain the fused features. An input unit is used to input the fused features into the network output layer to obtain prediction information, the prediction information including a prediction block diagram, the damage attachments corresponding to the prediction block diagram, and the prediction probability of the damage attachments; The generation unit is configured to generate the network loss value of the preset vehicle damage detection network based on the prediction information and the annotation results of the training images. The annotation results include an annotation box and the annotation type corresponding to the annotation box. The calculation formula for the network loss value is: ； ； ； in, This represents the network loss value. This represents the first loss value of the preset vehicle damage detection network. This represents the second loss value of the preset vehicle damage detection network. , , , This indicates the predicted probability corresponding to the loss attachment when the loss attachment and the annotation type are the same. This indicates the center position of the prediction diagram. This indicates the center position of the labeled diagram. This represents the squared distance between the center position of the predicted bounding box and the center position of the labeled bounding box. This represents the diagonal length of the minimum outer rectangle of the predicted bounding box and the labeled bounding box; An adjustment unit is used to adjust the parameters of the preset vehicle damage detection network based on the network loss value to obtain a vehicle damage detection model. The detection unit is used to detect the image to be identified based on the vehicle damage detection model and the pre-trained semantic segmentation model, and obtain the detection result.

7. An electronic device, characterized in that, The electronic device includes: Memory, which stores computer-readable instructions; and The processor executes computer-readable instructions stored in the memory to implement the vehicle damage detection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which are executed by a processor in an electronic device to implement the vehicle damage detection method as described in any one of claims 1 to 5.

Images