Vehicle damage detection method, device and equipment and storage medium

Abstract

The embodiment of the invention provides a vehicle damage detection method and device, equipment and a storage medium, and belongs to the technical field of financial science and technology and artificial intelligence. The method comprises the following steps: training a teacher network and a student network according to a marked vehicle damage image sample set to obtain a trained teacher network and a trained student network; based on active learning, a to-be-labeled vehicle damage image sample is screened out from the unlabeled vehicle damage image sample set through a trained teacher network for labeling, a new labeled vehicle damage image sample is obtained, and the new labeled vehicle damage image sample is added to a labeled vehicle damage image sample set to obtain a target labeled vehicle damage image sample set; performing incremental training on the trained teacher network and the trained student network according to the target labeled vehicle damage image sample set to obtain a trained damage detection model; and performing damage detection through the trained damage detection model. The embodiment of the invention can reduce the labor cost and improve the accuracy of damage detection.

Description

Technical Field

[0001] The present application relates to the fields of financial technology and artificial intelligence technology, and in particular to a vehicle damage detection method, device, equipment and storage medium. Background Art

[0002] In the vehicle damage assessment scenario in the field of financial technology, if the insured vehicle is involved in an accident such as a collision or scratch, the insurance agency's assessor needs to conduct an on-site inspection and take pictures of the damage images of the insured vehicle, and then perform damage detection on the damage images of the insured vehicle to obtain the damage category of the insured vehicle as the basis for the insurance agency's claims.

[0003] At present, target detection models such as Fast Regional Convolutional Neural Networks (Faster RCNN) and Single Shot MultiBox Detector (SSD) are usually used to achieve damage detection. However, the performance of the target detection model is positively correlated with the quality of the training data, which requires the training data to meet the conditions of large volume and carrying annotations. This requires a lot of manpower for manual annotation, which increases the labor cost. A small amount of training data with annotations will cause the target detection model to overfit, affecting the accuracy of damage detection.

[0004] Therefore, how to reduce the labor cost of manual labeling and improve the accuracy of damage detection has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The main purpose of the embodiments of the present application is to propose a vehicle damage detection method, device, equipment and storage medium, aiming to reduce the manpower cost of manual labeling and improve the accuracy of damage detection.

[0006] To achieve the above objectives, a first aspect of an embodiment of the present application provides a vehicle damage detection method, the method comprising:

[0007] Obtaining a sample set of labeled vehicle damage images and calling a preset damage detection model, wherein the preset damage detection model includes a teacher network and a student network;

[0008] According to the labeled vehicle damage image sample set, the teacher network and the student network are trained to obtain a trained teacher network and a trained student network;

[0009] Obtain a sample set of unlabeled vehicle damage images;

[0010] Based on active learning, the trained teacher network is used to select vehicle damage image samples to be labeled from the unlabeled vehicle damage image sample set;

[0011] Performing labeling processing on the to-be-labeled vehicle damage image samples to obtain new labeled vehicle damage image samples;

[0012] Adding the new labeled vehicle damage image sample to the labeled vehicle damage image sample set to obtain a target labeled vehicle damage image sample set;

[0013] According to the target labeled vehicle damage image sample set, incremental training is performed on the trained teacher network and the trained student network to obtain a trained damage detection model;

[0014] Acquire the target vehicle damage image;

[0015] The trained damage detection model is used to perform damage detection on the target vehicle damage image to obtain a target damage category of the target vehicle damage image.

[0016] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a vehicle damage detection device, the device comprising:

[0017] A labeled sample acquisition module is used to acquire a labeled vehicle damage image sample set and call a preset damage detection model, wherein the preset damage detection model includes a teacher network and a student network;

[0018] A model training module, used for training the teacher network and the student network according to the labeled vehicle damage image sample set to obtain a trained teacher network and a trained student network;

[0019] An unlabeled sample acquisition module is used to acquire an unlabeled vehicle damage image sample set;

[0020] A sample screening module, used for screening to-be-labeled vehicle damage image samples from the unlabeled vehicle damage image sample set through the trained teacher network based on active learning;

[0021] A sample labeling module is used to label the vehicle damage image sample to be labeled to obtain a new labeled vehicle damage image sample;

[0022] A sample adding module, used for adding the new labeled vehicle damage image sample to the labeled vehicle damage image sample set to obtain a target labeled vehicle damage image sample set;

[0023] A model incremental training module, which performs incremental training on the trained teacher network and the trained student network according to the target labeled vehicle damage image sample set to obtain a trained damage detection model;

[0024] An image acquisition module, used to acquire a target vehicle damage image;

[0025] The damage detection module is used to perform damage detection on the target vehicle damage image through the trained damage detection model to obtain a target damage category of the target vehicle damage image.

[0026] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes a computer device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the method described in the first aspect when executing the computer program.

[0027] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0028] The vehicle damage detection method, device, equipment and storage medium proposed in the present application use a labeled vehicle damage image sample set to train a teacher network and a student network of a preset damage detection model to obtain a trained teacher network and a trained student network, thereby based on active learning, through the trained teacher network, it is possible to screen out the most informative or valuable vehicle damage image samples to be labeled from the unlabeled vehicle damage image sample set, obtain new labeled vehicle damage image samples, add them to the labeled vehicle damage image sample set, and obtain the target labeled vehicle damage image sample set, which can significantly reduce the amount of labeling and reduce the time and effort of manual labeling. The labor cost is reduced in order to make full use of the original labeled vehicle damage image samples and the new labeled vehicle damage image samples in the target labeled vehicle damage image sample set, and perform incremental training on the trained teacher network and the trained student network to ensure that while learning the knowledge of the new labeled vehicle damage image samples, the knowledge of the original labeled vehicle damage image samples is retained, thereby ensuring the adaptability and stability of the trained damage detection model, greatly improving the performance of the trained damage detection model, so as to realize damage detection of the target vehicle damage image through the trained damage detection model with higher performance, greatly improving the accuracy of damage detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a flow chart of a vehicle damage detection method provided in an embodiment of the present application;

[0030] Figure 2 yes Figure 1 Flow chart of step S102 in FIG.

[0031] Figure 3 yes Figure 1 Flow chart of step S104 in FIG.

[0032] Figure 4 yes Figure 3 Flowchart of step S1042 in FIG.

[0033] Figure 5 is a schematic block diagram of a vehicle damage detection device provided in an embodiment of the present application;

[0034] Figure 6 It is a schematic block diagram of the structure of the computer device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0036] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0038] First, some nouns involved in this application are analyzed:

[0039] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0040] YOLOv7 network: It is an efficient and advanced deep network model in the field of object detection and is the latest version of the YOLO (You Only Look Once) series. YOLOv7 inherits the advantages of the YOLO series and is optimized to have better real-time detection performance and higher accuracy.

[0041] Active Learning: It is a machine learning method that can significantly reduce the number of required labeled samples by intelligently selecting the most informative or valuable samples to label, while improving the training efficiency and performance of the machine learning model.

[0042] Mean Average Precision (mAP): is a commonly used evaluation metric in object detection tasks, used to measure the detection performance of the object detection model on all categories. mAP is the average of the mean precision (AP) of all categories. AP measures the detection accuracy and recall of the model on a certain category. A high mAP value indicates that the model has strong accuracy and recall in detecting objects.

[0043] Entropy: It is a measure of the uncertainty or confusion of information, which comes from information theory. For a given prediction result, entropy can be used to quantify the certainty of the prediction. The higher the entropy, the more uncertain the prediction result is and the more confusing the information is; the lower the entropy, the more certain the prediction result is and the clearer the information is.

[0044] Based on this, the embodiments of the present application provide a vehicle damage detection method, device, equipment and storage medium, aiming to reduce the manpower cost of manual labeling and improve the accuracy of damage detection.

[0045] The vehicle damage detection method, device, equipment and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the vehicle damage detection method in the embodiments of the present application is described.

[0046] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0047] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0048] The vehicle damage detection method provided in the embodiment of the present application relates to the fields of financial technology and artificial intelligence technology. The vehicle damage detection method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the vehicle damage detection method, etc., but is not limited to the above forms.

[0049] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0050] Figure 1 is an optional flow chart of the vehicle damage detection method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S109.

[0051] Step S101, obtaining a sample set of labeled vehicle damage images, and calling a preset damage detection model, wherein the preset damage detection model includes a teacher network and a student network.

[0052] Taking the application in the vehicle damage assessment scenario as an example, the vehicle damage detection method provided in the embodiment of the present application is described in detail.

[0053] It can be understood that the labeled vehicle damage image sample set includes multiple labeled vehicle damage image samples.

[0054] The labeled vehicle damage image samples refer to vehicle damage image samples with manually labeled actual damage category labels. For example, the names of the damage category labels include scratches, scrapes, dents, wrinkles, dead folds, tears, missing, etc.

[0055] The damage category label can be annotated by annotating a damage rectangle. The damage rectangle refers to the minimum circumscribed rectangle of the outline of the damage area contained in the vehicle damage image sample.

[0056] The volume of the labeled vehicle damage image sample set is limited. For example, the labeled vehicle damage image sample set may include several hundred (eg, 300 to 500) labeled vehicle damage image samples.

[0057] The preset damage detection model includes a teacher network and a student network. Exemplarily, the teacher network can be a complete YOLOv7 network, and the student network can be a smaller or simplified lightweight YOLOv7 network.

[0058] Step S102: training the teacher network and the student network according to the labeled vehicle damage image sample set to obtain a trained teacher network and a trained student network.

[0059] To ensure that the training process is more efficient and stable, the labeled vehicle damage image sample set can be divided into several batches, each batch containing a certain number of labeled vehicle damage image samples, such as 32 or 64 labeled vehicle damage image samples as a batch. A training iteration (epoch) can include multiple batches, such as an epoch includes 5 batches.

[0060] In each epoch, each batch of labeled vehicle damage image samples is input into the teacher network and the student network for forward prediction. The parameters of the teacher network and the student network are updated through the training optimizer, and the learning rate is adaptively adjusted to ensure the stability of the training until the end of an epoch. Among them, the Stochastic Gradient Descent (SGD) algorithm can be used as a training optimizer to increase the training speed.

[0061] In each epoch, the student network learns the knowledge of the teacher network based on knowledge distillation, making the student network's prediction effect on damage categories closer to that of the teacher network.

[0062] After every several epochs, the teacher network and the student network are evaluated using a preset test set. For example, a variable best_mAP can be set, and after every 10 epochs, the current mAP is calculated using a preset test set, and the current mAP value is compared with best_mAP. If the current mAP is higher than best_mAP, best_mAP is updated to the current mAP, and the current parameters of the teacher network and the student network are saved as best_model_params. In other words, best_model_params refers to the parameters with the highest mAP on the test set.

[0063] Thus, the parameters of the teacher network and the student network that perform better on the test set can be obtained, thereby obtaining the trained teacher network and the trained student network. In this way, the overfitting of the teacher network and the student network during the training process can be avoided, and the generalization ability of the trained teacher network and the trained student network can be improved. The trained teacher network and the trained student network constitute a trained damage detection model.

[0064] Step S103, obtaining a sample set of unlabeled vehicle damage images.

[0065] It is understandable that the unlabeled vehicle damage image sample set includes a plurality of unlabeled vehicle damage image samples, and the unlabeled vehicle damage image samples do not carry the actual damage category labels manually labeled. There is no limit on the volume of the unlabeled vehicle damage image sample set.

[0066] Step S104 , based on active learning, the trained teacher network is used to filter out vehicle damage image samples to be labeled from the unlabeled vehicle damage image sample set.

[0067] Afterwards, based on active learning, the unlabeled vehicle damage image sample set is subjected to damage prediction through the trained teacher network, and the predicted damage rectangle of each unlabeled vehicle damage image sample in the unlabeled vehicle damage image sample set is obtained. Thus, based on the information entropy of the predicted damage rectangle, the vehicle damage image samples to be labeled are screened out from the unlabeled vehicle damage image sample set.

[0068] It is worth noting that based on information entropy, the most informative or valuable vehicle damage image samples to be labeled can be screened out from the unlabeled vehicle damage image sample set.

[0069] Step S105 , labeling the vehicle damage image samples to be labeled to obtain new labeled vehicle damage image samples.

[0070] The vehicle damage image samples to be labeled and the predicted damage rectangular frames of the vehicle damage image samples to be labeled can be sent to the annotator, so that the annotator can manually label the vehicle damage image samples to be labeled according to the predicted damage rectangular frames of the vehicle damage image samples to be labeled to obtain new labeled vehicle damage image samples.

[0071] In this way, only the most informative or valuable unlabeled vehicle damage image samples in the unlabeled vehicle damage image sample set need to be manually labeled, which can significantly reduce the amount of labeling and thus reduce the manpower cost of manual labeling.

[0072] Step S106: adding the new labeled vehicle damage image sample to the labeled vehicle damage image sample set to obtain a target labeled vehicle damage image sample set.

[0073] The new labeled vehicle damage image samples are added to the labeled vehicle damage image sample set to obtain the target labeled vehicle damage image sample set.

[0074] Therefore, based on active learning, it is possible to select the most informative or valuable vehicle damage image samples to be labeled from the unlabeled vehicle damage image sample set, and then add them to the labeled vehicle damage image sample set to obtain the target labeled vehicle damage image sample set. While significantly reducing the amount of labeling, the volume of the labeled vehicle damage image sample set is increased, so as to more effectively perform incremental training on the trained teacher network and the trained student network.

[0075] Step S107, performing incremental training on the trained teacher network and the trained student network according to the target labeled vehicle damage image sample set to obtain a trained damage detection model.

[0076] Based on the target labeled vehicle damage image sample set, the trained teacher network and the trained student network are incrementally trained to obtain a trained teacher network and a trained student network. The trained teacher network and the trained student network constitute a trained damage detection model.

[0077] In this way, the original labeled vehicle damage image samples and the new labeled vehicle damage image samples can be fully utilized, and the risk of "catastrophic forgetting" (i.e. forgetting the knowledge of old labeled vehicle damage image samples when training new labeled vehicle damage image samples) can be avoided, thereby ensuring the adaptability and stability of the trained damage detection model and greatly improving the performance of the trained damage detection model.

[0078] Step S108, obtaining a target vehicle damage image.

[0079] Acquire the target vehicle damage image that needs to be detected for damage.

[0080] Step S109: Perform damage detection on the target vehicle damage image using the trained damage detection model to obtain a target damage category of the target vehicle damage image.

[0081] The target vehicle damage image is input into the trained damage detection model, and the damage is predicted by the trained teacher network to obtain the target soft label of the target vehicle damage image output; the target vehicle damage image and the target soft label are input into the trained student network for damage prediction, and the target damage category output by the trained student network is obtained. In this way, the target soft label output by the trained teacher network can help the trained student network to perform more detailed and comprehensive damage detection on the target vehicle damage image, thereby improving the detection accuracy of damage detection.

[0082] Therefore, damage detection of the target vehicle damage image is achieved through a higher performance damage detection model, which greatly improves the accuracy of damage detection, thereby providing a more accurate basis for vehicle damage assessment in vehicle damage assessment scenarios and helping to improve the efficiency of vehicle damage assessment.

[0083] Steps S101 to S109 shown in the embodiment of the present application train the teacher network and the student network of the preset damage detection model through the labeled vehicle damage image sample set to obtain the trained teacher network and the trained student network, thereby, based on active learning, the trained teacher network is used to screen out the most informative or valuable vehicle damage image samples to be labeled from the unlabeled vehicle damage image sample set, obtain new labeled vehicle damage image samples, add them to the labeled vehicle damage image sample set, and obtain the target labeled vehicle damage image sample set, which can significantly reduce the amount of labeling and reduce the labor of manual labeling. The labor cost is reduced in order to make full use of the original labeled vehicle damage image samples and the new labeled vehicle damage image samples in the target labeled vehicle damage image sample set, and the trained teacher network and the trained student network are incrementally trained to ensure that while learning the knowledge of the new labeled vehicle damage image samples, the knowledge of the original labeled vehicle damage image samples is retained, thereby ensuring the adaptability and stability of the trained damage detection model, greatly improving the performance of the trained damage detection model, so as to realize damage detection of the target vehicle damage image through the trained damage detection model with higher performance, greatly improving the accuracy of damage detection.

[0084] In step S101 of some embodiments, an original labeled vehicle damage image sample set may be obtained; the original labeled vehicle damage image sample set may be enhanced to obtain an enhanced labeled vehicle damage image sample set; and the enhanced labeled vehicle damage image sample set may be scaled to obtain a labeled vehicle damage image sample set.

[0085] Among them, historical vehicle damage images can be obtained from the vehicle damage database of the insurance agency as original vehicle damage image samples, and manually annotated in batches to obtain an original annotated vehicle damage image sample set.

[0086] Then, each original labeled vehicle damage image sample in the original labeled vehicle damage image sample set is subjected to enhancement processing such as random cropping, random rotation, random change of saturation, random change of hue and / or random change of contrast to obtain an enhanced labeled vehicle damage image sample set.

[0087] Then, each enhanced labeled vehicle damage image sample in the enhanced labeled vehicle damage image sample set is scaled, such as scaling the enhanced labeled vehicle damage image sample to 640*640, thereby obtaining a labeled vehicle damage image sample set.

[0088] In this way, the quality of the labeled vehicle damage image sample set can be improved, making the labeled vehicle damage image sample set more suitable for training a preset damage detection model.

[0089] See also Figure 2 In some embodiments, the preset damage detection model includes a teacher network and a student network, and step S102 may include but is not limited to steps S1021 to S1022.

[0090] Step S1021: Input the labeled vehicle damage image sample set into the teacher network and the student network for training respectively to obtain the damage classification loss value and the knowledge distillation loss value.

[0091] The labeled vehicle damage image sample set is divided into several batches, each batch contains a certain number of labeled vehicle damage image samples, such as 32 or 64 labeled vehicle damage image samples as a batch. A training iteration (epoch) can include multiple batches, such as an epoch includes 5 batches.

[0092] In each epoch, each batch of labeled vehicle damage image samples is input into the teacher network and the student network for forward prediction to obtain the damage classification loss value and the knowledge distillation loss value.

[0093] In step S1021 of some embodiments, the labeled vehicle damage image sample set can be input into the teacher network to obtain the soft label of each labeled vehicle damage image sample in the labeled vehicle damage image sample set; the labeled vehicle damage image sample set can be input into the student network to obtain the predicted damage category label of each labeled vehicle damage image sample; the damage classification loss value can be obtained based on the predicted damage category label and the actual damage category label of each labeled vehicle damage image sample; the knowledge distillation loss value can be obtained based on the predicted damage category label and the soft label.

[0094] Specifically, in each epoch, each batch of labeled vehicle damage image samples is input into the teacher network to obtain the soft label of each labeled vehicle damage image sample. Each batch of labeled vehicle damage image samples is input into the student network to obtain the predicted damage category label of each labeled vehicle damage image sample.

[0095] The loss between the predicted damage category label and the actual damage category label of each labeled vehicle damage image sample is calculated to obtain the damage classification loss value.

[0096] Calculate the loss between the predicted damage category label and the soft label to obtain the knowledge distillation loss value.

[0097] Step S1022, updating the parameters of the teacher network and the student network according to the damage classification loss value and the knowledge distillation loss value until the teacher network and the student network converge to obtain a trained damage detection model.

[0098] According to the damage classification loss value and the knowledge distillation loss value, the parameters of the teacher network and the student network are updated through the training optimizer until the end of an epoch.

[0099] After each epoch, the teacher network and the student network are evaluated for performance using the preset test set to obtain the parameters with the highest mAP for the teacher network and the student network on the test set.

[0100] When the parameters of the teacher network and the student network with the highest mAP on the test set are obtained, it is determined that the teacher network and the student network converge, and a trained teacher network and a trained student network are obtained.

[0101] Through the above method, the teacher network and the student network can be effectively trained simultaneously in each epoch. While improving the prediction effect of the teacher network on the damage category, the student network can learn the knowledge of the teacher network, ensuring that the prediction effect of the student network on the damage category is closer to that of the teacher network.

[0102] See also Figure 3In some embodiments, step S104 may include but is not limited to steps S1041 to S1042.

[0103] Step S1041, based on active learning, damage prediction is performed on the unlabeled vehicle damage image sample set through the trained teacher network, and a predicted damage rectangle of each unlabeled vehicle damage image sample in the unlabeled vehicle damage image sample set is obtained.

[0104] Specifically, each unlabeled vehicle damage image sample in the unlabeled vehicle damage image sample set is input into the trained teacher network for damage prediction to obtain a predicted damage rectangular box for each unlabeled vehicle damage image sample, wherein the number of predicted damage rectangular boxes is at least two.

[0105] Step S1042 , selecting vehicle damage image samples to be labeled from the unlabeled vehicle damage image sample set according to the information entropy of the predicted damage rectangular frame.

[0106] According to the information entropy of the predicted damage rectangle, the vehicle damage image samples to be labeled are screened out from the unlabeled vehicle damage image sample set.

[0107] In this way, based on active learning, through the trained teacher network, the predicted damage rectangle of the unlabeled vehicle damage image sample can be quickly obtained, and the accuracy of the predicted damage rectangle can be improved. It can provide an accurate basis for screening out the most informative or valuable vehicle damage image samples to be labeled, thereby improving the screening efficiency and accuracy.

[0108] See also Figure 4 In some embodiments, step S1042 may include but is not limited to steps S401 to S403.

[0109] Step S401, summing up the information entropy of the predicted damage rectangle to obtain the total information entropy of each unlabeled vehicle damage image sample.

[0110] Specifically, for each unlabeled vehicle damage image sample, the information entropy of all predicted damage rectangular boxes contained therein is summed up to obtain the sum of the information entropy of all predicted damage rectangular boxes as the total information entropy of each unlabeled vehicle damage image sample.

[0111] It should be noted that the total information entropy is used to characterize the uncertainty of the trained teacher network in predicting the overall damage classification of unlabeled vehicle damage image samples. Unlabeled vehicle damage image samples with larger total information entropy contain complex or difficult-to-classify damage, indicating that the trained teacher network has higher uncertainty in predicting its damage classification, and has greater information content or higher value.

[0112] Step S402 , averaging the information entropy of the predicted damage rectangle to obtain the average information entropy of each unlabeled vehicle damage image sample.

[0113] For each unlabeled vehicle damage image sample, the average information entropy of all predicted damage rectangles is calculated as the average information entropy of each unlabeled vehicle damage image sample.

[0114] Step S403 , filtering out to-be-labeled vehicle damage image samples from the unlabeled vehicle damage image sample set according to the total information entropy and the average information entropy.

[0115] Therefore, based on the total information entropy and the average information entropy, the vehicle damage image samples to be labeled can be quickly and accurately screened out from the unlabeled vehicle damage image sample set.

[0116] In step S403 of some embodiments, the screening number K of vehicle damage image samples to be labeled can be obtained according to the average information entropy; the total information entropy can be sorted in order from large to small to screen out the top K unlabeled vehicle damage image samples with the largest total information entropy from the unlabeled vehicle damage image sample set; and the screened unlabeled vehicle damage image samples can be used as the vehicle damage image samples to be labeled.

[0117] Specifically, the average information entropy of all unlabeled vehicle damage image samples is summed to obtain the sum of the average information entropy of all unlabeled vehicle damage image samples, and the sum of the average information entropy is rounded down to an integer to obtain the screening number K of vehicle damage image samples to be screened out. In this way, the screening number K is determined based on the average information entropy, which improves the rationality of the screening number K.

[0118] The total information entropy of each unlabeled vehicle damage image sample is sorted in order from large to small, and the top K unlabeled vehicle damage image samples with the largest total information entropy are selected.

[0119] The first K unlabeled vehicle damage image samples with the largest total information entropy represent the worst damage classification effect of the trained teacher network on these K unlabeled vehicle damage image samples. For the trained teacher network, these are the most informative or valuable vehicle damage image samples to be labeled. Labeling these K unlabeled vehicle damage image samples can maximize the effectiveness of the labeling, thereby more effectively helping to improve the performance of the trained vehicle damage detection model, and can also help optimize the labeling process to significantly reduce the amount of labeling.

[0120] See also Figure 5 , Figure 5It is a schematic block diagram of a vehicle damage detection device provided in an embodiment of the present application. The vehicle damage detection device can be configured in a computer device to execute the aforementioned vehicle damage detection method.

[0121] like Figure 5 As shown, the vehicle damage detection device 500 includes: a labeled sample acquisition module 501, a model training module 502, an unlabeled sample acquisition module 503, a sample screening module 504, a sample annotation module 505, a sample adding module 506, a model incremental training module 507, an image acquisition module 508 and a damage detection module 509.

[0122] The labeled sample acquisition module 501 is used to acquire a labeled vehicle damage image sample set and call a preset damage detection model, wherein the preset damage detection model includes a teacher network and a student network;

[0123] A model training module 502 is used to train the teacher network and the student network according to the labeled vehicle damage image sample set to obtain a trained teacher network and a trained student network;

[0124] An unlabeled sample acquisition module 503 is used to acquire an unlabeled vehicle damage image sample set;

[0125] A sample screening module 504 is used to screen out to-be-labeled vehicle damage image samples from the unlabeled vehicle damage image sample set through the trained teacher network based on active learning;

[0126] The sample labeling module 505 is used to label the vehicle damage image sample to be labeled to obtain a new labeled vehicle damage image sample;

[0127] A sample adding module 506 is used to add the new labeled vehicle damage image sample to the labeled vehicle damage image sample set to obtain a target labeled vehicle damage image sample set;

[0128] A model incremental training module 507 performs incremental training on the trained teacher network and the trained student network according to the target labeled vehicle damage image sample set to obtain a trained damage detection model;

[0129] An image acquisition module 508 is used to acquire a target vehicle damage image;

[0130] The damage detection module 509 is used to perform damage detection on the target vehicle damage image by using the trained damage detection model to obtain a target damage category of the target vehicle damage image.

[0131] In one embodiment, the sample screening module 504 is further configured to:

[0132] Based on active learning, damage prediction is performed on the unlabeled vehicle damage image sample set through the trained teacher network to obtain a predicted damage rectangle of each unlabeled vehicle damage image sample in the unlabeled vehicle damage image sample set;

[0133] The vehicle damage image samples to be labeled are screened out from the unlabeled vehicle damage image sample set according to the information entropy of the predicted damage rectangular frame.

[0134] In one embodiment, the sample screening module 504 is further configured to:

[0135] The information entropy of the predicted damage rectangular frame is summed to obtain the total information entropy of each unlabeled vehicle damage image sample;

[0136] The information entropy of the predicted damage rectangular frame is averaged to obtain the average information entropy of each unlabeled vehicle damage image sample;

[0137] The vehicle damage image samples to be labeled are screened out from the unlabeled vehicle damage image sample set according to the total information entropy and the average information entropy.

[0138] In one embodiment, the sample screening module 504 is further configured to:

[0139] According to the average information entropy, obtaining the screening number K of the vehicle damage image samples to be labeled;

[0140] Sorting the total information entropy in descending order to select the top K unlabeled vehicle damage image samples with the largest total information entropy from the unlabeled vehicle damage image sample set;

[0141] The screened unlabeled vehicle damage image samples are used as the vehicle damage image samples to be labeled.

[0142] In one embodiment, the model training module 502 is further used to:

[0143] Inputting the labeled vehicle damage image sample set into the teacher network and the student network for training respectively to obtain a damage classification loss value and a knowledge distillation loss value;

[0144] According to the damage classification loss value and the knowledge distillation loss value, parameters of the teacher network and the student network are updated until the teacher network and the student network converge to obtain the trained damage detection model.

[0145] In one embodiment, the model training module 502 is further used to:

[0146] Inputting the labeled vehicle damage image sample set into the teacher network to obtain a soft label for each labeled vehicle damage image sample in the labeled vehicle damage image sample set;

[0147] Inputting the labeled vehicle damage image sample set into the student network to obtain a predicted damage category label for each labeled vehicle damage image sample;

[0148] Obtaining the damage classification loss value according to the predicted damage category label and the actual damage category label of each of the labeled vehicle damage image samples;

[0149] The knowledge distillation loss value is obtained according to the predicted damage category label and the soft label.

[0150] In one embodiment, the labeled sample acquisition module 501 is further used for:

[0151] Obtaining a sample set of original annotated vehicle damage images;

[0152] Performing enhancement processing on the original labeled vehicle damage image sample set to obtain an enhanced labeled vehicle damage image sample set;

[0153] The enhanced labeled vehicle damage image sample set is scaled to obtain the labeled vehicle damage image sample set.

[0154] Among them, each module in the above-mentioned vehicle damage detection device 500 corresponds to each step in the above-mentioned vehicle damage detection method embodiment, and its functions and implementation processes will not be repeated here one by one.

[0155] The vehicle damage detection device 500 can execute the vehicle damage detection method provided in the embodiment of the present application, and therefore can achieve the beneficial effects that can be achieved by the vehicle damage detection method provided in the embodiment of the present application. Please refer to the previous embodiments for details and will not be repeated here.

[0156] The method and apparatus of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0157] Exemplarily, the above method and apparatus may be implemented in the form of a computer program. The computer program may be implemented in Figure 6 Runs on the computer device shown.

[0158] See also Figure 6 , Figure 6 It is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application.

[0159] See also Figure 6 The computer device includes a processor and a memory connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0160] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0161] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any vehicle damage detection method.

[0162] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0163] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0164] Obtaining a sample set of labeled vehicle damage images and calling a preset damage detection model, wherein the preset damage detection model includes a teacher network and a student network;

[0165] According to the labeled vehicle damage image sample set, the teacher network and the student network are trained to obtain a trained teacher network and a trained student network;

[0166] Obtain a sample set of unlabeled vehicle damage images;

[0167] Based on active learning, the trained teacher network is used to filter out vehicle damage image samples to be labeled from the unlabeled vehicle damage image sample set;

[0168] Performing labeling processing on the to-be-labeled vehicle damage image samples to obtain new labeled vehicle damage image samples;

[0169] Adding the new labeled vehicle damage image sample to the labeled vehicle damage image sample set to obtain a target labeled vehicle damage image sample set;

[0170] According to the target labeled vehicle damage image sample set, incremental training is performed on the trained teacher network and the trained student network to obtain a trained damage detection model;

[0171] Acquire the target vehicle damage image;

[0172] The trained damage detection model is used to perform damage detection on the target vehicle damage image to obtain a target damage category of the target vehicle damage image.

[0173] In one embodiment, when the processor implements the active learning-based method of filtering out the to-be-labeled vehicle damage image samples from the unlabeled vehicle damage image sample set through the trained teacher network, the processor is configured to implement:

[0174] Based on active learning, damage prediction is performed on the unlabeled vehicle damage image sample set through the trained teacher network to obtain a predicted damage rectangle of each unlabeled vehicle damage image sample in the unlabeled vehicle damage image sample set;

[0175] The vehicle damage image samples to be labeled are screened out from the unlabeled vehicle damage image sample set according to the information entropy of the predicted damage rectangular frame.

[0176] In one embodiment, when the processor implements the step of filtering out the to-be-labeled vehicle damage image samples from the unlabeled vehicle damage image sample set according to the information entropy of the predicted damage rectangular frame, it is configured to implement:

[0177] The information entropy of the predicted damage rectangular frame is summed to obtain the total information entropy of each unlabeled vehicle damage image sample;

[0178] The information entropy of the predicted damage rectangular frame is averaged to obtain the average information entropy of each unlabeled vehicle damage image sample;

[0179] The vehicle damage image samples to be labeled are screened out from the unlabeled vehicle damage image sample set according to the total information entropy and the average information entropy.

[0180] In one embodiment, when the processor implements the step of filtering out the to-be-labeled vehicle damage image samples from the unlabeled vehicle damage image sample set according to the total information entropy and the average information entropy, the processor is configured to implement:

[0181] According to the average information entropy, obtaining the screening number K of the vehicle damage image samples to be labeled;

[0182] Sorting the total information entropy in descending order to select the top K unlabeled vehicle damage image samples with the largest total information entropy from the unlabeled vehicle damage image sample set;

[0183] The screened unlabeled vehicle damage image samples are used as the vehicle damage image samples to be labeled.

[0184] In one embodiment, the preset damage detection model includes a teacher network and a student network. When the processor trains the teacher network and the student network according to the labeled vehicle damage image sample set to obtain the trained teacher network and the trained student network, the processor is used to implement:

[0185] Inputting the labeled vehicle damage image sample set into the teacher network and the student network for training respectively to obtain a damage classification loss value and a knowledge distillation loss value;

[0186] According to the damage classification loss value and the knowledge distillation loss value, parameters of the teacher network and the student network are updated until the teacher network and the student network converge to obtain the trained damage detection model.

[0187] In one embodiment, when the processor implements the step of inputting the labeled vehicle damage image sample set into the teacher network and the student network for training and obtaining the damage classification loss value and the knowledge distillation loss value, it is used to implement:

[0188] Inputting the labeled vehicle damage image sample set into the teacher network to obtain a soft label for each labeled vehicle damage image sample in the labeled vehicle damage image sample set;

[0189] Inputting the labeled vehicle damage image sample set into the student network to obtain a predicted damage category label for each labeled vehicle damage image sample;

[0190] Obtaining the damage classification loss value according to the predicted damage category label and the actual damage category label of each of the labeled vehicle damage image samples;

[0191] The knowledge distillation loss value is obtained according to the predicted damage category label and the soft label.

[0192] In one embodiment, when the processor implements the acquisition of the labeled vehicle damage image sample set, it is used to implement:

[0193] Obtaining a sample set of original annotated vehicle damage images;

[0194] Performing enhancement processing on the original labeled vehicle damage image sample set to obtain an enhanced labeled vehicle damage image sample set;

[0195] The enhanced labeled vehicle damage image sample set is scaled to obtain the labeled vehicle damage image sample set.

[0196] The computer device can execute the vehicle damage detection method provided in the embodiment of the present application, and therefore can achieve the beneficial effects that can be achieved by the vehicle damage detection method provided in the embodiment of the present application. Please refer to the previous embodiments for details and will not be repeated here.

[0197] An embodiment of the present application also provides a computer-readable storage medium.

[0198] The computer-readable storage medium of the present application stores a computer program, and when the computer program is executed by a processor, the steps of the vehicle damage detection method as described above are implemented.

[0199] The computer-readable storage medium may be an internal storage unit of the vehicle damage detection device or computer device described in the above embodiments, such as a hard disk or memory of the vehicle damage detection device or computer device. The computer-readable storage medium may also be an external storage device of the vehicle damage detection device or computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital card (Secure Digital Card, SD Card), a flash card (Flash Card), etc. equipped on the vehicle damage detection device or computer device.

[0200] Since the computer program stored in the computer-readable storage medium can execute any one of the vehicle damage detection methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the vehicle damage detection methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0201] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0202] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0203] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0204] The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and these modifications or substitutions should be included in the protection scope of the present application.

Claims

1. A vehicle damage detection method, characterized in that: The method comprises: Obtaining a sample set of labeled vehicle damage images and calling a preset damage detection model, wherein the preset damage detection model includes a teacher network and a student network; According to the labeled vehicle damage image sample set, the teacher network and the student network are trained to obtain a trained teacher network and a trained student network; Obtain a sample set of unlabeled vehicle damage images; Based on active learning, the trained teacher network is used to select vehicle damage image samples to be labeled from the unlabeled vehicle damage image sample set; Performing labeling processing on the to-be-labeled vehicle damage image samples to obtain new labeled vehicle damage image samples; Adding the new labeled vehicle damage image sample to the labeled vehicle damage image sample set to obtain a target labeled vehicle damage image sample set; According to the target labeled vehicle damage image sample set, incremental training is performed on the trained teacher network and the trained student network to obtain a trained damage detection model; Acquire the target vehicle damage image; The trained damage detection model is used to perform damage detection on the target vehicle damage image to obtain a target damage category of the target vehicle damage image.

2. The method according to claim 1, characterized in that: The active learning-based method, through the trained teacher network, filters out the to-be-labeled vehicle damage image samples from the unlabeled vehicle damage image sample set, including: Based on active learning, damage prediction is performed on the unlabeled vehicle damage image sample set through the trained teacher network to obtain a predicted damage rectangle of each unlabeled vehicle damage image sample in the unlabeled vehicle damage image sample set; The vehicle damage image samples to be labeled are screened out from the unlabeled vehicle damage image sample set according to the information entropy of the predicted damage rectangular frame.

3. The method according to claim 2, characterized in that The filtering out the to-be-labeled vehicle damage image samples from the unlabeled vehicle damage image sample set according to the information entropy of the predicted damage rectangular frame comprises: The information entropy of the predicted damage rectangular frame is summed to obtain the total information entropy of each unlabeled vehicle damage image sample; The information entropy of the predicted damage rectangular frame is averaged to obtain the average information entropy of each unlabeled vehicle damage image sample; The vehicle damage image samples to be labeled are screened out from the unlabeled vehicle damage image sample set according to the total information entropy and the average information entropy.

4. The method according to claim 3, characterized in that The step of selecting the to-be-labeled vehicle damage image samples from the unlabeled vehicle damage image sample set according to the total information entropy and the average information entropy includes: According to the average information entropy, obtaining the screening number K of the vehicle damage image samples to be labeled; Sorting the total information entropy in descending order to select the top K unlabeled vehicle damage image samples with the largest total information entropy from the unlabeled vehicle damage image sample set; The screened unlabeled vehicle damage image samples are used as the vehicle damage image samples to be labeled.

5. The method according to claim 1, characterized in that The training of the teacher network and the student network according to the labeled vehicle damage image sample set to obtain a trained teacher network and a trained student network includes: Inputting the labeled vehicle damage image sample set into the teacher network and the student network for training respectively to obtain a damage classification loss value and a knowledge distillation loss value; According to the damage classification loss value and the knowledge distillation loss value, parameters of the teacher network and the student network are updated until the teacher network and the student network converge, thereby obtaining a trained teacher network and a trained student network.

6. The method according to claim 5, characterized in that The labeled vehicle damage image sample set is input into the teacher network and the student network for training respectively to obtain a damage classification loss value and a knowledge distillation loss value, including: Inputting the labeled vehicle damage image sample set into the teacher network to obtain a soft label for each labeled vehicle damage image sample in the labeled vehicle damage image sample set; Inputting the labeled vehicle damage image sample set into the student network to obtain a predicted damage category label for each labeled vehicle damage image sample; Obtaining the damage classification loss value according to the predicted damage category label and the actual damage category label of each of the labeled vehicle damage image samples; The knowledge distillation loss value is obtained according to the predicted damage category label and the soft label.

7. The method according to any one of claims 1 to 6, characterized in that: The step of obtaining a sample set of labeled vehicle damage images includes: Obtaining a sample set of original annotated vehicle damage images; Performing enhancement processing on the original labeled vehicle damage image sample set to obtain an enhanced labeled vehicle damage image sample set; The enhanced labeled vehicle damage image sample set is scaled to obtain the labeled vehicle damage image sample set.

8. A vehicle damage detection device, characterized in that: The device comprises: A labeled sample acquisition module is used to acquire a labeled vehicle damage image sample set and call a preset damage detection model, wherein the preset damage detection model includes a teacher network and a student network; A model training module, used for training the teacher network and the student network according to the labeled vehicle damage image sample set to obtain a trained teacher network and a trained student network; An unlabeled sample acquisition module is used to acquire an unlabeled vehicle damage image sample set; A sample screening module, used for screening to-be-labeled vehicle damage image samples from the unlabeled vehicle damage image sample set through the trained teacher network based on active learning; A sample labeling module is used to label the vehicle damage image sample to be labeled to obtain a new labeled vehicle damage image sample; A sample adding module, used for adding the new labeled vehicle damage image sample to the labeled vehicle damage image sample set to obtain a target labeled vehicle damage image sample set; A model incremental training module, which performs incremental training on the trained teacher network and the trained student network according to the target labeled vehicle damage image sample set to obtain a trained damage detection model; An image acquisition module, used to acquire a target vehicle damage image; The damage detection module is used to perform damage detection on the target vehicle damage image through the trained damage detection model to obtain a target damage category of the target vehicle damage image.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the vehicle damage detection method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the vehicle damage detection method according to any one of claims 1 to 7 is implemented.

Images