Automobile part damage detection method, device and equipment and storage medium

Abstract

The invention discloses an automobile part damage detection method, device and equipment and a storage medium, and relates to the technical field of image processing and the financial science and technology field, and the method comprises the steps: carrying out the preliminary prediction of a to-be-detected vehicle damage image through a pre-trained vehicle damage part segmentation model and a pre-fine-adjusted damage segmentation large model; a first output result and a second output result are obtained respectively, auxiliary fusion of the generative adversarial network is carried out on outputs of the two models, so that a discriminator outputs a detection result of the vehicle damage image, and whether the vehicle damage image is damaged or not is judged according to the fact that the detection result is true or false. The advantages of different models can be combined through multi-model fusion, the vehicle part damage detection precision is improved, and the technical problems that in the prior art, precision is insufficient, when damage of different types and severity degrees is processed, sufficient flexibility and adaptability are lacked, and the user satisfaction degree is reduced due to low detection efficiency are solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology and financial technology, and in particular to a method, device, equipment and storage medium for detecting damage to automobile parts. Background Art

[0002] In the field of fintech, the accuracy of vehicle damage assessment is crucial to the automobile insurance and repair industries. With the development of technology, the demand for vehicle damage detection continues to grow, and the vehicle damage assessment business based on image inspection has become a key link. With the increase in the number of cars, the demand for vehicle damage detection is also growing. The vehicle damage assessment business based on image inspection, especially vehicle component damage detection, has become a key link. This process usually relies on image segmentation models to identify and assess the extent of vehicle damage. However, existing vehicle component segmentation models have shown some significant defects in practical applications.

[0003] First, existing vehicle component segmentation models are insufficient in terms of damage detection accuracy. When processing complex images, these models often make false detections and missed detections, resulting in some damage not being accurately identified, or incorrectly marking undamaged parts as damaged. This not only affects the accuracy of damage assessment, but can also lead to disputes and unnecessary damage in the subsequent claims process.

[0004] Secondly, existing technologies lack sufficient flexibility and adaptability when dealing with damage of different types and severity. Vehicle damage manifests itself in a variety of forms, and traditional segmentation models often have difficulty effectively coping with these changes, resulting in reduced reliability of detection results. In addition, existing models are highly dependent on labeled data during training and lack effective use of unlabeled data, further limiting the generalization ability of the model.

[0005] Finally, existing damage detection methods often fail to provide accurate references for subsequent damage assessment and claims. This not only affects the insurance company's claims efficiency, but also reduces customer satisfaction. Therefore, a new method is urgently needed to improve the accuracy and reliability of vehicle component damage detection to meet the industry's demand for efficient and accurate damage assessment. Summary of the invention

[0006] The present application provides a method, device, equipment and storage medium for detecting damage to automobile parts, which solves the technical problems existing in the prior art, such as insufficient accuracy, lack of sufficient flexibility and adaptability when dealing with damage of different types and severity, and low detection efficiency, which reduces user satisfaction.

[0007] In view of this, the first aspect of the present application provides a method for detecting damage to an automobile component, the method comprising:

[0008] Acquire a first vehicle damage image to be detected;

[0009] Inputting the first vehicle damage image into a pre-trained vehicle damaged parts segmentation model and a pre-fine-tuned damage segmentation large model respectively to obtain a first output result and a second output result;

[0010] Using the first output result and the second output result as inputs of a pre-trained generator network, and having the generator network perform feature fusion and feature extraction to obtain a first weight and a second weight of the first output result and the second output result;

[0011] The first output result and the second output result are weightedly fused based on the first weight and the second weight to obtain a detection result of the first vehicle damage image output by a pre-trained discriminator network.

[0012] A second aspect of the present application provides an automobile component damage detection device, the device comprising:

[0013] An acquisition unit, used for acquiring a first vehicle damage image to be detected;

[0014] A processing unit, configured to input the first vehicle damage image into a pre-trained vehicle damaged parts segmentation model and a pre-fine-tuned damage segmentation large model, respectively, to obtain a first output result and a second output result;

[0015] A generating unit, configured to use the first output result and the second output result as inputs of a pre-trained generator network, and the generator network performs feature fusion and feature extraction to obtain a first weight and a second weight of the first output result and the second output result;

[0016] A discriminant unit is used to perform weighted fusion on the first output result and the second output result based on the first weight and the second weight to obtain a detection result of the first vehicle damage image output by a pre-trained discriminator network.

[0017] A third aspect of the present application provides an automobile component damage detection device, the device comprising a processor and a memory:

[0018] The memory is used to store program code and transmit the program code to the processor;

[0019] The processor is used to execute the steps of the method for detecting damage to automobile parts as described in the first aspect according to the instructions in the program code.

[0020] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the method described in the first aspect above.

[0021] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0022] In the present application, a method, device, equipment and storage medium for detecting damage to automobile parts are provided. A pre-trained vehicle damage part segmentation model and a pre-fine-tuned damage segmentation large model are used to make a preliminary prediction of a vehicle damage image to be detected, and a first output result and a second output result are obtained respectively. The outputs of the two models are fused with the assistance of a generative adversarial network, so that a discriminator outputs a detection result of the vehicle damage image, and determines whether the vehicle damage image is damaged based on whether the detection result is true or false. Multi-model fusion can combine the advantages of different models to improve the accuracy of vehicle part damage detection, and solves the technical problems of insufficient accuracy in the prior art, lack of sufficient flexibility and adaptability when dealing with damage of different types and severity, and low detection efficiency, which reduces user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a schematic diagram of an application environment of an automobile component damage detection method in one embodiment of the present invention;

[0024] Figure 2 This is a method flow chart of the automobile component damage detection method in the embodiment of the present application;

[0025] Figure 3 This is a schematic diagram of the structure of the automobile component damage detection device in the embodiment of the present application;

[0026] Figure 4 It is a schematic diagram of the structure of the automobile component damage detection device in the embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] The present application designs a method, device, equipment and storage medium for detecting damage to automobile parts, which solves the technical problems existing in the prior art, such as insufficient accuracy, lack of sufficient flexibility and adaptability when dealing with damage of different types and severity, and low detection efficiency, which reduces user satisfaction.

[0029] The automobile component damage detection method provided by the embodiment of the present invention can be applied in the following aspects: Figure 1In an application environment, the client communicates with the server through a network. The server can obtain a first vehicle damage image to be detected through the client, input the first vehicle damage image into a pre-trained vehicle damage component segmentation model and a pre-fine-tuned damage segmentation model, respectively, obtain a first output result and a second output result, use the first output result and the second output result as inputs of a pre-trained generator network, perform feature fusion and feature extraction by the generator network, obtain a first weight and a second weight of the first output result and the second output result, perform weighted fusion on the first output result and the second output result based on the first weight and the second weight, obtain a detection result of the first vehicle damage image output by the pre-trained discriminator network, feed the detection result back to the client, and determine whether the vehicle damage image has damage according to whether the detection result is true or false. Multi-model fusion can combine the advantages of different models to improve the accuracy of vehicle component damage detection, and solve the technical problems of insufficient accuracy, lack of sufficient flexibility and adaptability when dealing with damage of different types and severity, and low detection efficiency that reduce user satisfaction. Among them, the client can be but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server side can be implemented by an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0030] For easier understanding, see Figure 2 , Figure 2 Flow chart of the method for detecting damage to automobile parts in the embodiment of the present application. Figure 2 As shown, specifically:

[0031] Step S101, obtaining a first vehicle damage image to be detected;

[0032] It should be noted that the vehicle images that need to be inspected for damage are collected. These images may come from vehicle insurance companies, repair shops or car owners, and usually need to ensure that the images are clear and the damaged parts are visible.

[0033] Step S102, inputting the first vehicle damage image into the pre-trained vehicle damaged parts segmentation model and the pre-fine-tuned damage segmentation large model respectively, to obtain a first output result and a second output result;

[0034] It should be noted that the image obtained in step S101 is input into two different models to obtain preliminary damage segmentation results. The first model is a pre-trained vehicle damaged parts segmentation model, and the second model is a pre-fine-tuned damage segmentation large model. These two models may have been trained or fine-tuned on different data sets to adapt to specific damage types and vehicle models.

[0035] Step S103: using the first output result and the second output result as inputs of a pre-trained generator network, and having the generator network perform feature fusion and feature extraction to obtain first weights and second weights of the first output result and the second output result;

[0036] It should be noted that the output result in step S102 is input into the generator network for feature fusion and weight calculation. The generator network concatenates the outputs of the two models in the channel dimension, fuses the features through techniques such as skip connections or attention mechanisms, and extracts the weights of the two output results.

[0037] Step S104: After weighted fusion of the first output result and the second output result based on the first weight and the second weight, a detection result of the first vehicle damage image output by the pre-trained discriminator network is obtained.

[0038] It should be noted that the output results of the two models are weighted and fused according to the weights calculated by the generator network, and then input into the discriminator network to obtain the final detection result. Weighted fusion is achieved by calculating the weighted average of the two output results. The discriminator network performs a binary classification judgment on the fusion result and outputs the authenticity or falsity of the detection result.

[0039] Furthermore, before acquiring the first vehicle damage image to be detected, the method further includes:

[0040] Acquire a second vehicle damage image including true segmentation annotations;

[0041] Initialize the weights of the object detection and instance segmentation models;

[0042] Based on the first preset damage function and the first preset optimizer, the target detection and instance segmentation model is iteratively trained through the second vehicle damage image to obtain a pre-trained vehicle damaged parts segmentation model.

[0043] It should be noted that before starting the detection, you need to prepare a pre-trained vehicle damage parts segmentation model. The vehicle damage parts segmentation model is specifically a target detection and instance segmentation model, such as: Cascade Mask R-CNN model, Faster R-CNN model, YOLO (You Only Look Once) model, SSD (Single Shot MultiBox Detector) model, RetinaNet model, CornerPoint model, Mask R-CNN model, TensorMask model, etc. Take the Cascade MaskR-CNN model as an example, specifically:

[0044] 1. Dataset preparation:

[0045] Collect a dataset containing vehicle damage images and their corresponding segmentation annotations.

[0046] The dataset should contain various types of vehicle damage, such as scratches, dents, and crushed parts.

[0047] The dataset is preprocessed, including image cropping, scaling, rotation, flipping, etc., to increase data diversity and improve the generalization ability of the model.

[0048] 2. Annotation format conversion:

[0049] Make sure the images are annotated in a format that is compatible with the Cascade Mask R-CNN model, which usually requires conversion to JSON or XML format.

[0050] According to the requirements of different models, the images are converted accordingly.

[0051] 3. Model training:

[0052] Load the Cascade Mask R-CNN model architecture.

[0053] Use the pre-trained weights of the model as initialization, if available.

[0054] Define the loss function, which usually includes classification loss, bounding box regression loss, and mask segmentation loss.

[0055] Configure optimizers such as SGD or Adam, as well as hyperparameters such as learning rate and weight decay.

[0056] 4. Model training process:

[0057] Split the dataset into training and validation sets.

[0058] Iterative training is performed on the training set, and each iteration includes forward propagation, loss calculation, backpropagation, and parameter update.

[0059] Evaluate model performance on the validation set, monitoring metrics such as loss and accuracy.

[0060] 5. Model fine-tuning:

[0061] Based on the performance on the validation set, fine-tune the model and adjust the network structure or hyperparameters.

[0062] 6. Model evaluation:

[0063] The final performance of the model is evaluated on an independent test set.

[0064] Use evaluation metrics such as precision, recall, mIOU (mean intersection over union), etc.

[0065] 7. Model optimization:

[0066] Based on the evaluation results, the model can be further optimized, which may include adjusting the network structure, adding data enhancement strategies, adjusting the loss function, etc.

[0067] Through this process, the Cascade Mask R-CNN model can be used to train a model that can accurately identify and segment damaged vehicle parts, providing accurate technical support for vehicle damage assessment.

[0068] Furthermore, before acquiring the first vehicle damage image to be detected, the method further includes:

[0069] Obtain a third vehicle damage image containing a true mask annotation;

[0070] Based on a grid-like dense sampling point method, a prompt point set of the third vehicle damage image is generated;

[0071] The cue point set is used as the input of the image segmentation model to obtain the predicted mask annotation output by the image segmentation model;

[0072] The difference between the real mask annotation and the predicted mask annotation is used as the second preset damage function. Combined with the preset parameter fine-tuning method and the second preset optimizer, the image segmentation model is iteratively trained to obtain a pre-fine-tuned damage segmentation model.

[0073] It should be noted that before starting the detection, a pre-fine-tuned damage segmentation model needs to be prepared, which involves using a third vehicle damage image containing real mask annotations, generating a set of prompt points through a grid-like dense sampling point method, and inputting them into the image segmentation model. Then, the difference between the predicted mask annotation and the real mask annotation is used as the loss function to fine-tune the model. Specifically:

[0074] 1. Dataset preparation:

[0075] Prepare a vehicle damage segmentation dataset with ground-truth mask annotations that define the precise locations of vehicle damage.

[0076] 2. Grid-like dense sampling point generation:

[0077] For each image in the dataset, a grid of densely sampled points is generated. These points will be used as input to the model to guide the model's predictions.

[0078] The sampling points may cover the entire image, but may be more densely concentrated in areas where damage is expected.

[0079] 3. Model selection:

[0080] Choose a pre-trained large model, such as SAM (SegmentAnything Model), which is a general model that can handle a variety of segmentation tasks.

[0081] In addition, you can also choose the Mask R-CNN model, YOLACT model, etc.

[0082] 4. Model input and output:

[0083] The generated grid-like dense sampling points are provided as input to the SAM model.

[0084] The SAM model receives input points and predicts a mask for each point, generating predicted mask annotations of vehicle damage.

[0085] 5. Loss function calculation:

[0086] Calculate the difference between the predicted mask annotation and the true mask annotation, and use Dice Loss as the loss function. Dice Loss is a loss function used to compare the similarity of two sample sets, especially suitable for segmentation tasks.

[0087] The Dice Loss calculation formula is: Among them, X is the predicted mask annotation and Y is the actual mask annotation.

[0088] 6. LoRA fine-tuning:

[0089] The model is fine-tuned using the LoRA (Layer-wise Optimized Representation Alignment) method. LoRA is a parameter-efficient fine-tuning method that is implemented by introducing additional trainable parameters in specific layers of the model.

[0090] LoRA fine-tunes the model by inserting small trainable matrices between existing layers of the network to adjust the feature representations.

[0091] 7. Training process:

[0092] Use an optimization algorithm such as Adam to update the model’s weights to minimize the loss function.

[0093] During training, you may want to use learning rate decay or other regularization techniques to prevent overfitting.

[0094] 8. Model evaluation:

[0095] Evaluate the performance of the fine-tuned model on the validation set, monitoring Dice Loss and other performance metrics.

[0096] Through this process, the large damage segmentation model can be effectively fine-tuned to make it better suited to specific vehicle damage segmentation tasks and improve the accuracy of damage detection.

[0097] Furthermore, before acquiring the first vehicle damage image to be detected, the method further includes:

[0098] Initialize the weights of the pre-trained generator network and discriminator network;

[0099] The collection of the second vehicle damage image and the third vehicle damage image is used as a training set, and the training set is respectively input into a pre-trained vehicle damage component segmentation model and a pre-fine-tuned damage segmentation large model to obtain a third output result and a fourth output result;

[0100] The third output result and the fourth output result are used as inputs of the generator network, combined with the true segmentation annotations of the second vehicle damage image and the third vehicle damage image and the true mask annotations, and according to the output of the discriminator network, the generator network and the discriminator network are alternately iteratively trained to obtain pre-trained generator network and discriminator network.

[0101] It should be noted that before starting the detection, the generator network and the discriminator network need to be trained. This involves using the collection of the second vehicle damage image and the third vehicle damage image as a training set, inputting them into the pre-trained vehicle damage component segmentation model and the pre-fine-tuned damage segmentation model, and obtaining the output results. Then, using these output results and true annotations, the generator network and the discriminator network are alternately iterated to train. Specifically:

[0102] 1. Initialize network parameters

[0103] Generator: Usually initialized with random weights, it is responsible for generating fake data that is as close to the real data distribution as possible.

[0104] Discriminator: Also initialized with random weights, it is responsible for distinguishing real data from fake data generated by the generator.

[0105] 2. Training the Discriminator

[0106] Real data: A batch of data is extracted from the real dataset, and the discriminator learns to label this data as "real".

[0107] Fake Data: At the same time, the generator produces a batch of fake data, and the discriminator learns to label this data as "fake".

[0108] Discriminator training: The discriminator is trained by minimizing the classification error between real and fake data.

[0109] 3. Training the Generator

[0110] Generate fake data: The generator produces a new batch of fake data.

[0111] Discriminator evaluation: Use the trained discriminator to evaluate these fake data.

[0112] Generator training: The goal of the generator is to trick the discriminator into classifying the fake data it generates as “real”. This is usually achieved by minimizing the log probability of the discriminator’s output.

[0113] 4. Alternate training

[0114] The GAN training process is carried out alternately, that is, the discriminator and the generator are trained alternately.

[0115] With each iteration, the generator learns how to generate data better, while the discriminator learns how to better distinguish between real and fake data.

[0116] 5. Optimize Algorithms

[0117] Gradient descent methods such as the Adam optimizer are often used to update the weights of the network.

[0118] 6. Loss Function

[0119] Discriminator loss: usually consists of two parts, one is the loss of classifying real samples as real, and the other is the loss of classifying fake samples as fake.

[0120] Generator loss: usually the logarithm of the probability that a fake sample generated by the generator is classified as real by the discriminator.

[0121] 7. Training Dynamics

[0122] During the training process, the generator and the discriminator continuously perform adversarial training, with the generator striving to generate more and more realistic data, while the discriminator strives to better distinguish between true and false data.

[0123] 8. Model evaluation and adjustment

[0124] During the training process, the performance of the GAN is regularly evaluated.

[0125] It may be necessary to adjust the learning rate, batch size, network architecture, or other hyperparameters to improve training stability and result quality.

[0126] In vehicle damage segmentation, the generator network can be used to fuse the segmentation results of different models, and the discriminator can be used to evaluate the quality of the fusion results and provide feedback to improve the fusion process.

[0127] Furthermore, the first output result and the second output result are used as inputs of the generator network, and the generator network performs feature fusion and feature extraction to obtain first weights and second weights of the first output result and the second output result, which specifically include:

[0128] Concatenate the first output result and the second output result in the channel dimension to obtain a concatenated dimension vector;

[0129] Inputting the first output result, the second output result, and the vector dimension into the generator network, fusing the first features of the first output result and the second output result by a skip connection or an attention mechanism of the generator network, and extracting the second feature of the dimension vector by a convolution layer and an activation function of the generator network;

[0130] According to the first feature and the second feature, a first weight and a second weight of the first output result and the second output result are obtained respectively.

[0131] It should be noted that in order to combine the output results of the two models into a single input so that the generator network can process them simultaneously, the first output result x1 and the second output result x2 are concatenated in the channel dimension. Assume that the dimensions of x1 and x2 are (b, c1, w, h) and (b, c2, w, h), respectively, where b is the batch size, c1 and c2 are the number of channels, and w and h are the width and height. The concatenated dimension vector is (b, c1 + c2, w, h).

[0132] The concatenated input is processed through the generator network to extract and fuse features from the two models, specifically:

[0133] Input: The concatenated dimension vector along with the original x1 and x2 are provided as input to the generator network.

[0134] Fusion: The generator network uses skip connections (such as in ResNet) or attention mechanisms to fuse features. These mechanisms can help the network learn how to combine the outputs of different models.

[0135] Extraction: The generator network contains convolutional layers and activation functions (such as ReLU) to extract features of the concatenated vector.

[0136] Assign a weight to each model’s output that reflects their relative importance in the final fusion result, including:

[0137] First feature: The generator network processes the fused features through a specific layer (which may be a convolutional layer or a fully connected layer) to generate the first weights.

[0138] Second features: Similarly, the generator network processes the extracted features to generate second weights.

[0139] Weight calculation: Weights can be calculated by different methods, for example, weights can be calculated dynamically by learning the importance of each feature, or weights can be assigned using a predefined strategy.

[0140] This process allows the generator network to learn how to best combine the outputs of the two models in order to produce more accurate damage detection results. In this way, the strengths of each model can be utilized and the limitations that may exist in a single model can be reduced.

[0141] Furthermore, after weighted fusion of the first output result and the second output result based on the first weight and the second weight, the detection result of the first vehicle damage image output by the pre-trained discriminator network specifically includes:

[0142] Obtaining a weighted average of the first output result and the second output result according to the first weight and the second weight;

[0143] The weighted average value is used as the input of the pre-trained discriminator network, and the discriminator network performs a binary classification judgment to obtain a detection result of the discriminator network on the first vehicle damage image, where the detection result includes true or false.

[0144] It should be noted that the output results of the two models are combined and a weight is assigned to each result to generate a comprehensive segmentation result. Specifically:

[0145] A weighted average of the first output result and the second output result is calculated using the first weight and the second weight generated by the generator network.

[0146] The discriminator network is used to judge the fused segmentation results to determine their authenticity.

[0147] The discriminator network usually contains several convolutional layers, fully connected layers and activation functions, and an output layer for binary classification.

[0148] The output layer may use a sigmoid activation function to map the output value to between 0 and 1, indicating the probability of real damage. According to the output probability of the discriminator network, a threshold (such as 0.5) is set, and the results with a probability higher than the threshold are judged as real damage, and the results with a probability lower than the threshold are judged as false damage.

[0149] This judgment result is output as the final detection result.

[0150] This process improves the accuracy and reliability of damage detection by combining the capabilities of the generator network, which is responsible for fusing the outputs of different models, and the discriminator network, which is responsible for verifying the authenticity of the fusion results.

[0151] Further, after obtaining the detection result of the first vehicle damage image output by the pre-trained discriminator network, the method further includes:

[0152] Evaluate the confidence of the detection results output by the discriminator network based on a preset confidence evaluation model;

[0153] If the confidence level is not lower than a preset threshold, the detection result is used as the final damage detection result of the first vehicle damage image;

[0154] If the confidence level is lower than a preset threshold, the first vehicle damage image, the first output result, the second output result and the detection result are taken as questionable data sets and stored in the target database.

[0155] It should be noted that the confidence of the detection results output by the pre-trained discriminator network is evaluated;

[0156] When the confidence exceeds the preset threshold, the detection result is taken as the final damage detection result;

[0157] When the confidence level does not exceed the preset threshold, the first vehicle damage image and the corresponding first output result and second output result are submitted to manual review to determine the final damage detection result;

[0158] The step of evaluating the confidence includes using a confidence evaluation model trained based on the output of the discriminator network and the content of the first vehicle damage image to predict the reliability of the damage detection result;

[0159] The confidence assessment model can be a logistic regression model, a support vector machine, a neural network, or other statistical learning model, which learns how to map the output of the discriminator to a confidence score based on historical data;

[0160] The training of the confidence assessment model includes using a set of damage detection results marked as true or false as training data and adjusting the model parameters by optimizing a preset confidence assessment loss function;

[0161] The confidence evaluation loss function can be a cross entropy loss, a mean square error loss, or other suitable loss functions;

[0162] Manual review is performed by professional damage assessors who analyze and judge the damage images and their output results based on their professional knowledge and experience;

[0163] The results of manual review are used to update and optimize the damaged parts segmentation model, the damage segmentation model, the generator network, and the discriminator network to improve the overall performance of the automotive parts damage detection method.

[0164] Based on the automobile component damage detection method provided in the embodiment of the present application, a specific application scenario example is provided here:

[0165] Insurance companies use a deep learning-based automotive component damage detection system that uses the multi-model fusion method and generator-discriminator network mentioned above to automate the damage assessment process.

[0166] Operation process:

[0167] Image acquisition:

[0168] When a vehicle is involved in an accident, the owner or claims adjuster uses a smartphone or professional equipment to take images of the damaged parts of the vehicle.

[0169] Image upload:

[0170] The captured images are uploaded to the insurance company's cloud service platform.

[0171] Image preprocessing:

[0172] The cloud platform pre-processes the uploaded images, including cropping, rotation correction, image enhancement, etc., to ensure that the image quality is suitable for damage detection.

[0173] Damage Detection:

[0174] The preprocessed images are input into the damage detection system, which uses the pre-trained damaged parts segmentation model and the pre-fine-tuned damage segmentation model for analysis.

[0175] Feature Fusion:

[0176] The generator network receives the output results of the two models, performs feature fusion and weight calculation, and generates a comprehensive segmentation result.

[0177] Authenticity judgment:

[0178] The discriminator network performs a binary classification on the output of the generator network to determine the authenticity of the damaged image.

[0179] Result output:

[0180] The system outputs damage detection results, including segmented images of the damaged area and judgment of the authenticity of the damage.

[0181] Claims decision:

[0182] The insurance company can quickly determine the amount of compensation based on the test results provided by the system and the vehicle's repair cost database.

[0183] Manual review:

[0184] If the confidence of the discriminator network falls below a preset threshold, or if the claims adjuster has doubts about the result, the case can be marked for manual review.

[0185] Through this deep learning-driven automated damage detection system, insurance companies are able to provide more efficient and accurate services while also improving customer trust and satisfaction.

[0186] In the field of automobile insurance claims, the accuracy of vehicle damage detection is crucial to compensation decisions. This method combines a pre-trained vehicle damage parts segmentation model with a pre-fine-tuned damage segmentation model, uses a generator network for feature fusion, and a discriminator network for authenticity judgment to achieve automatic detection of vehicle damage. The system first obtains the vehicle damage image, and then obtains preliminary segmentation results through two models. The generator network performs weighted fusion on these results, and the discriminator network evaluates the confidence of the fusion results. If the confidence exceeds the threshold, the result is accepted; otherwise, the questionable data is stored for manual review. This process improves the efficiency and accuracy of damage assessment while ensuring the reliability of claims decisions.

[0187] See also Figure 3 , Figure 3 FIG. 1 is a schematic diagram of the structure of the automobile component damage detection device in the embodiment of the present application. Figure 3 As shown, specifically:

[0188] An acquisition unit 201 is used to acquire a first vehicle damage image to be detected;

[0189] A processing unit 202 is used to input the first vehicle damage image into a pre-trained vehicle damaged parts segmentation model and a pre-fine-tuned damage segmentation model, respectively, to obtain a first output result and a second output result;

[0190] A generating unit 203 is used to use the first output result and the second output result as inputs of a pre-trained generator network, and the generator network performs feature fusion and feature extraction to obtain first weights and second weights of the first output result and the second output result;

[0191] The discriminant unit 204 is used to perform weighted fusion on the first output result and the second output result based on the first weight and the second weight to obtain the detection result of the first vehicle damage image output by the pre-trained discriminator network.

[0192] Furthermore, before acquiring the first vehicle damage image to be detected, the method further includes:

[0193] Acquire a second vehicle damage image including true segmentation annotations;

[0194] Initialize the weights of the object detection and instance segmentation models;

[0195] Based on the first preset damage function and the first preset optimizer, the target detection and instance segmentation model is iteratively trained through the second vehicle damage image to obtain a pre-trained vehicle damaged parts segmentation model.

[0196] Furthermore, before acquiring the first vehicle damage image to be detected, the method further includes:

[0197] Obtain a third vehicle damage image containing a true mask annotation;

[0198] Based on a grid-like dense sampling point method, a prompt point set of the third vehicle damage image is generated;

[0199] The cue point set is used as the input of the image segmentation model to obtain the predicted mask annotation output by the image segmentation model;

[0200] The difference between the real mask annotation and the predicted mask annotation is used as the second preset damage function. Combined with the preset parameter fine-tuning method and the second preset optimizer, the image segmentation model is iteratively trained to obtain a pre-fine-tuned damage segmentation model.

[0201] Furthermore, before acquiring the first vehicle damage image to be detected, the method further includes:

[0202] Initialize the weights of the pre-trained generator network and discriminator network;

[0203] The collection of the second vehicle damage image and the third vehicle damage image is used as a training set, and the training set is respectively input into a pre-trained vehicle damage component segmentation model and a pre-fine-tuned damage segmentation large model to obtain a third output result and a fourth output result;

[0204] The third output result and the fourth output result are used as inputs of the generator network, combined with the true segmentation annotations of the second vehicle damage image and the third vehicle damage image and the true mask annotations, and according to the output of the discriminator network, the generator network and the discriminator network are alternately iteratively trained to obtain pre-trained generator network and discriminator network.

[0205] Furthermore, the first output result and the second output result are used as inputs of the generator network, and the generator network performs feature fusion and feature extraction to obtain first weights and second weights of the first output result and the second output result, which specifically include:

[0206] Concatenate the first output result and the second output result in the channel dimension to obtain a concatenated dimension vector;

[0207] Inputting the first output result, the second output result, and the vector dimension into the generator network, fusing the first features of the first output result and the second output result by a skip connection or an attention mechanism of the generator network, and extracting the second feature of the dimension vector by a convolution layer and an activation function of the generator network;

[0208] According to the first feature and the second feature, a first weight and a second weight of the first output result and the second output result are obtained respectively.

[0209] Furthermore, after weighted fusion of the first output result and the second output result based on the first weight and the second weight, the detection result of the first vehicle damage image output by the pre-trained discriminator network specifically includes:

[0210] Obtaining a weighted average of the first output result and the second output result according to the first weight and the second weight;

[0211] The weighted average value is used as the input of the pre-trained discriminator network, and the discriminator network performs a binary classification judgment to obtain a detection result of the discriminator network on the first vehicle damage image, where the detection result includes true or false.

[0212] Further, after obtaining the detection result of the first vehicle damage image output by the pre-trained discriminator network, the method further includes:

[0213] Evaluate the confidence of the detection results output by the discriminator network based on a preset confidence evaluation model;

[0214] If the confidence level is not lower than a preset threshold, the detection result is used as the final damage detection result of the first vehicle damage image;

[0215] If the confidence level is lower than a preset threshold, the first vehicle damage image, the first output result, the second output result and the detection result are taken as questionable data sets and stored in the target database.

[0216] Another embodiment of the present invention provides an automobile component damage detection device, such as Figure 4 As shown, the device 10 includes:

[0217] One or more processors 110 and memory 120, Figure 3 A processor 110 is used as an example for description. The processor 110 and the memory 120 may be connected via a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0218] The processor 110 is used to complete various control logics of the device 10, and it can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISCMachine) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of these components. In addition, the processor 110 can also be any traditional processor, microprocessor or state machine. The processor 110 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0219] The memory 120 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions corresponding to the method for constructing a multilingual phoneme representation model in the embodiment of the present invention. The processor 110 executes various functional applications and data processing of the device 10 by running the non-volatile software programs, instructions and units stored in the memory 120, that is, implements the method for constructing a multilingual phoneme representation model in the above method embodiment.

[0220] The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created according to the use of the device 10, etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 120 may optionally include a memory remotely arranged relative to the processor 110, and these remote memories may be connected to the device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0221] One or more units are stored in the memory 120, and when executed by one or more processors 110, the following steps are implemented:

[0222] Acquire a first vehicle damage image to be detected;

[0223] Inputting the first vehicle damage image into a pre-trained vehicle damaged parts segmentation model and a pre-fine-tuned damage segmentation large model respectively, to obtain a first output result and a second output result;

[0224] The first output result and the second output result are used as inputs of a pre-trained generator network, and the generator network performs feature fusion and feature extraction to obtain first weights and second weights of the first output result and the second output result;

[0225] The first output result and the second output result are weightedly fused based on the first weight and the second weight to obtain a detection result of the first vehicle damage image output by the pre-trained discriminator network.

[0226] An embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, any one of the implementation methods of an automobile component damage detection method described in the above embodiments is implemented.

[0227] As an example, non-volatile storage media can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) as external cache memory. By way of illustration and not limitation, RAM can be obtained in many forms such as synchronous RAM (SRAM), dynamic RAM, (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The disclosed memory components or memories of the operating environment described herein are intended to include one or more of these and / or any other suitable types of memory.

[0228] In an embodiment of the present application, a method, device, equipment and storage medium for detecting damage to automobile parts are provided. A pre-trained vehicle damage part segmentation model and a pre-fine-tuned damage segmentation large model are used to make a preliminary prediction of a vehicle damage image to be detected, and a first output result and a second output result are obtained respectively. The outputs of the two models are fused with the assistance of a generative adversarial network, so that a discriminator outputs a detection result of the vehicle damage image, and determines whether the vehicle damage image is damaged based on whether the detection result is true or false. Multi-model fusion can combine the advantages of different models to improve the accuracy of vehicle part damage detection, and solves the technical problems of insufficient accuracy in the prior art, lack of sufficient flexibility and adaptability when dealing with damage of different types and severity, and low detection efficiency, which reduces user satisfaction.

[0229] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0230] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0231] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0232] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0233] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0234] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0235] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (full name in English: Read-Only Memory, English abbreviation: ROM), random access memory (full name in English: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store program codes.

[0236] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0237] It should be noted that if software tools or components other than those of our company appear in the embodiments of the present application, they are only used for illustrative purposes and do not represent actual use.

Claims

1. A method for detecting damage to automobile parts, characterized in that: include: Acquire a first vehicle damage image to be detected; Inputting the first vehicle damage image into a pre-trained vehicle damaged parts segmentation model and a pre-fine-tuned damage segmentation large model respectively to obtain a first output result and a second output result; Using the first output result and the second output result as inputs of a pre-trained generator network, and having the generator network perform feature fusion and feature extraction to obtain a first weight and a second weight of the first output result and the second output result; The first output result and the second output result are weightedly fused based on the first weight and the second weight to obtain a detection result of the first vehicle damage image output by a pre-trained discriminator network.

2. The automobile component damage detection method according to claim 1, characterized in that: Before acquiring the first vehicle damage image to be detected, the method further includes: Acquire a second vehicle damage image including true segmentation annotations; Initialize the weights of the object detection and instance segmentation models; Based on the first preset damage function and the first preset optimizer, the target detection and instance segmentation model is iteratively trained through the second vehicle damage image to obtain a pre-trained vehicle damaged parts segmentation model.

3. The automobile component damage detection method according to claim 2, characterized in that: Before acquiring the first vehicle damage image to be detected, the method further includes: Obtain a third vehicle damage image containing a true mask annotation; Based on a grid-like dense sampling point method, generating a prompt point set of the third vehicle damage image; Using the cue point set as input of a large image segmentation model to obtain a predicted mask annotation output by the large image segmentation model; The difference between the real mask annotation and the predicted mask annotation is used as a second preset damage function. Combined with a preset parameter fine-tuning method and a second preset optimizer, the image segmentation large model is iteratively trained to obtain a pre-fine-tuned damage segmentation large model.

4. The automobile component damage detection method according to claim 3, characterized in that: Before acquiring the first vehicle damage image to be detected, the method further includes: Initialize the weights of the pre-trained generator network and discriminator network; Using a collection of the second vehicle damage image and the third vehicle damage image as a training set, and inputting the training set into a pre-trained vehicle damage component segmentation model and a pre-fine-tuned damage segmentation large model, respectively, to obtain a third output result and a fourth output result; The third output result and the fourth output result are used as inputs of the generator network, combined with the true segmentation annotations of the second vehicle damage image and the true mask annotations of the third vehicle damage image, and according to the output of the discriminator network, the generator network and the discriminator network are alternately iteratively trained to obtain pre-trained generator network and discriminator network.

5. The automobile component damage detection method according to claim 1, characterized in that: The step of using the first output result and the second output result as inputs of a generator network, performing feature fusion and feature extraction by the generator network, and obtaining first weights and second weights of the first output result and the second output result specifically includes: Concatenate the first output result and the second output result in the channel dimension to obtain a concatenated dimensional vector; Inputting the first output result, the second output result, and the vector dimension into a generator network, fusing the first features of the first output result and the second output result by a skip connection or an attention mechanism of the generator network, and extracting the second feature of the dimensional vector by a convolutional layer and an activation function of the generator network; A first weight and a second weight of the first output result and a second weight of the second output result are obtained respectively according to the first feature and the second feature.

6. The automobile component damage detection method according to claim 5, characterized in that: After weighted fusion of the first output result and the second output result based on the first weight and the second weight, the detection result of the first vehicle damage image output by the pre-trained discriminator network specifically includes: Obtaining a weighted average of the first output result and the second output result according to the first weight and the second weight; The weighted average value is used as an input of a pre-trained discriminator network, and the discriminator network performs a binary classification judgment to obtain a detection result of the discriminator network on the first vehicle damage image, where the detection result includes true or false.

7. The automobile component damage detection method according to claim 1, characterized in that: After obtaining the detection result of the first vehicle damage image output by the pre-trained discriminator network, the method further includes: Evaluating the confidence of the detection result output by the discriminator network based on a preset confidence evaluation model; If the confidence level is not lower than a preset threshold, taking the detection result as a final damage detection result of the first vehicle damage image; If the confidence level is lower than the preset threshold, the first vehicle damage image, the first output result, the second output result and the detection result are taken as a questionable data set and stored in a target database.

8. An automobile component damage detection device, characterized in that: include: An acquisition unit, used for acquiring a first vehicle damage image to be detected; A processing unit, configured to input the first vehicle damage image into a pre-trained vehicle damaged parts segmentation model and a pre-fine-tuned damage segmentation large model, respectively, to obtain a first output result and a second output result; A generating unit, configured to use the first output result and the second output result as inputs of a pre-trained generator network, and the generator network performs feature fusion and feature extraction to obtain a first weight and a second weight of the first output result and the second output result; A discriminant unit is used to perform weighted fusion on the first output result and the second output result based on the first weight and the second weight to obtain a detection result of the first vehicle damage image output by a pre-trained discriminator network.

9. An automobile component damage detection device, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the automobile component damage detection method according to any one of claims 1-7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and the program codes are used to execute the automobile component damage detection method according to any one of claims 1 to 7.

Images