Vehicle loss assessment method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a vehicle loss assessment method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring a to-be-assessed vehicle picture; inputting the picture of the vehicle to be subjected to loss assessment into an encoder to generate potential features; inputting the potential features into a decoder to obtain a target feature vector; converting the potential features into multi-scale features through a multi-scale adapter; inputting the multi-scale features into an auxiliary detection head, and generating a plurality of candidate region feature matrixes; adopting a one-to-many matching mode to match the plurality of candidate region feature matrixes with the marked truth value to obtain a matched target region feature matrix; and according to the target region feature matrix and the target feature vector, obtaining a vehicle loss assessment result corresponding to the picture of the vehicle to be subjected to loss assessment. According to the embodiment of the invention, the feature expression capability of the encoder in the DETR framework can be improved, so that the accuracy of applying the DETR framework to the automatic loss assessment scene of the vehicle is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a vehicle damage assessment method and device, electronic equipment and storage medium. Background Art

[0002] DETR (DEtection TRansformer) is an end-to-end object detection network based on Transformer, which has been widely used in the field of computer vision. When DETR is applied to the automatic damage assessment of vehicles, the object detection of vehicle damage can be transformed into a set prediction problem. This process is similar to the role of traditional anchor points, which converts the vehicle damage image sequence into a set sequence. However, most of the current research on DETR in the industry focuses on the one-to-one matching mechanism. However, experimental results show that the proposed one-to-one matching mechanism leads to unstable losses in the early stage of model training, and the real label cannot be stably matched with the query. Moreover, when DETR identifies smaller damage on vehicles, the model performance and training strategy need to be optimized. Therefore, the accuracy of the existing DETR framework is not high when applied to the automatic damage assessment of vehicles. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a vehicle damage assessment method and device, an electronic device and a storage medium, which can improve the feature expression capability of the encoder in the DETR framework, thereby effectively improving the accuracy of the DETR framework in the vehicle automatic damage assessment scenario, making the DETR framework more suitable for vehicle damage detection.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application provides a vehicle damage assessment method, the method comprising:

[0005] Obtain images of vehicles to be damaged;

[0006] Inputting the image of the vehicle to be damaged into an encoder to generate potential features;

[0007] Inputting the latent features into a decoder to obtain a target feature vector;

[0008] Converting the latent features into multi-scale features through a multi-scale adapter;

[0009] Inputting the multi-scale features into an auxiliary detection head to generate multiple candidate region feature matrices;

[0010] A one-to-many matching method is used to match the plurality of candidate region feature matrices with the annotated true values ​​to obtain a matched target region feature matrix;

[0011] According to the target area feature matrix and the target feature vector, a vehicle damage assessment result corresponding to the image of the vehicle to be assessed is obtained.

[0012] In some embodiments, the method is applied to a vehicle damage assessment model, which includes the encoder, the decoder, the multi-scale adapter and the auxiliary detection head, the output end of the encoder is respectively connected to the input end of the decoder and the input end of the multi-scale adapter, the output end of the multi-scale adapter is connected to the input end of the auxiliary detection head, the output end of the decoder and the output end of the auxiliary detection head are spliced ​​together, the branch where the encoder and the decoder are located constitutes a main branch, and the branch where the encoder, the multi-scale adapter and the auxiliary detection head are located constitutes an auxiliary branch.

[0013] In some embodiments, the vehicle damage assessment model training method includes:

[0014] Using the Hungarian matching function to calculate the first loss function of the main branch;

[0015] A Focal Loss function is used to calculate a second loss function of the auxiliary branch;

[0016] Determine a target loss function according to the first loss function and the second loss function;

[0017] The vehicle damage assessment model is trained based on the target loss function to obtain the trained vehicle damage assessment model.

[0018] In some embodiments, the step of inputting the image of the vehicle to be assessed for damage into an encoder to generate potential features includes:

[0019] Performing bilinear interpolation processing on the image of the vehicle to be assessed for damage to obtain a preprocessed image;

[0020] The preprocessed image is input into an encoder to generate the latent features.

[0021] In some embodiments, the step of performing bilinear interpolation processing on the image of the vehicle to be damaged to obtain a preprocessed image includes:

[0022] Determine four corner points around each pixel in the image of the vehicle to be damaged;

[0023] Determine the pixel value of each pixel point according to the weighted average value of the four corner points;

[0024] The image of the vehicle to be damaged is converted into the pre-processed image according to the pixel value of each pixel point.

[0025] In some embodiments, the one-to-many matching method is used to match the plurality of candidate region feature matrices with the annotated true values ​​to obtain a matched target region feature matrix, including:

[0026] Matching each of the candidate region feature matrices with the annotated true value one by one, to obtain an overlapping area ratio value corresponding to an overlapping area between each of the candidate region feature matrixes and the annotated true value;

[0027] The candidate region feature matrix whose overlapping area ratio value is greater than a preset threshold is determined as the target region feature matrix.

[0028] In some embodiments, the encoder is a Transformer encoder, the decoder is a Transformer decoder, the multi-scale adapter adopts an identity mapping structure, and the auxiliary detection head is an adaptive training sample selection ATSS auxiliary detection head.

[0029] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a vehicle damage assessment device, the device comprising:

[0030] An acquisition module is used to obtain images of vehicles to be assessed for damage;

[0031] An encoding module, used for inputting the image of the vehicle to be damaged into an encoder to generate potential features;

[0032] A decoding module, used for inputting the latent features into a decoder to obtain a target feature vector;

[0033] A conversion module, for converting the latent features into multi-scale features through a multi-scale adapter;

[0034] A generation module, used for inputting the multi-scale features into an auxiliary detection head to generate a plurality of candidate region feature matrices;

[0035] A matching module, used to match the plurality of candidate region feature matrices with the annotated true values ​​in a one-to-many matching manner to obtain a matched target region feature matrix;

[0036] The output module is used to obtain the vehicle damage assessment result corresponding to the image of the vehicle to be assessed based on the target area feature matrix and the target feature vector.

[0037] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0038] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0039] The vehicle damage assessment method and device, electronic device and storage medium proposed in the present application obtain the image of the vehicle to be assessed; input the image of the vehicle to be assessed into the encoder to generate potential features; input the potential features into the decoder to obtain the target feature vector; convert the potential features into multi-scale features through a multi-scale adapter; input the multi-scale features into the auxiliary detection head to generate multiple candidate area feature matrices; use a one-to-many matching method to match multiple candidate area feature matrices with the annotated true value to obtain a matched target area feature matrix; according to the target area feature matrix and the target feature vector, obtain the vehicle damage assessment result corresponding to the image of the vehicle to be assessed. The embodiment of the present application introduces an auxiliary detection head on the basis of the existing DETR framework, so that the potential features output by the encoder will be additionally input into the auxiliary detection head to generate multiple candidate area feature matrices, and use a one-to-many matching method to match multiple candidate area feature matrices with the annotated true value to obtain a matched target area feature matrix. Then, the matched target area feature matrix is ​​spliced ​​with the target feature vector output by the decoder, and finally the vehicle damage assessment result corresponding to the image of the vehicle to be assessed is output. Based on this, the embodiments of the present application can improve the feature expression capability of the encoder in the DETR framework, thereby effectively improving the accuracy of the DETR framework in the vehicle automatic damage assessment scenario, making the DETR framework more suitable for vehicle damage detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flow chart of a vehicle damage assessment method provided in an embodiment of the present application;

[0041] Figure 2 It is a structural schematic diagram of a vehicle damage assessment model provided in an embodiment of the present application;

[0042] Figure 3 is a flow chart of a training method for a vehicle damage assessment model provided in an embodiment of the present application;

[0043] Figure 4 yes Figure 1 Flow chart of step S102 in FIG.

[0044] Figure 5 yes Figure 4 Flow chart of step S401 in FIG.

[0045] Figure 6 yes Figure 1 Flow chart of step S106 in FIG.

[0046] Figure 7 is a schematic diagram of the structure of a vehicle damage assessment device provided in an embodiment of the present application;

[0047] Figure 8 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0051] First, some nouns involved in this application are analyzed:

[0052] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0053] DETR (DEtection TRansformer): is a Transformer-based end-to-end object detection model proposed by the Facebook AI Research team. It uses the Transformer architecture to convert the object detection task into a set prediction problem, that is, predicting the category, location, and number of objects by matching the encoding of the input image and the target set. DETR is characterized by not using NMS post-processing steps and anchors. Its performance is comparable to Faster RCNN and can be easily migrated to other tasks such as panoramic segmentation. The introduction of DETR represents a new approach in the field of object detection, bringing alternatives to traditional methods and demonstrating the potential of using Transformers for object detection.

[0054] ATSS (Adaptive Training Sample Selection): is an adaptive training sample selection method that aims to automatically select positive and negative training samples based on the statistical characteristics of the object. The core idea of ​​this method is that in object detection, the key to the performance difference between anchor-based and anchor-free detectors lies in the definition of positive and negative samples. ATSS experimentally demonstrates the essential difference between the two methods and proposes an adaptive training sample selection method to address the difference between anchor-free and anchor-based methods.

[0055] DETR (DEtection TRansformer) is an end-to-end object detection network based on Transformer, which has been widely used in the field of computer vision. When DETR is applied to the automatic damage assessment of vehicles, the object detection of vehicle damage can be transformed into a set prediction problem. This process is similar to the role of traditional anchor points, which converts the vehicle damage image sequence into a set sequence. However, most of the current research on DETR in the industry focuses on the one-to-one matching mechanism. However, experimental results show that the proposed one-to-one matching mechanism leads to unstable losses in the early stage of model training, and the real label cannot be stably matched with the query. Moreover, when DETR identifies smaller damage on vehicles, the model performance and training strategy need to be optimized. Therefore, the accuracy of the existing DETR framework is not high when applied to the automatic damage assessment of vehicles.

[0056] Based on this, the embodiment of the present application provides a vehicle damage assessment method and device, electronic device and storage medium, obtains the image of the vehicle to be assessed; inputs the image of the vehicle to be assessed into the encoder to generate potential features; inputs the potential features into the decoder to obtain the target feature vector; converts the potential features into multi-scale features through a multi-scale adapter; inputs the multi-scale features into an auxiliary detection head to generate multiple candidate area feature matrices; uses a one-to-many matching method to match multiple candidate area feature matrices with the annotated true values ​​to obtain a matched target area feature matrix; obtains the vehicle damage assessment result corresponding to the image of the vehicle to be assessed according to the target area feature matrix and the target feature vector. The embodiment of the present application adds an auxiliary detection head on the basis of the existing DETR framework, so that the potential features output by the encoder will be additionally input into the auxiliary detection head to generate multiple candidate area feature matrices, and uses a one-to-many matching method to match multiple candidate area feature matrices with the annotated true values ​​to obtain a matched target area feature matrix. Then the matched target area feature matrix is ​​spliced ​​with the target feature vector output by the decoder, and finally the vehicle damage assessment result corresponding to the image of the vehicle to be assessed is output. Based on this, the embodiments of the present application can improve the feature expression capability of the encoder in the DETR framework, thereby effectively improving the accuracy of the DETR framework in the vehicle automatic damage assessment scenario, making the DETR framework more suitable for vehicle damage detection.

[0057] The vehicle damage assessment method and device, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the vehicle damage assessment method in the embodiments of the present application is described.

[0058] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0059] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0060] The vehicle damage assessment method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The vehicle damage assessment method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the vehicle damage assessment method, etc., but is not limited to the above forms.

[0061] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0062] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0063] Figure 1 is an optional flow chart of the vehicle damage assessment method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S107.

[0064] Step S101, obtaining a picture of the vehicle to be assessed for damage;

[0065] Step S102, inputting the image of the vehicle to be damaged into the encoder to generate potential features;

[0066] Step S103, inputting the latent features into the decoder to obtain the target feature vector;

[0067] Step S104, converting the latent features into multi-scale features through a multi-scale adapter;

[0068] Step S105, inputting the multi-scale features into the auxiliary detection head to generate multiple candidate region feature matrices;

[0069] Step S106, matching multiple candidate region feature matrices with the labeled true values ​​in a one-to-many matching manner to obtain a matched target region feature matrix;

[0070] Step S107, obtaining a vehicle damage assessment result corresponding to the image of the vehicle to be assessed based on the target area feature matrix and the target feature vector.

[0071] In step S101 of some embodiments, a picture of the vehicle to be assessed is obtained. Specifically, the picture of the vehicle to be assessed uploaded by the user can be obtained from a mobile terminal or a cloud server. The picture of the vehicle to be assessed can be obtained by photographing the vehicle to be assessed from different angles using a mobile device with a camera, and the vehicle to be assessed may include a number of different damaged vehicle parts. In the automatic vehicle damage assessment scenario, the user can use the self-service vehicle damage claim service interface, and the customer can assess the vehicle damage by photographing the damaged vehicle parts and uploading them to the vehicle damage assessment system.

[0072] In step S102 of some embodiments, the image of the vehicle to be damaged is input into an encoder to generate potential features. Specifically, the image of the vehicle to be damaged is input into a pre-trained encoder, and the encoder can convert the image of the vehicle to be damaged into a series of values, i.e., potential features, which capture key information in the image of the vehicle to be damaged, such as the damage to the vehicle.

[0073] In step S103 of some embodiments, the latent features are input to the decoder to obtain a target feature vector. The latent features obtained from the encoder are passed as input to the decoder, which processes the latent features and outputs a target feature vector, which can be used to represent the damage category and damage degree of the vehicle as part of the vehicle damage assessment test result.

[0074] In step S104 of some embodiments, the latent features are converted into multi-scale features by a multi-scale adapter. The multi-scale adapter can be used to convert the latent features output by the encoder into a multi-scale feature pyramid {F1, F2..., F J}. A multi-scale adapter can be one or more neural network layers that are able to receive latent features and output feature representations at different scales. A multi-scale adapter can contain convolution kernels of different sizes, pooling layers, upsampling layers, or attention mechanisms to capture features at different scales. Based on this, a multi-scale adapter can convert latent features into multi-scale features, which can capture information at different levels, thereby improving the performance and robustness of the model in processing complex tasks. In the scenario of automatic vehicle damage assessment, the use of multi-scale features can help the model capture key information in vehicle images more accurately and generate more reliable prediction results.

[0075] In step S105 of some embodiments, the multi-scale features are input into the auxiliary detection head to generate multiple candidate region feature matrices. J} is sent to the auxiliary detection head, which uses a one-to-many label allocation method. In the embodiment of the present application, an auxiliary detection head is added on the basis of the existing DETR framework, so that the potential features output by the encoder will be additionally input into the auxiliary detection head to generate multiple candidate region feature matrices. It should be noted that the auxiliary detection head can adopt a traditional ATSS detection head, which is a lightweight structure, so the additional training overhead is small.

[0076] In step S106 of some embodiments, a one-to-many matching method is used to match multiple candidate region feature matrices with the labeled true value to obtain a matched target region feature matrix. Multiple candidate region feature matrices contain partial features of the detected target candidate region, and these multiple candidate region feature matrices will be matched with the labeled true value to obtain a target region feature matrix that is relatively close to the labeled true value.

[0077] In step S107 of some embodiments, the vehicle damage assessment result corresponding to the image of the vehicle to be assessed is obtained based on the target area feature matrix and the target feature vector. The matched target area feature matrix and the target feature vector output by the decoder can be concatenated to finally output the vehicle damage assessment result corresponding to the image of the vehicle to be assessed.

[0078] Steps S101 to S107 shown in the embodiment of the present application are to obtain a picture of a vehicle to be damaged; input the picture of the vehicle to be damaged into an encoder to generate potential features; input the potential features into a decoder to obtain a target feature vector; convert the potential features into multi-scale features through a multi-scale adapter; input the multi-scale features into an auxiliary detection head to generate multiple candidate area feature matrices; use a one-to-many matching method to match multiple candidate area feature matrices with the annotated true values ​​to obtain a matched target area feature matrix; obtain the vehicle damage assessment result corresponding to the picture of the vehicle to be damaged according to the target area feature matrix and the target feature vector. The embodiment of the present application adds an auxiliary detection head on the basis of the existing DETR framework, so that the potential features output by the encoder will be additionally input into the auxiliary detection head to generate multiple candidate area feature matrices, and use a one-to-many matching method to match multiple candidate area feature matrices with the annotated true values ​​to obtain a matched target area feature matrix. Then, the matched target area feature matrix is ​​spliced ​​with the target feature vector output by the decoder, and finally the vehicle damage assessment result corresponding to the picture of the vehicle to be damaged is output. Based on this, the embodiments of the present application can improve the feature expression capability of the encoder in the DETR framework, thereby effectively improving the accuracy of the DETR framework in the vehicle automatic damage assessment scenario, making the DETR framework more suitable for vehicle damage detection.

[0079] In some embodiments, the vehicle damage assessment method of the embodiment of the present application can be applied to a vehicle damage assessment model, such as Figure 2 As shown, the vehicle damage assessment model includes an encoder, a decoder, a multi-scale adapter and an auxiliary detection head.

[0080] Compared to the existing DETR framework, the vehicle damage assessment model of the embodiment of the present application mainly adds a multi-scale adapter and an auxiliary detection head on its basis, thereby forming two branches, namely a one-to-one matching main branch and a one-to-many matching auxiliary branch. Specifically, the output end of the encoder is respectively connected to the input end of the decoder and the input end of the multi-scale adapter, the output end of the multi-scale adapter is connected to the input end of the auxiliary detection head, the output end of the decoder and the output end of the auxiliary detection head are spliced ​​together, the branch where the encoder and the decoder are located constitutes the main branch, and the branch where the encoder, the multi-scale adapter and the auxiliary detection head are located constitutes the auxiliary branch. The vehicle damage assessment model of the embodiment of the present application integrates the auxiliary head with the output of the encoder, and uses one-to-many label allocation to improve the training efficiency and effect of the encoder and decoder, making the DETR framework more suitable for vehicle damage detection.

[0081] Based on this, the vehicle damage assessment method of the embodiment of the present application adds an auxiliary detection head on the basis of the existing DETR framework, so that the potential features output by the encoder will be additionally input into the auxiliary detection head to generate multiple candidate region feature matrices, and a one-to-many matching method is used to match the multiple candidate region feature matrices with the annotated true values ​​to obtain a matched target region feature matrix. The matched target region feature matrix is ​​then spliced ​​with the target feature vector output by the decoder, and finally the vehicle damage assessment result corresponding to the image of the vehicle to be assessed is output.

[0082] In some embodiments, the vehicle damage assessment model of the embodiment of the present application adopts a Transformer encoder as the encoder, a Transformer decoder as the decoder, an identity mapping structure as the multi-scale adapter, and an ATSS auxiliary detection head as the auxiliary detection head selected for adaptive training samples.

[0083] See also Figure 3 In some embodiments, the training method of the vehicle damage assessment model may include but is not limited to steps S301 to S304:

[0084] Step S301, using the Hungarian matching function to calculate the first loss function of the main branch;

[0085] Step S302, using the Focal Loss function to calculate the second loss function of the auxiliary branch;

[0086] Step S303, determining a target loss function according to the first loss function and the second loss function;

[0087] Step S304, training the vehicle damage assessment model based on the target loss function to obtain a trained vehicle damage assessment model.

[0088] During the training of the vehicle damage assessment model, a collaborative hybrid distribution training method is adopted to improve feature learning in the encoder and attention learning in the decoder.

[0089] The vehicle damage assessment model has two branches, the main branch and the auxiliary branch. The main branch uses one-to-one matching, and the auxiliary branch uses one-to-many matching. During the training phase, the loss function of the entire algorithm consists of two parts, the original one-to-one set matching branch, and the loss of the decoder layer in the auxiliary branch. The formula is:

[0090] L global =L dec +αL enc

[0091] Among them, L dec represents the first loss function of the main branch, which uses the Hungarian matching function; L encrepresents the loss of the L-th decoder layer in the auxiliary branch, using the Focal Loss function, α is the balanced loss coefficient, αL enc The second loss function of the auxiliary branch is formed. All queries in the auxiliary branch are considered positive queries. The vehicle damage assessment model trained by collaborative hybrid allocation can be used for target detection and vehicle damage assessment. After inputting the image X of the vehicle to be assessed into the trained vehicle damage assessment model, the prediction result Y can be obtained.

[0092] Based on this, this application proposes a collaborative hybrid allocation training method for vehicle automatic damage assessment to learn a higher DETR-based detector from a variety of label allocation methods. This new training method can improve the encoder feature learning ability by training multiple auxiliary heads supervised by one-to-many label allocation. This method extracts positive sample coordinates from these auxiliary heads to perform additional customized positive queries to improve the training efficiency of positive samples in the decoder.

[0093] See also Figure 4 In some embodiments, step S102 may include but is not limited to steps S401 to S402:

[0094] Step S401, performing bilinear interpolation processing on the image of the vehicle to be assessed to obtain a preprocessed image;

[0095] Step S402: input the preprocessed image into the encoder to generate latent features.

[0096] In some embodiments, bilinear interpolation is performed on the image of the vehicle to be assessed, which has the advantages of accurate normal calculation, accurate highlight domain, no restrictions on light sources and viewpoints, and fast speed. The preprocessed image is input to the encoder to generate potential features. The encoder can convert the preprocessed image into a series of values, i.e., potential features, which capture key information in the image of the vehicle to be assessed, including the damage to the vehicle.

[0097] See also Figure 5 In some embodiments, step S401 may include but is not limited to steps S501 to S502:

[0098] Step S501, determining four corner points around each pixel in the image of the vehicle to be damaged;

[0099] Step S502, determining the pixel value of each pixel point according to the weighted average value of the four corner points;

[0100] Step S503, converting the image of the vehicle to be damaged into a pre-processed image according to the pixel value of each pixel point.

[0101] In some embodiments, bilinear interpolation processing is performed on the damaged vehicle image X to be assessed. Specifically, each pixel point H(x, y) on the damaged vehicle image X to be assessed is processed.

[0102]

[0103]

[0104] Among them, Q11(x1,y1,Q12(x1,y2),Q21(x2,y1),Q22(x2,y1) are the four points around the pixel H. Through this operation, the image is converted from the original image (X) 1000×667 to the new image 624×328.

[0105] See also Figure 6 In some embodiments, step S106 may include but is not limited to steps S601 to S602:

[0106] Step S601, matching each candidate region feature matrix with the annotated true value one by one, and obtaining an overlapping area ratio value corresponding to the overlapping area between each candidate region feature matrix and the annotated true value;

[0107] Step S602: Determine the candidate region feature matrix whose overlap area ratio value is greater than a preset threshold as the target region feature matrix.

[0108] In some embodiments, in the auxiliary branch of the one-to-many matching of the vehicle damage assessment model, each candidate region feature matrix can be compared with the labeled true value one by one to obtain the overlapping area ratio value corresponding to the overlapping area between each candidate region feature matrix and the labeled true value. All candidate region feature matrices whose overlapping area ratio values ​​exceed the preset threshold represent that these candidate region feature matrices are relatively close to the labeled true value, and therefore, these candidate region feature matrices will be judged as target region feature matrices. It should be noted that the preset threshold can be configured according to actual business requirements, and this embodiment does not limit the specific value of the preset threshold.

[0109] See also Figure 7 The embodiment of the present application also provides a vehicle damage assessment device, which can implement the above-mentioned vehicle damage assessment method, and the device includes:

[0110] An acquisition module 710 is used to acquire a picture of a vehicle to be assessed for damage;

[0111] The encoding module 720 is used to input the image of the vehicle to be damaged into the encoder to generate potential features;

[0112] A decoding module 730, used to input the latent features into a decoder to obtain a target feature vector;

[0113] A conversion module 740, for converting latent features into multi-scale features through a multi-scale adapter;

[0114] A generating module 750 is used to input the multi-scale features into the auxiliary detection head to generate multiple candidate region feature matrices;

[0115] A matching module 760 is used to match multiple candidate region feature matrices with labeled true values ​​in a one-to-many matching manner to obtain a matched target region feature matrix;

[0116] The output module 770 is used to obtain the vehicle damage assessment result corresponding to the image of the vehicle to be assessed based on the target area feature matrix and the target feature vector.

[0117] In some embodiments of the present application, the acquisition module 710 acquires a picture of a vehicle to be damaged; the encoding module 720 inputs the picture of the vehicle to be damaged into the encoder to generate potential features; the decoding module 730 inputs the potential features into the decoder to obtain a target feature vector; the conversion module 740 converts the potential features into multi-scale features through a multi-scale adapter; the generation module 750 inputs the multi-scale features into an auxiliary detection head to generate multiple candidate area feature matrices; the matching module 760 uses a one-to-many matching method to match multiple candidate area feature matrices with the annotated true values ​​to obtain a matched target area feature matrix; the output module 770 obtains the vehicle damage assessment result corresponding to the picture of the vehicle to be damaged based on the target area feature matrix and the target feature vector.

[0118] In some embodiments of the present application, images of vehicles to be assessed for damage are obtained. Specifically, images of vehicles to be assessed for damage uploaded by users can be obtained from a mobile terminal or a cloud server. Images of vehicles to be assessed for damage can be obtained by photographing the vehicles to be assessed for damage from different angles using a mobile device with a camera, and the vehicles to be assessed for damage may include vehicles with multiple different damaged parts. In the automatic vehicle damage assessment scenario, users can access the vehicle damage self-service claims service interface, where customers can assess vehicle damage by photographing damaged vehicle parts and uploading them to the vehicle damage assessment system.

[0119] In some embodiments of the present application, a picture of a vehicle to be damaged is input into an encoder to generate potential features. Specifically, the picture of the vehicle to be damaged is input into a pre-trained encoder, and the encoder can convert the picture of the vehicle to be damaged into a series of values, i.e., potential features, which capture key information in the picture of the vehicle to be damaged, such as the damage to the vehicle.

[0120] In some embodiments of the present application, the latent features are input to the decoder to obtain a target feature vector. The latent features obtained from the encoder are passed as input to the decoder, which processes the latent features and outputs a target feature vector, which can be used to represent the damage category and damage degree of the vehicle as part of the vehicle damage assessment test result.

[0121] In some embodiments of the present application, the latent features are converted into multi-scale features by a multi-scale adapter. The multi-scale adapter can be used to convert the latent features output by the encoder into a multi-scale feature pyramid {F1, F2..., F J}. A multi-scale adapter can be one or more neural network layers that are able to receive latent features and output feature representations at different scales. A multi-scale adapter can contain convolution kernels of different sizes, pooling layers, upsampling layers, or attention mechanisms to capture features at different scales. Based on this, a multi-scale adapter can convert latent features into multi-scale features, which can capture information at different levels, thereby improving the performance and robustness of the model in processing complex tasks. In the scenario of automatic vehicle damage assessment, the use of multi-scale features can help the model capture key information in vehicle images more accurately and generate more reliable prediction results.

[0122] In some embodiments of the present application, the multi-scale features are input to the auxiliary detection head to generate multiple candidate region feature matrices. J} is sent to the auxiliary detection head, which uses a one-to-many label allocation method. In the embodiment of the present application, an auxiliary detection head is added on the basis of the existing DETR framework, so that the potential features output by the encoder will be additionally input into the auxiliary detection head to generate multiple candidate region feature matrices. It should be noted that the auxiliary detection head can adopt a traditional ATSS detection head, which is a lightweight structure, so the additional training overhead is small.

[0123] In some embodiments of the present application, a one-to-many matching method is used to match multiple candidate region feature matrices with the labeled true value to obtain a matched target region feature matrix. Multiple candidate region feature matrices contain partial features of the detected target candidate region, and these multiple candidate region feature matrices will be matched with the labeled true value to obtain a target region feature matrix that is relatively close to the labeled true value.

[0124] In some embodiments of the present application, a vehicle damage assessment result corresponding to the image of the vehicle to be assessed is obtained based on the target area feature matrix and the target feature vector. The matched target area feature matrix and the target feature vector output by the decoder can be concatenated to finally output the vehicle damage assessment result corresponding to the image of the vehicle to be assessed.

[0125] Based on this, in the vehicle damage assessment device of the embodiment of the present application, the acquisition module 710 acquires the image of the vehicle to be assessed; the encoding module 720 inputs the image of the vehicle to be assessed into the encoder to generate potential features; the decoding module 730 inputs the potential features into the decoder to obtain the target feature vector; the conversion module 740 converts the potential features into multi-scale features through a multi-scale adapter; the generation module 750 inputs the multi-scale features into the auxiliary detection head to generate multiple candidate area feature matrices; the matching module 760 uses a one-to-many matching method to match the multiple candidate area feature matrices with the labeled true values ​​to obtain the matched target area feature matrix; the output module 770 obtains the vehicle damage assessment result corresponding to the image of the vehicle to be assessed based on the target area feature matrix and the target feature vector. The present application obtains a picture of a vehicle to be damaged; inputs the picture of the vehicle to be damaged into an encoder to generate potential features; inputs the potential features into a decoder to obtain a target feature vector; converts the potential features into multi-scale features through a multi-scale adapter; inputs the multi-scale features into an auxiliary detection head to generate multiple candidate area feature matrices; uses a one-to-many matching method to match multiple candidate area feature matrices with the annotated true values ​​to obtain a matched target area feature matrix; obtains a vehicle damage assessment result corresponding to the picture of the vehicle to be damaged based on the target area feature matrix and the target feature vector. The present application embodiment introduces an auxiliary detection head on the basis of the existing DETR framework, so that the potential features output by the encoder will be additionally input into the auxiliary detection head to generate multiple candidate area feature matrices, and uses a one-to-many matching method to match multiple candidate area feature matrices with the annotated true values ​​to obtain a matched target area feature matrix. The matched target area feature matrix is ​​then spliced ​​with the target feature vector output by the decoder, and finally the vehicle damage assessment result corresponding to the picture of the vehicle to be damaged is output. Based on this, the embodiments of the present application can improve the feature expression capability of the encoder in the DETR framework, thereby effectively improving the accuracy of the DETR framework in the vehicle automatic damage assessment scenario, making the DETR framework more suitable for vehicle damage detection.

[0126] The specific implementation of the vehicle damage assessment device is basically the same as the specific implementation of the above-mentioned vehicle damage assessment method, and will not be repeated here.

[0127] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned vehicle damage assessment method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0128] See also Figure 8 , Figure 8 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0129] The processor 801 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided in the embodiment of the present application.

[0130] The memory 802 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 802, and the processor 801 is called to execute the vehicle damage assessment method of the embodiment of this application, that is, by obtaining a picture of the vehicle to be assessed; inputting the picture of the vehicle to be assessed into an encoder to generate potential features; inputting the potential features into a decoder to obtain a target feature vector; converting the potential features into multi-scale features through a multi-scale adapter; inputting the multi-scale features into an auxiliary detection head to generate multiple candidate region feature matrices; using a one-to-many matching method to match multiple candidate region feature matrices with the annotated true value to obtain a matched target region feature matrix; according to the target region feature matrix and the target feature vector, a vehicle damage assessment result corresponding to the picture of the vehicle to be assessed is obtained. The embodiment of the present application introduces an auxiliary detection head on the basis of the existing DETR framework, so that the potential features output by the encoder will be additionally input into the auxiliary detection head to generate multiple candidate area feature matrices, and a one-to-many matching method is used to match the multiple candidate area feature matrices with the annotated true values ​​to obtain a matched target area feature matrix. The matched target area feature matrix is ​​then spliced ​​with the target feature vector output by the decoder, and finally the vehicle damage assessment result corresponding to the image of the vehicle to be assessed is output. Based on this, the embodiment of the present application can improve the feature expression capability of the encoder in the DETR framework, thereby effectively improving the accuracy of the DETR framework in the vehicle automatic damage assessment scenario, making the DETR framework more suitable for vehicle damage detection.

[0131] The input / output interface 803 is used to implement information input and output.

[0132] The communication interface 804 is used to realize the communication interaction between this device and other devices. Communication can be realized through wired methods (such as USB, network cable, etc.) or wireless methods (such as mobile network, WIFI, Bluetooth, etc.).

[0133] The bus transmits information between various components of the device (eg, the processor 801 , the memory 802 , the input / output interface 803 , and the communication interface 804 ).

[0134] The processor 801 , the memory 802 , the input / output interface 803 and the communication interface 804 are connected to each other in communication within the device via a bus.

[0135] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned vehicle damage assessment method is implemented.

[0136] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0137] The vehicle damage assessment method, vehicle damage assessment device, electronic device and storage medium provided in the embodiment of the present application are obtained by obtaining a picture of the vehicle to be assessed; inputting the picture of the vehicle to be assessed into an encoder to generate potential features; inputting the potential features into a decoder to obtain a target feature vector; converting the potential features into multi-scale features through a multi-scale adapter; inputting the multi-scale features into an auxiliary detection head to generate multiple candidate area feature matrices; using a one-to-many matching method to match multiple candidate area feature matrices with the annotated true values ​​to obtain a matched target area feature matrix; according to the target area feature matrix and the target feature vector, a vehicle damage assessment result corresponding to the picture of the vehicle to be assessed is obtained. The embodiment of the present application introduces an auxiliary detection head on the basis of the existing DETR framework, so that the potential features output by the encoder will be additionally input into the auxiliary detection head to generate multiple candidate area feature matrices, and a one-to-many matching method is used to match multiple candidate area feature matrices with the annotated true values ​​to obtain a matched target area feature matrix. Then the matched target area feature matrix is ​​spliced ​​with the target feature vector output by the decoder, and finally the vehicle damage assessment result corresponding to the picture of the vehicle to be assessed is output. Based on this, the embodiments of the present application can improve the feature expression capability of the encoder in the DETR framework, thereby effectively improving the accuracy of the DETR framework in the vehicle automatic damage assessment scenario, making the DETR framework more suitable for vehicle damage detection.

[0138] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as a computer-readable program, a data structure, a program module or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, disk storage or other magnetic storage device, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media generally contain computer-readable programs, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0139] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0140] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0141] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0142] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0143] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0144] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0145] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0146] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0147] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0148] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0149] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A vehicle damage assessment method, characterized in that: The method comprises: Obtain images of vehicles to be damaged; Inputting the image of the vehicle to be damaged into an encoder to generate potential features; Inputting the latent features into a decoder to obtain a target feature vector; Converting the latent features into multi-scale features through a multi-scale adapter; Inputting the multi-scale features into an auxiliary detection head to generate multiple candidate region feature matrices; A one-to-many matching method is used to match the plurality of candidate region feature matrices with the annotated true values ​​to obtain a matched target region feature matrix; According to the target area feature matrix and the target feature vector, a vehicle damage assessment result corresponding to the image of the vehicle to be assessed is obtained.

2. The method according to claim 1, characterized in that The method is applied to a vehicle damage assessment model, which includes the encoder, the decoder, the multi-scale adapter and the auxiliary detection head. The output end of the encoder is respectively connected to the input end of the decoder and the input end of the multi-scale adapter, the output end of the multi-scale adapter is connected to the input end of the auxiliary detection head, the output end of the decoder and the output end of the auxiliary detection head are spliced ​​together, the branch where the encoder and the decoder are located constitutes a main branch, and the branch where the encoder, the multi-scale adapter and the auxiliary detection head are located constitutes an auxiliary branch.

3. The method according to claim 2, characterized in that The training method of the vehicle damage assessment model includes: Using the Hungarian matching function to calculate the first loss function of the main branch; A Focal Loss function is used to calculate a second loss function of the auxiliary branch; Determine a target loss function according to the first loss function and the second loss function; The vehicle damage assessment model is trained based on the target loss function to obtain the trained vehicle damage assessment model.

4. The method according to claim 1, characterized in that: The step of inputting the image of the vehicle to be damaged into an encoder to generate potential features includes: Performing bilinear interpolation processing on the image of the vehicle to be assessed for damage to obtain a preprocessed image; The preprocessed image is input into an encoder to generate the latent features.

5. The method according to claim 4, characterized in that The step of performing bilinear interpolation processing on the image of the vehicle to be damaged to obtain a preprocessed image includes: Determine four corner points around each pixel in the image of the vehicle to be damaged; Determine the pixel value of each pixel point according to the weighted average value of the four corner points; The image of the vehicle to be damaged is converted into the pre-processed image according to the pixel value of each pixel point.

6. The method according to claim 1, characterized in that The method of matching the plurality of candidate region feature matrices with the annotated true values ​​in a one-to-many matching manner to obtain a matched target region feature matrix includes: Matching each of the candidate region feature matrices with the annotated true value one by one, to obtain an overlapping area ratio value corresponding to an overlapping area between each of the candidate region feature matrixes and the annotated true value; The candidate region feature matrix whose overlapping area ratio value is greater than a preset threshold is determined as the target region feature matrix.

7. The method according to any one of claims 1 to 6, characterized in that: The encoder is a Transformer encoder, the decoder is a Transformer decoder, the multi-scale adapter adopts an identity mapping structure, and the auxiliary detection head is an adaptive training sample selection ATSS auxiliary detection head.

8. A vehicle damage assessment device, characterized in that: The device comprises: An acquisition module is used to obtain images of vehicles to be assessed for damage; An encoding module, used for inputting the image of the vehicle to be damaged into an encoder to generate potential features; A decoding module, used for inputting the latent features into a decoder to obtain a target feature vector; A conversion module, for converting the latent features into multi-scale features through a multi-scale adapter; A generation module, used for inputting the multi-scale features into an auxiliary detection head to generate a plurality of candidate region feature matrices; A matching module, used to match the plurality of candidate region feature matrices with the annotated true values ​​in a one-to-many matching manner to obtain a matched target region feature matrix; The output module is used to obtain the vehicle damage assessment result corresponding to the image of the vehicle to be assessed based on the target area feature matrix and the target feature vector.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the vehicle damage assessment method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the vehicle damage assessment method described in any one of claims 1 to 7 is implemented.

Images