Vehicle loss assessment method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a vehicle loss assessment method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring a to-be-recognized image uploaded by a user; performing image recognition on the to-be-recognized image to obtain a recognition result; when it is determined that the identification result is a vehicle image to be subjected to damage assessment, information extraction is carried out on the vehicle image to be subjected to damage assessment through an image-text pre-training model, image features, component information and damage information of the vehicle image to be subjected to damage assessment are obtained, and the image-text pre-training model is obtained through training of a labeling sample set containing different text label sets; performing fine adjustment processing on the image features, the component information and the damage information through an adapter to obtain vehicle damage assessment information matched with the target service; and determining a vehicle loss assessment result corresponding to the target business according to the vehicle loss assessment information. The embodiment of the invention can be applied to a vehicle loss self-service claim settlement scene, and vehicle loss assessment can be carried out according to business requirements of different business parties.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a vehicle damage assessment method and device, electronic equipment and storage medium. Background Art

[0002] In the scenario of self-service claims for vehicle damage, users take and upload photos of vehicle damage, and the artificial intelligence system automatically identifies the parts involved in the photos, and detects whether the parts are damaged and the degree of damage, etc., which is used as the basis for insurance claims. The core modules of this artificial intelligence system usually include detection (or segmentation) models for automobile parts and detection (or segmentation) models for damage. The existing technology is to collect and annotate a large number of samples for each specific problem, and train their respective expert models (specialist models). For example, for automobile exterior parts, a detection (or segmentation) model for exterior parts is trained; for automobile interior parts, a detection (or segmentation) model for interior parts is trained; for other attribute information of the automobile, such as color, a classification model for corresponding attribute information is trained. However, different insurance claims business parties have different needs for the above information, and different business parties have the same needs for expert models. Therefore, how to assess vehicle damage according to the business needs of different business parties has become a technical problem that needs to be solved in the scenario of self-service claims for vehicle damage. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a vehicle damage assessment method and device, an electronic device and a storage medium, which use an expert model suitable for the specific needs of the business party to assess the vehicle damage, so that the vehicle damage can be assessed according to the business needs of different business parties.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application provides a vehicle damage assessment method, the method comprising:

[0005] Get the image to be identified uploaded by the user;

[0006] Performing image recognition on the image to be recognized by using a picture-text pre-training model to obtain a recognition result;

[0007] When it is determined that the recognition result is an image of a vehicle to be damaged, information is extracted from the image of the vehicle to be damaged by using the image-text pre-training model to obtain image features, component information, and damage information of the image of the vehicle to be damaged, wherein the image-text pre-training model is trained by a labeled sample set containing different text label sets;

[0008] Fine-tuning the image features, the component information, and the damage information through an adapter to obtain vehicle damage assessment information matching the target business, wherein the adapter is trained by sample data of the target business;

[0009] The vehicle damage assessment result corresponding to the target business is determined based on the vehicle damage assessment information.

[0010] In some embodiments, the labeled sample set includes a first labeled sample set for vehicle component segmentation and a second labeled sample set for vehicle damage segmentation, wherein different text labels corresponding to the same component in the first labeled sample set are associated with the same component image semantics, and different text labels corresponding to the same damage in the second labeled sample set are associated with the same damage image semantics.

[0011] In some embodiments, the training method of the image-text pre-training model includes:

[0012] Acquire a training sample set, wherein the training sample set includes an original vehicle image, a label mask, and a text label, wherein the text label includes at least one of component information and damage information;

[0013] Extracting features from the original vehicle image using an image encoding module to obtain image features;

[0014] Extracting features of the text label through a text encoding module to obtain text features;

[0015] Inputting the image features, the text features and N learnable query vectors into a segmentation mask decoding module to obtain N segmentation masks;

[0016] Calculating the loss between the N segmentation masks and the annotation mask to obtain a loss function;

[0017] The image-text pre-training model is trained based on the loss function to obtain the trained image-text pre-training model.

[0018] In some embodiments, obtaining a training sample set includes:

[0019] Acquire a first labeled sample set for vehicle component segmentation and a second labeled sample set for vehicle damage segmentation;

[0020] Associating different text labels corresponding to the same component in the first annotation sample set to the same component image semantics to obtain a first text label set;

[0021] Associating different text labels corresponding to the same damage in the second annotated sample set to the same damage image semantics to obtain a second text label set;

[0022] The training sample set is obtained based on the first text label set and the second text label set.

[0023] In some embodiments, the text label includes a target agreed word set by a user, the segmentation mask corresponding to the target agreed word is a set of target parts and target damage in the vehicle image, and the target agreed word appears in the training process of the image-text pre-training model according to a preset probability.

[0024] In some embodiments, the method further comprises:

[0025] Knowledge distillation is performed on the image-text pre-training model to obtain a lightweight target image-text pre-training model.

[0026] In some embodiments, the component information includes component name features, component segmentation mask, component detection box and component attributes; the damage information includes damage type features, damage segmentation mask, damage detection box and damage degree.

[0027] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a vehicle damage assessment device, the device comprising:

[0028] The acquisition module is used to acquire the image to be identified uploaded by the user;

[0029] A recognition module, used to perform image recognition on the image to be recognized through a pre-trained image-text model to obtain a recognition result;

[0030] an extraction module, for extracting information from the image of the vehicle to be damaged by the image-text pre-training model to obtain image features, component information, and damage information of the image of the vehicle to be damaged, when the recognition result is determined to be an image of a vehicle to be damaged. The image-text pre-training model is trained by a labeled sample set containing different text label sets;

[0031] A matching module, used for fine-tuning the image features, the component information and the damage information through an adapter to obtain vehicle damage assessment information matching the target business, wherein the adapter is trained by sample data of the target business;

[0032] A damage assessment module is used to determine the vehicle damage assessment result corresponding to the target business based on the vehicle damage assessment information.

[0033] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0034] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0035] The vehicle damage assessment method and device, electronic device and storage medium proposed in the present application obtain the image to be identified uploaded by the user; perform image recognition on the image to be identified through the image-text pre-training model to obtain the recognition result; when it is determined that the recognition result is the image of the vehicle to be identified, extract information from the image of the vehicle to be identified through the image-text pre-training model to obtain the image features, component information and damage information of the vehicle to be identified, wherein the image-text pre-training model is trained by a labeled sample set containing different text label sets; fine-tune the image features, component information and damage information through an adapter to obtain the vehicle damage assessment information matching the target business, wherein the adapter is trained by the sample data of the target business; determine the vehicle damage assessment result corresponding to the target business according to the vehicle damage assessment information. By inputting the image to be identified uploaded by the user into the image-text pre-training model, when it is determined that the image to be identified is the image of the vehicle to be identified, extract the image features, component information and damage information of the vehicle to be identified, and since the image-text pre-training model is trained by a labeled sample set containing different text label sets, the image-text pre-training model can correspond to the corresponding image semantics according to the text label during the training stage. And using the image and text pre-training model as the basic model, the image features, component information, and damage information output by the image and text pre-training model are fine-tuned through the adapter to obtain vehicle damage assessment information that matches the target business, and based on this, an expert model suitable for the business party is formed to output the vehicle damage assessment result corresponding to the target business, so that vehicle damage assessment can be performed according to the business needs of different business parties. Based on this, the embodiment of the present application can be applied to the scenario of self-service claims for vehicle damage, and the vehicle damage can be assessed by adopting an expert model suitable for the specific needs of the business party, so that the vehicle damage can be assessed according to the business needs of different business parties. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of a vehicle damage assessment method provided in an embodiment of the present application;

[0037] Figure 2 It is a schematic diagram of the input and output of the image and text pre-training model provided in the embodiment of the present application;

[0038] Figure 3 It is a schematic diagram of the structure of the expert model provided in the embodiment of the present application;

[0039] Figure 4 It is a flow chart of the training method of the image and text pre-training model;

[0040] Figure 5This is a diagram showing the principle of the training method for the image and text pre-training model;

[0041] Figure 6 yes Figure 4 Flow chart of step S401 in FIG.

[0042] Figure 7 is a schematic diagram of the structure of a vehicle damage assessment device provided in an embodiment of the present application;

[0043] Figure 8 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0047] First, some nouns involved in this application are analyzed:

[0048] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0049] Adapter-tuning: is a method for fine-tuning pre-trained models, which has some advantages and application scenarios compared to traditional fine-tuning methods. Adapter-tuning optimizes specific tasks by introducing adapter modules, which are a set of lightweight parameters added to the middle layer of the model to protect the parameters of the original pre-trained model. The goal of this method is to adapt to new tasks by adjusting the parameters of the adapter module without changing the overall model structure.

[0050] Knowledge distillation: It is a common method for model compression. It builds a lightweight small model and uses the supervision information of a large model with better performance to train the small model in order to achieve better performance and accuracy. In the process of knowledge distillation, the large model is called the teacher model, and the small model is called the student model. In this way, the knowledge in the teacher model is extracted and distilled and concentrated into the student model, thereby achieving model compression and optimization.

[0051] Segmentation mask: It is a technique in computer vision that is used to accurately separate objects in an image from the background. It achieves fine-grained division of image regions by classifying and labeling each pixel. Each pixel is assigned a label to indicate whether it belongs to the foreground or background, or to a different object category. Such label information forms a two-dimensional matrix, namely the segmentation mask. The generation of segmentation masks relies on the technical support of deep learning and artificial intelligence. Neural networks are trained with a large amount of image data so that they can learn and understand the characteristics of various objects. During the calculation process, the neural network gradually extracts the feature information of the image by performing operations such as convolution and pooling on the input image. Finally, the mask output by the network can accurately describe the position and boundaries of different objects in the image.

[0052] In the scenario of self-service claims for vehicle damage, users take and upload photos of vehicle damage, and the artificial intelligence system automatically identifies the parts involved in the photos, and detects whether the parts are damaged and the degree of damage, etc., which is used as the basis for insurance claims. The core modules of this artificial intelligence system usually include detection (or segmentation) models for automobile parts and detection (or segmentation) models for damage. The existing technology is to collect and annotate a large number of samples for each specific problem and train their respective expert models. For example, for automobile exterior parts, a detection (or segmentation) model for exterior parts is trained; for automobile interior parts, a detection (or segmentation) model for interior parts is trained; for other attribute information of the automobile, such as color, a classification model for corresponding attribute information is trained. However, different insurance claims business parties have different needs for the above information, and different business parties have different needs for expert models, and different business parties may use different text descriptions for the same part or damage. Therefore, how to assess vehicle damage according to the business needs of different business parties has become a technical problem that needs to be solved in the scenario of self-service claims for vehicle damage.

[0053] Based on this, the embodiment of the present application provides a vehicle damage assessment method and device, an electronic device and a storage medium, which obtains an image to be identified uploaded by a user; performs image recognition on the image to be identified through a pre-trained image-text model to obtain a recognition result; when the recognition result is determined to be a vehicle image to be identified, information is extracted from the vehicle image to be identified through the pre-trained image-text model to obtain image features, component information and damage information of the vehicle image to be identified, wherein the pre-trained image-text model is trained by a labeled sample set containing different text label sets; fine-tunes the image features, component information and damage information through an adapter to obtain vehicle damage assessment information matching the target business, wherein the adapter is trained by sample data of the target business; and determines the vehicle damage assessment result corresponding to the target business according to the vehicle damage assessment information. By inputting the image to be identified uploaded by the user into the pre-trained image-text model, when it is determined that the image to be identified is a vehicle image to be identified, the image features, component information and damage information of the vehicle image to be identified are extracted. Since the pre-trained image-text model is trained by a labeled sample set containing different text label sets, the pre-trained image-text model can correspond to the corresponding image semantics according to the text label during the training stage. And using the image and text pre-training model as the basic model, the image features, component information, and damage information output by the image and text pre-training model are fine-tuned through the adapter to obtain vehicle damage assessment information that matches the target business, and based on this, an expert model suitable for the business party is formed to output the vehicle damage assessment result corresponding to the target business, so that vehicle damage assessment can be performed according to the business needs of different business parties. Based on this, the embodiment of the present application can be applied to the scenario of self-service claims for vehicle damage, and the vehicle damage can be assessed by adopting an expert model suitable for the specific needs of the business party, so that the vehicle damage can be assessed according to the business needs of different business parties.

[0054] The vehicle damage assessment method and device, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the vehicle damage assessment method in the embodiments of the present application is described.

[0055] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0056] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0057] The vehicle damage assessment method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The vehicle damage assessment method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the vehicle damage assessment method, etc., but is not limited to the above forms.

[0058] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0059] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0060] Figure 1 is an optional flow chart of the vehicle damage assessment method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105.

[0061] Step S101, obtaining an image to be identified uploaded by a user;

[0062] Step S102, performing image recognition on the image to be recognized by using the image-text pre-training model to obtain a recognition result;

[0063] Step S103, when it is determined that the recognition result is an image of a vehicle to be damaged, information is extracted from the image of the vehicle to be damaged by using a pre-trained image-text model to obtain image features, component information, and damage information of the image of the vehicle to be damaged, wherein the pre-trained image-text model is trained by a labeled sample set containing different text label sets;

[0064] Step S104, fine-tuning the image features, component information and damage information through an adapter to obtain vehicle damage assessment information matching the target business, wherein the adapter is trained with sample data of the target business;

[0065] Step S105, determining the vehicle damage assessment result corresponding to the target business according to the vehicle damage assessment information.

[0066] In step S101 of some embodiments, an image to be identified uploaded by a user is obtained. Specifically, the image to be identified uploaded by the user can be obtained from a mobile terminal or a cloud server. The image to be identified can be obtained by photographing the damaged vehicle from different angles using a mobile device with a camera, and the damaged vehicle can include a plurality of different damaged vehicle parts. In the self-service vehicle damage claim scenario, the user can use the self-service vehicle damage claim business interface, and the customer can assess the vehicle damage by photographing the damaged vehicle parts and uploading them to the business system.

[0067] In step S102 of some embodiments, the image to be identified is recognized by the image-text pre-training model to obtain a recognition result. For the image to be identified uploaded by the user, the image to be identified must first be pre-identified by the image-text pre-training model, that is, it is determined whether the image to be identified is a vehicle-related image. If the recognition result is a non-vehicle-related image mistakenly transmitted by the user, further processing of the image is stopped, thereby intercepting the non-vehicle-related image and avoiding subsequent model computing power resource waste. Only when the recognition result is a vehicle-related image will the subsequent information extraction process be entered, thereby improving the model processing efficiency.

[0068] In step S103 of some embodiments, when it is determined that the recognition result is an image of a vehicle to be damaged, the image of the vehicle to be damaged is extracted through a pre-trained image-text model to obtain image features, component information and damage information of the image of the vehicle to be damaged, wherein the pre-trained image-text model is trained by a labeled sample set containing different text label sets. Only when it is recognized that the image to be recognized uploaded by the user is an image of a vehicle to be damaged, the pre-trained image-text model will extract information from the image of the vehicle to be damaged to obtain image features, component information and damage information of the image of the vehicle to be damaged. Among them, the component information may include but is not limited to component name features, component segmentation masks, component detection boxes and component attributes, and the damage information may include but is not limited to damage type features, damage segmentation masks, damage detection boxes and damage degrees. Since the pre-trained image-text model is trained by a labeled sample set containing different text label sets, the pre-trained image-text model can correspond to the corresponding image semantics according to the text labels during the training stage. Using the pre-trained image-text model as a general model, the component information and damage information output by it are also more comprehensive.

[0069] In some embodiments, Figure 2As shown in the figure, the image-text pre-training model is used as a general model to extract information of several fields from the input image, including whether it is a vehicle-related image, image features, component information, and damage information. Among them, the component information field includes component name features, component segmentation mask matrix, component detection box (bounding box) coordinates, and component attribute information (such as color); the damage information field includes damage type features (types such as scratches, dents, bends, cracks), damage area segmentation mask (mask) matrix, damage detection box (bounding box) coordinates, and damage degree (such as slight, moderate, and severe).

[0070] In step S104 of some embodiments, the image features, component information and damage information are fine-tuned by an adapter to obtain vehicle damage assessment information that matches the target business, wherein the adapter is trained with sample data of the target business. The image-text pre-training model is used as the basic model, and the image features, component information and damage information output by the image-text pre-training model are fine-tuned by the adapter to obtain vehicle damage assessment information that matches the target business. Based on a general image-text pre-training model, its output is connected to the adapter. Since the adapter is trained with sample data of the target business, the vehicle damage assessment information obtained can match the target business, thereby being suitable for the specific needs of the business party. It should be noted that in order to make the image-text pre-training model lightweight, the image-text pre-training model can also be subjected to knowledge distillation.

[0071] In some embodiments, Figure 3 As shown in the figure, the output result of the image and text pre-training model as a general model can be directly used in vehicle damage assessment business, or it can be used as a foundation model. By using the image features, component information and damage information extracted by the foundation model, an adapter can be added to train a more accurate expert model for specific business needs, or a combination of distillation and adapter can be used to train an expert model that has both accuracy and speed and is suitable for the specific needs of the business party.

[0072] In step S105 of some embodiments, the vehicle damage assessment result corresponding to the target business is determined based on the vehicle damage assessment information. An expert model is formed based on the pre-trained model of graphics and text plus an adapter, and an expert model suitable for the business party is formed to output the vehicle damage assessment result corresponding to the target business, so that vehicle damage assessment can be performed according to the business needs of different business parties.

[0073] Steps S101 to S105 shown in the embodiment of the present application are to obtain the image to be identified uploaded by the user; perform image recognition on the image to be identified through the image-text pre-training model to obtain the identification result; when it is determined that the identification result is the image of the vehicle to be damaged, extract information from the image to be damaged through the image-text pre-training model to obtain the image features, component information and damage information of the image to be damaged, wherein the image-text pre-training model is trained by a labeled sample set containing different text label sets; fine-tune the image features, component information and damage information through an adapter to obtain vehicle damage information matching the target business, wherein the adapter is trained by sample data of the target business; determine the vehicle damage result corresponding to the target business according to the vehicle damage information. By inputting the image to be identified uploaded by the user into the image-text pre-training model, when it is determined that the image to be identified is the image of the vehicle to be damaged, extract the image features, component information and damage information of the image to be damaged, because the image-text pre-training model is trained by a labeled sample set containing different text label sets, the image-text pre-training model can correspond to the corresponding image semantics according to the text label during the training stage. And using the image and text pre-training model as the basic model, the image features, component information, and damage information output by the image and text pre-training model are fine-tuned through the adapter to obtain vehicle damage assessment information that matches the target business, and based on this, an expert model suitable for the business party is formed to output the vehicle damage assessment result corresponding to the target business, so that vehicle damage assessment can be performed according to the business needs of different business parties. Based on this, the embodiment of the present application can be applied to the scenario of self-service claims for vehicle damage, and the vehicle damage can be assessed by adopting an expert model suitable for the specific needs of the business party, so that the vehicle damage can be assessed according to the business needs of different business parties.

[0074] In some embodiments, the definition of labels by the model developer may be different from that by the business party. The business party can provide the damage images and corresponding damage assessment results stored in the past to the model developer for iterative training of a more accurate model, but the business party may use different text descriptions for the same component or damage. For example, the water tank and the radiator are the same component, which is not conducive to the automatic organization of training samples. In this regard, the image-text pre-training model of the embodiment of the present application is trained by a labeled sample set containing different sets of text labels. For the labeled sample set, it may include a first labeled sample set for vehicle component segmentation and a second labeled sample set for vehicle damage segmentation, wherein different text labels corresponding to the same component in the first labeled sample set are associated with the same component image semantics, and different text labels corresponding to the same damage in the second labeled sample set are associated with the same damage image semantics. In the training stage of the image-text pre-training model, the first labeled sample set for vehicle component segmentation can be used to construct the component part in the training set. The second labeled sample set for vehicle damage segmentation can be used to construct the damage part in the training set. Based on this, the embodiment of the present application associates different text labels corresponding to the same component in the first annotation sample set to the same component image semantics, and associates different text labels corresponding to the same damage in the second annotation sample set to the same damage image semantics. Therefore, the text labels in the embodiment of the present application can overcome the differences in language expression. For example, "water tank" and "radiator" are the same component, and they will correspond to the same image semantics during the training process, which is conducive to the automatic organization of training samples and is more convenient to use.

[0075] See also Figure 4 In some embodiments, the training method of the image-text pre-training model may include but is not limited to steps S401 to S406:

[0076] Step S401, obtaining a training sample set, the training sample set including an original vehicle image, a label mask and a text label, the text label including at least one of component information and damage information;

[0077] Step S402, extracting features from the original vehicle image through an image encoding module to obtain image features;

[0078] Step S403, extracting features from the text label through a text encoding module to obtain text features;

[0079] Step S404, inputting the image features, text features and N learnable query vectors into a segmentation mask decoding module to obtain N segmentation masks;

[0080] Step S405, calculating the loss between the N segmentation masks and the annotation masks to obtain a loss function;

[0081] Step S406: training the image-text pre-training model based on the loss function to obtain a trained image-text pre-training model.

[0082] In some embodiments, Figure 5 As shown in the figure, in the training stage of the image-text pre-training model, each training sample includes the original image, segmentation mask, and text label, where the text label can be component information or damage information. In the training stage, the image encoding module is used to extract features of the original vehicle image to obtain image features. The text encoding module is used to extract features of the text label to obtain text features. The segmentation mask decoding module accepts three inputs, namely image features, text features, and N learnable query vectors. The segmentation mask decoding module outputs N segmentation masks, calculates the loss with the annotation mask, and performs gradient backpropagation to realize the training process. Among them, the image encoding module uses ViT (Vision Transformer), and the text encoding module uses CLIP (Contrastive Language-Image Pre-training) that supports Chinese. The principle of N learnable query vectors comes from DETR (Detection Transformer), and the value of N can be determined based on ablation experiments.

[0083] See also Figure 6 In some embodiments, step S401 may include but is not limited to steps S601 to S604:

[0084] Step S601, obtaining a first labeled sample set for vehicle component segmentation and a second labeled sample set for vehicle damage segmentation;

[0085] Step S602, associating different text labels corresponding to the same component in the first annotation sample set to the same component image semantics to obtain a first text label set;

[0086] Step S603, associating different text labels corresponding to the same damage in the second annotated sample set with the same damage image semantics to obtain a second text label set;

[0087] Step S604: obtaining a training sample set based on the first text label set and the second text label set.

[0088] In the training stage of the image and text pre-training model, the first labeled sample set of vehicle component segmentation can be used to construct the component part in the training set. The second labeled sample set of vehicle damage segmentation can be used to construct the damage part in the training set. In order to avoid different business parties using different text descriptions for the same component or damage, which in turn affects the organization of the training samples, the embodiment of the present application associates different text labels corresponding to the same component in the first labeled sample set to the same component image semantics to obtain a first text label set. And associate different text labels corresponding to the same damage in the second labeled sample set to the same damage image semantics to obtain a second text label set. The training sample set is obtained based on the first text label set and the second text label set. Therefore, the text labels of the embodiment of the present application can overcome the differences in language expression. For example, "water tank" and "radiator" are the same component, and they will correspond to the same image semantics during the training process, which is conducive to the automatic organization of training samples, more convenient to use, and better training effect.

[0089] In some embodiments, a labeled sample set of vehicle component segmentation is used to construct the component part in the training set. Assuming that an image contains 4 components, the sample set of component segmentation has been labeled with a label number corresponding to the semantics of "left front door", and a set of text labels "left front door", "left door", "front door", "front door", "door", "left door" is assigned to this segmentation mask. In this way, 4 groups of text labels for this image are obtained. Randomly sample any text label in any group of text labels, for example, sample "front door", and then find all groups containing "front door" in the 4 groups of text labels, and merge the segmentation masks of these groups as the segmentation mask of the text label "front door". For example, it is possible to merge "right front door" and "left front door" together. At this time, 1 training sample about the component is obtained.

[0090] In some embodiments, the second labeled sample set of vehicle damage segmentation is used to construct the damage part in the training set. This is similar to using the labeled sample set of vehicle component segmentation to construct the component part in the training set, except that the category is changed to "scratches", "dents", etc., and a label indicating the degree of damage such as "minor", "moderate", etc. is added.

[0091] In some embodiments, the text label includes a target agreed word set by the user, and the segmentation mask corresponding to the target agreed word is a set of target parts and target damage in the vehicle image. The target agreed word appears in the training process of the image-text pre-training model according to a preset probability. That is, the text label can also be a predetermined word. For example, a predetermined word "all parts and damages" can be designed, and its corresponding segmentation mask is the sum of all parts and all damages in the image, so that the model can output the summary results of all parts and all damages of the vehicle, thereby obtaining complete information about parts and damages of the image of the vehicle to be damaged.

[0092] In some embodiments, during the prediction stage, text labels can output initial results as a basis for annotation and provide manual annotation and correction, thereby reducing the labor cost of annotation.

[0093] In some embodiments, during the prediction stage, a specific component or damage name is used as a text label to obtain a segmentation result of the component or damage in the image of the vehicle to be damaged, thereby assisting in labeling the specific component or damage, thereby reducing the labor cost of labeling.

[0094] In some embodiments, the image-text pre-training model can be used as a general visual language model for vehicle damage assessment scenarios, and can be used as a business scenario. The image-text pre-training model can simultaneously output information such as component name, component attributes, component location, damage name, damage degree, and damage degree.

[0095] In some embodiments, the image and text pre-training model can also be used as a basic model, and after adapters and / or knowledge distillation, an expert model suitable for specific needs is obtained.

[0096] See also Figure 7 The embodiment of the present application also provides a vehicle damage assessment device, which can implement the above-mentioned vehicle damage assessment method, and the device includes:

[0097] The acquisition module 710 is used to acquire the image to be identified uploaded by the user;

[0098] The recognition module 720 is used to perform image recognition on the image to be recognized through the image-text pre-training model to obtain a recognition result;

[0099] The extraction module 730 is used to extract information from the image of the vehicle to be damaged by the image-text pre-training model to obtain image features, component information and damage information of the image of the vehicle to be damaged, when the recognition result is determined to be the image of the vehicle to be damaged. The image-text pre-training model is trained by a labeled sample set containing different text label sets;

[0100] A matching module 740 is used to fine-tune the image features, component information and damage information through an adapter to obtain vehicle damage assessment information that matches the target business, wherein the adapter is trained by sample data of the target business;

[0101] The damage assessment module 750 is used to determine the vehicle damage assessment result corresponding to the target business based on the vehicle damage assessment information.

[0102] In some embodiments of the present application, the acquisition module 710 acquires an image to be identified uploaded by a user; the identification module 720 performs image recognition on the image to be identified through a pre-trained image-text model to obtain a recognition result; the extraction module 730, when determining that the recognition result is an image of a vehicle to be damaged, extracts information from the image to be damaged through a pre-trained image-text model to obtain image features, component information and damage information of the image to be damaged, wherein the pre-trained image-text model is trained by a labeled sample set containing different sets of text labels; the matching module 740 fine-tunes the image features, component information and damage information through an adapter to obtain vehicle damage information matching the target business, wherein the adapter is trained by sample data of the target business; the damage assessment module 750 determines the vehicle damage assessment result corresponding to the target business based on the vehicle damage assessment information.

[0103] In some embodiments of the present application, an image to be identified uploaded by a user is obtained. Specifically, the image to be identified uploaded by the user can be obtained from a mobile terminal or a cloud server. The image to be identified can be obtained by photographing the vehicle to be assessed from different angles using a mobile device with a camera. The vehicle to be assessed may include a number of different damaged vehicle parts. In the self-service vehicle damage claim scenario, users can use the self-service vehicle damage claim business interface, where customers can assess vehicle damage by photographing damaged vehicle parts and uploading them to the business system.

[0104] In some embodiments of the present application, image recognition is performed on the image to be recognized through the image-text pre-training model to obtain a recognition result. For the image to be recognized uploaded by the user, the image to be recognized must first be pre-recognized through the image-text pre-training model, that is, it is determined whether the image to be recognized is a vehicle-related image. If the recognition result is a non-vehicle-related image mistakenly transmitted by the user, further processing of the image is stopped, thereby intercepting the non-vehicle-related image and avoiding subsequent model computing power resource waste. Only when the recognition result is a vehicle-related image will the subsequent information extraction process be entered, thereby improving the model processing efficiency.

[0105] In some embodiments of the present application, when it is determined that the recognition result is an image of a vehicle to be damaged, the image of the vehicle to be damaged is extracted through a pre-trained image-text model to obtain image features, component information and damage information of the image of the vehicle to be damaged, wherein the pre-trained image-text model is trained by a labeled sample set containing different text label sets. Only when it is recognized that the image to be recognized uploaded by the user is an image of a vehicle to be damaged, the pre-trained image-text model will extract information from the image of the vehicle to be damaged to obtain image features, component information and damage information of the vehicle to be damaged. Among them, the component information may include but is not limited to component name features, component segmentation masks, component detection boxes and component attributes, and the damage information may include but is not limited to damage type features, damage segmentation masks, damage detection boxes and damage degrees. Since the pre-trained image-text model is trained by a labeled sample set containing different text label sets, the pre-trained image-text model can correspond to the corresponding image semantics according to the text labels during the training stage. Using the pre-trained image-text model as a general model, the component information and damage information output by it are also more comprehensive.

[0106] In some embodiments of the present application, the image features, component information and damage information are fine-tuned by an adapter to obtain vehicle damage assessment information that matches the target business, wherein the adapter is trained with sample data of the target business. The image-text pre-training model is used as the basic model, and the image features, component information and damage information output by the image-text pre-training model are fine-tuned by the adapter to obtain vehicle damage assessment information that matches the target business. Based on a general image-text pre-training model, its output is connected to the adapter. Since the adapter is trained with sample data of the target business, the vehicle damage assessment information obtained can match the target business, thereby being suitable for the specific needs of the business party. It should be noted that in order to make the image-text pre-training model lightweight, the image-text pre-training model can also be subjected to knowledge distillation.

[0107] In some embodiments of the present application, the vehicle damage assessment result corresponding to the target business is determined based on the vehicle damage assessment information. An expert model is formed based on the pre-trained model of graphics and text plus an adapter, and an expert model suitable for the business party is formed to output the vehicle damage assessment result corresponding to the target business, so that vehicle damage assessment can be performed according to the business needs of different business parties.

[0108] Based on this, in the vehicle damage assessment device of the embodiment of the present application, the acquisition module 710 acquires the image to be identified uploaded by the user; the recognition module 720 performs image recognition on the image to be identified through the image-text pre-training model to obtain a recognition result; the extraction module 730, when determining that the recognition result is the image of the vehicle to be damaged, extracts information from the image to be damaged through the image-text pre-training model to obtain the image features, component information and damage information of the image to be damaged, wherein the image-text pre-training model is trained by a labeled sample set containing different text label sets; the matching module 740 fine-tunes the image features, component information and damage information through an adapter to obtain vehicle damage assessment information matching the target business, wherein the adapter is trained by sample data of the target business; the damage assessment module 750 determines the vehicle damage assessment result corresponding to the target business based on the vehicle damage assessment information. The present application obtains the image to be identified uploaded by the user; performs image recognition on the image to be identified through the image-text pre-training model to obtain the identification result; when it is determined that the identification result is the image of the vehicle to be damaged, the image-text pre-training model is used to extract information from the image of the vehicle to be damaged to obtain the image features, component information and damage information of the image of the vehicle to be damaged, wherein the image-text pre-training model is trained by a labeled sample set containing different text label sets; fine-tunes the image features, component information and damage information through an adapter to obtain vehicle damage information matching the target business, wherein the adapter is trained by sample data of the target business; and determines the vehicle damage result corresponding to the target business according to the vehicle damage information. By inputting the image to be identified uploaded by the user into the image-text pre-training model, when it is determined that the image to be identified is the image of the vehicle to be damaged, the image features, component information and damage information of the image of the vehicle to be damaged are extracted. Since the image-text pre-training model is trained by a labeled sample set containing different text label sets, the image-text pre-training model can correspond to the corresponding image semantics according to the text labels during the training stage. And using the image and text pre-training model as the basic model, the image features, component information, and damage information output by the image and text pre-training model are fine-tuned through the adapter to obtain vehicle damage assessment information that matches the target business, and based on this, an expert model suitable for the business party is formed to output the vehicle damage assessment result corresponding to the target business, so that vehicle damage assessment can be performed according to the business needs of different business parties. Based on this, the embodiment of the present application can be applied to the scenario of self-service claims for vehicle damage, and the vehicle damage can be assessed by adopting an expert model suitable for the specific needs of the business party, so that the vehicle damage can be assessed according to the business needs of different business parties.

[0109] The specific implementation of the vehicle damage assessment device is basically the same as the specific implementation of the above-mentioned vehicle damage assessment method, and will not be repeated here.

[0110] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned vehicle damage assessment method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0111] See also Figure 8 , Figure 8 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0112] The processor 801 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided in the embodiment of the present application.

[0113] The memory 802 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 802, and the processor 801 calls and executes the vehicle damage assessment method of the embodiment of this application, that is, by obtaining the image to be identified uploaded by the user; performing image recognition on the image to be identified through the image-text pre-training model to obtain a recognition result; when it is determined that the recognition result is a vehicle image to be damaged, extracting information from the vehicle image to be damaged through the image-text pre-training model to obtain image features, component information, and damage information of the vehicle image to be damaged, wherein the image-text pre-training model is trained by a labeled sample set containing different text label sets; fine-tuning the image features, component information, and damage information through an adapter to obtain vehicle damage assessment information matching the target business, wherein the adapter is trained by sample data of the target business; and determining the vehicle damage assessment result corresponding to the target business according to the vehicle damage assessment information. By inputting the image to be identified uploaded by the user into the image-text pre-training model, when it is determined that the image to be identified is an image of a vehicle to be damaged, the image features, component information and damage information of the image of the vehicle to be damaged are extracted. Since the image-text pre-training model is trained by a labeled sample set containing different sets of text labels, the image-text pre-training model can correspond to the corresponding image semantics according to the text labels during the training stage. And using the image-text pre-training model as the basic model, the image features, component information and damage information output by the image-text pre-training model are fine-tuned through an adapter to obtain vehicle damage information matching the target business, and an expert model suitable for the business party is formed accordingly to output the vehicle damage result corresponding to the target business, so that the vehicle damage can be assessed according to the business needs of different business parties. Based on this, the embodiment of the present application can be applied to the scenario of self-service claims for vehicle damage, and the vehicle damage can be assessed according to the business needs of different business parties by adopting an expert model suitable for the specific needs of the business party.

[0114] The input / output interface 803 is used to implement information input and output.

[0115] The communication interface 804 is used to realize the communication interaction between this device and other devices. Communication can be realized through wired methods (such as USB, network cable, etc.) or wireless methods (such as mobile network, WIFI, Bluetooth, etc.).

[0116] The bus transmits information between various components of the device (eg, the processor 801 , the memory 802 , the input / output interface 803 , and the communication interface 804 ).

[0117] The processor 801 , the memory 802 , the input / output interface 803 and the communication interface 804 are connected to each other in communication within the device via a bus.

[0118] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned vehicle damage assessment method is implemented.

[0119] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0120] The vehicle damage assessment method, vehicle damage assessment device, electronic device and storage medium provided by the embodiment of the present application obtain the image to be identified uploaded by the user; perform image recognition on the image to be identified through the image-text pre-training model to obtain the recognition result; when it is determined that the recognition result is the image of the vehicle to be identified, extract information from the image of the vehicle to be identified through the image-text pre-training model to obtain the image features, component information and damage information of the image of the vehicle to be identified, wherein the image-text pre-training model is trained by a labeled sample set containing different text label sets; fine-tune the image features, component information and damage information through an adapter to obtain the vehicle damage assessment information matching the target business, wherein the adapter is trained by the sample data of the target business; determine the vehicle damage assessment result corresponding to the target business according to the vehicle damage assessment information. By inputting the image to be identified uploaded by the user into the image-text pre-training model, when it is determined that the image to be identified is the image of the vehicle to be identified, extract the image features, component information and damage information of the image of the vehicle to be identified, because the image-text pre-training model is trained by a labeled sample set containing different text label sets, the image-text pre-training model can correspond to the corresponding image semantics according to the text label during the training stage. And using the image and text pre-training model as the basic model, the image features, component information, and damage information output by the image and text pre-training model are fine-tuned through the adapter to obtain vehicle damage assessment information that matches the target business, and based on this, an expert model suitable for the business party is formed to output the vehicle damage assessment result corresponding to the target business, so that vehicle damage assessment can be performed according to the business needs of different business parties. Based on this, the embodiment of the present application can be applied to the scenario of self-service claims for vehicle damage, and the vehicle damage can be assessed by adopting an expert model suitable for the specific needs of the business party, so that the vehicle damage can be assessed according to the business needs of different business parties.

[0121] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as a computer-readable program, a data structure, a program module or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, disk storage or other magnetic storage device, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media generally contain computer-readable programs, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0122] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0123] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0124] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0125] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0126] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0127] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0128] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0129] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0131] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0132] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A vehicle damage assessment method, characterized in that: The method comprises: Get the image to be identified uploaded by the user; Performing image recognition on the image to be recognized by using a picture-text pre-training model to obtain a recognition result; When it is determined that the recognition result is an image of a vehicle to be damaged, information is extracted from the image of the vehicle to be damaged by using the image-text pre-training model to obtain image features, component information, and damage information of the image of the vehicle to be damaged, wherein the image-text pre-training model is trained by a labeled sample set containing different text label sets; Fine-tuning the image features, the component information, and the damage information through an adapter to obtain vehicle damage assessment information matching the target business, wherein the adapter is trained by sample data of the target business; The vehicle damage assessment result corresponding to the target business is determined based on the vehicle damage assessment information.

2. The method according to claim 1, characterized in that The labeled sample set includes a first labeled sample set for vehicle component segmentation and a second labeled sample set for vehicle damage segmentation, wherein different text labels corresponding to the same component in the first labeled sample set are associated with the same component image semantics, and different text labels corresponding to the same damage in the second labeled sample set are associated with the same damage image semantics.

3. The method according to claim 1, characterized in that The training method of the image-text pre-training model includes: Acquire a training sample set, wherein the training sample set includes an original vehicle image, a label mask, and a text label, wherein the text label includes at least one of component information and damage information; Extracting features from the original vehicle image using an image encoding module to obtain image features; Extracting features of the text label through a text encoding module to obtain text features; Inputting the image features, the text features and N learnable query vectors into a segmentation mask decoding module to obtain N segmentation masks; Calculating the loss between the N segmentation masks and the annotation mask to obtain a loss function; The image-text pre-training model is trained based on the loss function to obtain the trained image-text pre-training model.

4. The method according to claim 3, characterized in that The step of obtaining a training sample set includes: Acquire a first labeled sample set for vehicle component segmentation and a second labeled sample set for vehicle damage segmentation; Associating different text labels corresponding to the same component in the first annotation sample set to the same component image semantics to obtain a first text label set; Associating different text labels corresponding to the same damage in the second annotated sample set to the same damage image semantics to obtain a second text label set; The training sample set is obtained based on the first text label set and the second text label set.

5. The method according to claim 3, characterized in that: The text label includes a target agreed word set by a user, the segmentation mask corresponding to the target agreed word is a set of target parts and target damage in the vehicle image, and the target agreed word appears in the training process of the image-text pre-training model according to a preset probability.

6. The method according to claim 1, characterized in that The method further comprises: Knowledge distillation is performed on the image-text pre-training model to obtain a lightweight target image-text pre-training model.

7. The method according to any one of claims 1 to 6, characterized in that: The component information includes component name features, component segmentation mask, component detection box and component attributes; the damage information includes damage type features, damage segmentation mask, damage detection box and damage degree.

8. A vehicle damage assessment device, characterized in that: The device comprises: The acquisition module is used to acquire the image to be identified uploaded by the user; A recognition module, used to perform image recognition on the image to be recognized through a pre-trained image-text model to obtain a recognition result; an extraction module, for extracting information from the image of the vehicle to be damaged by the image-text pre-training model to obtain image features, component information, and damage information of the image of the vehicle to be damaged, when the recognition result is determined to be an image of a vehicle to be damaged. The image-text pre-training model is trained by a labeled sample set containing different text label sets; A matching module, used for fine-tuning the image features, the component information and the damage information through an adapter to obtain vehicle damage assessment information matching the target business, wherein the adapter is trained by sample data of the target business; A damage assessment module is used to determine the vehicle damage assessment result corresponding to the target business based on the vehicle damage assessment information.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the vehicle damage assessment method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the vehicle damage assessment method described in any one of claims 1 to 7 is implemented.

Images