Multi-modal large model vehicle loss assessment method and system based on cue word guidance

Abstract

The invention provides a multimodal large model vehicle loss assessment method and system based on cue word guidance, and the method comprises the following steps: carrying out the feature extraction of a vehicle damage panorama, and obtaining the encoded image features; segmenting the damaged part of the vehicle based on the encoded image features and cue words to obtain a segmented map of the damaged part of the vehicle; based on the cue word and the segmentation map of the damaged part of the vehicle, performing information interaction of multi-modal image-text pairs by using a multi-modal large model to obtain large model visual representation; and based on the large model visual representation, using a large language model to carry out vehicle loss assessment to obtain a damaged vehicle loss assessment result. According to the technical scheme, the two modes of language and picture are fused, and the vehicle loss assessment effect is more remarkable based on the multi-mode large model and the language large model.

Description

Background Art

[0002] Existing technologies only process images in a single mode, for example, using CNN or Transformer architectures to identify the damage of damaged vehicle images. For example, the patent "An intelligent vehicle damage detection method and system based on multi-image fusion" extracts frames from the vehicle damage information video, combines multiple frames and inputs them into the vehicle component segmentation and damage detection model based on the Transformer network structure, and finally makes a decision fusion to obtain the damage results. The disadvantage of this method is that it lacks the interaction of multiple modes, and the damage recognition effect is average. Summary of the invention

[0003] The present application provides a multi-modal large-model vehicle damage assessment method and system based on prompt word guidance, so as to improve the damage assessment effect.

[0004] In a first aspect, a multi-modal large model vehicle damage assessment method based on prompt word guidance is provided, comprising the following steps:

[0005] Extract features from the vehicle damage panorama to obtain encoded image features;

[0006] Based on the encoded image features and prompt words, the damaged part of the vehicle is segmented to obtain a segmentation map of the damaged part of the vehicle;

[0007] Based on the prompt words and the segmentation map of the damaged parts of the vehicle, a multimodal large model is used to perform multimodal image-text pair information interaction to obtain a large model visual representation;

[0008] Based on the large model visual representation, a large language model is used to perform vehicle damage assessment to obtain a damaged vehicle damage assessment result.

[0009] In the above technical scheme, feature extraction is performed on the panoramic image of vehicle damage to obtain encoded image features; based on the encoded image features and prompt words, the damaged parts of the vehicle are segmented to obtain a segmentation map of the damaged parts of the vehicle; based on the prompt words and the segmentation map of the damaged parts of the vehicle, a multimodal large model is used to perform multimodal information interaction of image and text pairs to obtain a large model visual representation; based on the large model visual representation, a large language model is used to perform vehicle damage assessment to obtain a damaged vehicle damage assessment result; the two modalities of language and picture are integrated, and based on the multimodal large model and the language large model, the effect of vehicle damage assessment is more significant.

[0010] In a specific implementation scheme, the step of extracting features from the vehicle damage panorama to obtain encoded image features is specifically as follows:

[0011] The ViT model is used to extract features from the vehicle damage panorama to obtain the encoded image features.

[0012] In a specific implementation scheme, the step of extracting features from the vehicle damage panorama using the ViT model to obtain the encoded image features specifically includes:

[0013] Performing a block operation on the vehicle damage panorama, and mapping the blocks into vectors using a linear mapping layer to obtain block vectors;

[0014] After adding the splicing position coding and the classification coding to the block vector, the vector is input into the Transformer Encoder structure to obtain the encoded image features.

[0015] In a specific implementation scheme, based on the encoded image features and prompt words, the steps of segmenting the damaged parts of the vehicle to obtain a segmentation map of the damaged parts of the vehicle are specifically as follows:

[0016] The SAM model is used to segment the damaged part of the vehicle to obtain a segmentation map of the damaged part of the vehicle.

[0017] In a specific implementation scheme, the step of using the SAM model to segment the damaged part of the vehicle to obtain a segmentation map of the damaged part of the vehicle specifically includes:

[0018] The damage to the vehicle is used as a prompt word, input into the prompt encoder, and the prompt word is mapped to the prompt feature space;

[0019] The encoded image features and the prompt words are fused and input into a mask decoder structure to decode the segmented damaged vehicle part image to obtain a segmentation map of the damaged vehicle part.

[0020] In a specific implementation scheme, based on the prompt word and the segmentation map of the damaged part of the vehicle, a multimodal large model is used to perform multimodal image-text pair information interaction to obtain a large model visual representation step, specifically:

[0021] Based on the prompt words and the segmentation map of the damaged parts of the vehicle, a BLIP-2 model is used to perform multimodal information interaction of image and text pairs to obtain the large model visual representation.

[0022] In a specific implementation scheme, based on the prompt word and the segmentation map of the damaged part of the vehicle, the BLIP-2 model is used to perform multimodal information interaction of the image and text pairs to obtain the step of visual representation of the large model, specifically including:

[0023] Mapping the segmentation map of the damaged part of the vehicle to extract a feature map of the corresponding ViT high-level feature;

[0024] The feature graphs of the prompt words and the ViT high-level features are input into the Q-Former module of the BLIP-2 model to perform multimodal information interaction of image-text pairs, and are fused with the query vector to obtain the visual representation of the large model.

[0025] In a specific implementation scheme, based on the large model visual representation, the large language model is used to perform vehicle damage assessment to obtain the damaged vehicle damage assessment result, specifically including:

[0026] Write instruction prompts;

[0027] The instruction prompt words and the large model visual representation are input into the large language model to obtain the damage assessment result of the damaged vehicle.

[0028] In a specific implementation scheme, the large language model adopts the Qwen-1.5-72b network structure.

[0029] In a second aspect, a multi-modal large-model vehicle damage assessment system based on prompt word guidance is provided, comprising:

[0030] A damaged vehicle panoramic image feature extraction module is used to extract features from the vehicle damage panoramic image to obtain encoded image features;

[0031] A vehicle damaged part segmentation module is used to segment the vehicle damaged part based on the encoded image features and prompt words to obtain a segmentation map of the vehicle damaged part;

[0032] A multimodal alignment module, for performing information interaction of multimodal image-text pairs using a multimodal large model based on the prompt word and the segmentation map of the damaged part of the vehicle to obtain a large model visual representation;

[0033] The large language model vehicle damage assessment module is used to perform vehicle damage assessment based on the large model visual representation and utilize the large language model to obtain a damaged vehicle damage assessment result.

[0034] In the above technical scheme, feature extraction is performed on the panoramic image of vehicle damage to obtain encoded image features; based on the encoded image features and prompt words, the damaged parts of the vehicle are segmented to obtain a segmentation map of the damaged parts of the vehicle; based on the prompt words and the segmentation map of the damaged parts of the vehicle, a multimodal large model is used to perform multimodal information interaction of image and text pairs to obtain a large model visual representation; based on the large model visual representation, a large language model is used to perform vehicle damage assessment to obtain a damaged vehicle damage assessment result; the two modalities of language and picture are integrated, and based on the multimodal large model and the language large model, the effect of vehicle damage assessment is more significant. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A flowchart of a multi-modal large-model vehicle damage assessment method based on prompt word guidance provided in an embodiment of the present application;

[0036] Figure 2 A structural block diagram of a multi-modal large-model vehicle damage assessment system based on prompt word guidance provided in an embodiment of the present application;

[0037] Figure 3 A specific flow chart of a multi-modal large-model vehicle damage assessment method based on prompt word guidance provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The present application is further described in detail below through the accompanying drawings and embodiments. Through these descriptions, the characteristics and advantages of the present application will become clearer and more specific.

[0039] The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise noted.

[0040] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0041] To facilitate understanding of the multimodal large-model vehicle damage assessment method and system based on prompt word guidance provided in the embodiment of the present application, its application scenario is first explained. The multimodal large-model vehicle damage assessment method and system based on prompt word guidance provided in the embodiment of the present application are used to improve the damage assessment effect. The existing technology only processes on a single mode such as pictures, for example, using CNN or Transformer architectures to identify the damage of damaged vehicle pictures. For example, the patent "A method and system for intelligent vehicle damage detection based on multi-image fusion" extracts frames from the vehicle damage information video, and combines multiple frames into a model for vehicle component segmentation and damage detection based on the Transformer network structure, and finally makes a decision fusion to obtain the damage injury result. The shortcoming of this method is the lack of interaction between multiple modes, and the damage identification effect is general. For this reason, the embodiment of the present application provides a multimodal large-model vehicle damage assessment method and system based on prompt word guidance to improve the damage assessment effect. The following is a detailed description of the embodiment in conjunction with specific drawings.

[0042] refer to Figures 1 to 3 , Figure 1 A flowchart of a multi-modal large-model vehicle damage assessment method based on prompt word guidance provided in an embodiment of the present application; Figure 2 A structural block diagram of a multi-modal large-model vehicle damage assessment system based on prompt word guidance provided in an embodiment of the present application; Figure 3 A specific flow chart of a multi-modal large-model vehicle damage assessment method based on prompt word guidance provided in an embodiment of the present application.

[0043] exist Figure 1 In the embodiment of the present application, a multi-modal large model vehicle damage assessment method based on prompt word guidance is provided, comprising the following steps:

[0044] Extract features from the vehicle damage panorama to obtain encoded image features;

[0045] Based on the encoded image features and prompt words, the damaged part of the vehicle is segmented to obtain a segmentation map of the damaged part of the vehicle;

[0046] Based on the prompt words and the segmentation map of the damaged parts of the vehicle, a multimodal large model is used to perform multimodal image-text pair information interaction to obtain a large model visual representation;

[0047] Based on the large model visual representation, a large language model is used to perform vehicle damage assessment to obtain a damaged vehicle damage assessment result.

[0048] In the above technical scheme, feature extraction is performed on the panoramic image of vehicle damage to obtain encoded image features; based on the encoded image features and prompt words, the damaged parts of the vehicle are segmented to obtain a segmentation map of the damaged parts of the vehicle; based on the prompt words and the segmentation map of the damaged parts of the vehicle, a multimodal large model is used to perform multimodal information interaction of image and text pairs to obtain a large model visual representation; based on the large model visual representation, a large language model is used to perform vehicle damage assessment to obtain a damaged vehicle damage assessment result; the two modalities of language and picture are integrated, and based on the multimodal large model and the language large model, the effect of vehicle damage assessment is more significant.

[0049] In a specific implementation scheme, the step of extracting features from the vehicle damage panorama to obtain encoded image features is specifically as follows:

[0050] The ViT model is used to extract features from the vehicle damage panorama to obtain the encoded image features.

[0051] In a specific implementation scheme, the step of extracting features from the vehicle damage panorama using the ViT model to obtain the encoded image features specifically includes:

[0052] Performing a block operation on the vehicle damage panorama, and mapping the blocks into vectors using a linear mapping layer to obtain block vectors;

[0053] After adding the splicing position coding and the classification coding to the block vector, the vector is input into the Transformer Encoder structure to obtain the encoded image features.

[0054] In a specific implementation scheme, based on the encoded image features and prompt words, the steps of segmenting the damaged parts of the vehicle to obtain a segmentation map of the damaged parts of the vehicle are specifically as follows:

[0055] The SAM model is used to segment the damaged part of the vehicle to obtain a segmentation map of the damaged part of the vehicle.

[0056] In a specific implementation scheme, the step of using the SAM model to segment the damaged part of the vehicle to obtain a segmentation map of the damaged part of the vehicle specifically includes:

[0057] The damage to the vehicle is used as a prompt word, input into the prompt encoder, and the prompt word is mapped to the prompt feature space;

[0058] The encoded image features and the prompt words are fused and input into a mask decoder structure to decode the segmented damaged vehicle part image to obtain a segmentation map of the damaged vehicle part.

[0059] In a specific implementation scheme, based on the prompt word and the segmentation map of the damaged part of the vehicle, a multimodal large model is used to perform multimodal image-text pair information interaction to obtain a large model visual representation step, specifically:

[0060] Based on the prompt words and the segmentation map of the damaged parts of the vehicle, a BLIP-2 model is used to perform multimodal information interaction of image and text pairs to obtain the large model visual representation.

[0061] In a specific implementation scheme, based on the prompt word and the segmentation map of the damaged part of the vehicle, the BLIP-2 model is used to perform multimodal information interaction of the image and text pairs to obtain the step of visual representation of the large model, specifically including:

[0062] Mapping the segmentation map of the damaged part of the vehicle to extract a feature map of the corresponding ViT high-level feature;

[0063] The feature graphs of the prompt words and the ViT high-level features are input into the Q-Former module of the BLIP-2 model to perform multimodal information interaction of image-text pairs, and are fused with the query vector to obtain the visual representation of the large model.

[0064] In a specific implementation scheme, based on the large model visual representation, the large language model is used to perform vehicle damage assessment to obtain the damaged vehicle damage assessment result, specifically including:

[0065] Write instruction prompts;

[0066] The instruction prompt words and the large model visual representation are input into the large language model to obtain the damage assessment result of the damaged vehicle.

[0067] In a specific implementation scheme, the large language model adopts the Qwen-1.5-72b network structure.

[0068] exist Figure 2 In the embodiment of the present application, a multi-modal large model vehicle damage assessment system based on prompt word guidance includes:

[0069] A damaged vehicle panoramic image feature extraction module is used to extract features from the vehicle damage panoramic image to obtain encoded image features;

[0070] A vehicle damaged part segmentation module is used to segment the vehicle damaged part based on the encoded image features and prompt words to obtain a segmentation map of the vehicle damaged part;

[0071] A multimodal alignment module, for performing information interaction of multimodal image-text pairs using a multimodal large model based on the prompt word and the segmentation map of the damaged part of the vehicle to obtain a large model visual representation;

[0072] The large language model vehicle damage assessment module is used to perform vehicle damage assessment based on the large model visual representation and utilize the large language model to obtain a damaged vehicle damage assessment result.

[0073] In the above technical scheme, feature extraction is performed on the panoramic image of vehicle damage to obtain encoded image features; based on the encoded image features and prompt words, the damaged parts of the vehicle are segmented to obtain a segmentation map of the damaged parts of the vehicle; based on the prompt words and the segmentation map of the damaged parts of the vehicle, a multimodal large model is used to perform multimodal information interaction of image and text pairs to obtain a large model visual representation; based on the large model visual representation, a large language model is used to perform vehicle damage assessment to obtain a damaged vehicle damage assessment result; the two modalities of language and picture are integrated, and based on the multimodal large model and the language large model, the effect of vehicle damage assessment is more significant.

[0074] Specifically, refer to Figure 3 The multimodal large-model vehicle damage assessment system based on prompt word guidance uses the ViT (Vision Transformer) model to extract high-level features from the input damaged vehicle panoramic image, and uses the SAM (Segmentation Anything Models) model to segment the damaged parts of the vehicle panoramic image under the guidance of prompt words (such as scratches, dents, deformations, tears and missing). Then, the ViT high-level features and prompt words mapped to the image area of ​​the damaged part after segmentation are input into the multimodal alignment model, the BLIP-2 model is used, and features with multimodal interaction are output. Finally, the prompt for vehicle damage assessment is designed, and the features with multimodal interaction output by the BLIP-2 model are input into the LLM (using qwen2-72b) to realize the final vehicle damage assessment function. Specifically including:

[0075] 1. Damaged vehicle panoramic image feature extraction module

[0076] For the vehicle damage panorama uploaded by mobile phone, the ViT model is used to extract high-level features and realize the encoding of image features. The main calculation steps are as follows:

[0077] Step 1.1 First, the input 224×224 RGB vehicle damage panorama is divided into 16×16 patches, and the patch is mapped to a one-dimensional vector of dimension 768 using a linear mapping layer.

[0078] After adding and concatenating the positional encoding and classification encoding in step 1.2, they are input into the Transformer Encoder structure, which is composed of 6 Encoder Blocks stacked together to obtain the encoded image features.

[0079] 2. Vehicle damaged parts segmentation module

[0080] This module uses the SAM model, takes the encoded image features obtained in the above steps and the vehicle damage prompt words (such as scratches, dents, deformations, tears and missing) as input, and obtains the segmentation map of the damaged parts of the vehicle. The main calculation steps are as follows:

[0081] Step 2.1 uses the five types of vehicle damage as prompt words and inputs them into the prompt encoder. Here, the encoder in the CLIP model is used to map the prompt words to the prompt feature space.

[0082] Step 2.2 combines the image encoding in step 1.2 and the prompt word encoding in step 2.1, and inputs them into the maskdecoder structure to decode the segmented damaged vehicle part image. The segmented result will be more reliable than directly using the damaged vehicle image, and it also reduces the pressure on the multimodal alignment model in the subsequent steps.

[0083] 3. Multimodal Alignment Module

[0084] This module uses BLIP-2 as the network architecture of the multimodal alignment model.

[0085] Step 3.1 maps the image area of ​​the segmented damaged vehicle output in step 2.2 and extracts the feature map corresponding to the ViT high-level feature.

[0086] Step 3.2 combines the vehicle damage prompt words (such as scratches, dents, deformations, tears, and missing) with the features of the ViT image encoding obtained in step 3.1 Figure 1 It is used as input and sent to the Q-Former module of the BLIP-2 model to perform multimodal information interaction between image and text pairs, and is fused with the query vector to output a visual representation that can be understood by the subsequent module LLM.

[0087] 4. Large language model vehicle damage assessment module

[0088] This module uses the large language model LLM to implement the final vehicle damage assessment function, using the Qwen-1.5-72b model.

[0089] Step 4.1 Write command prompt words so that the large language model can accurately identify the instructions to achieve a specific description of the vehicle damage assessment parts and damage conditions. An example of writing prompt words is: "You are a professional vehicle damage assessment expert. Now you need to analyze the vehicle's damaged parts and injuries based on the given vehicle accident information (including images and text descriptions) and generate a detailed damage assessment report.\n\n**Task requirements**:\n1. Identify the damaged parts of the vehicle from the input image.\n2. Based on the image and text description, analyze the damage of each damaged part and explain the type and severity of the damage.\n3. Describe the identified damaged parts and injuries in detail and output them in a clear manner.\n\n**Input**:\n- Scene image of the vehicle accident\n- Text of the accident Description (including vehicle type, accident process, eyewitness description, etc.)\n\n**Output**:\n-Detailed description of each damaged part (such as left front fender, rear bumper, etc.).\n-Type of damage to each part (such as minor scratches, dents, severe damage, etc.).\n-Severity of damage (such as minor, moderate, severe).\n\n\n**Note**:\n-When describing the damage, please be detailed and accurate to avoid missing any possible damage.\n-The output format is clear and concise, easy to understand and apply in the actual damage assessment process.\n\nNow please generate a detailed description of the damaged parts and injuries based on the input content. "

[0090] Step 4.2 uses the command prompt and the data representation obtained in step 3.2 as the input of LLM, adopts the Qwen-1.5-72b network structure, and finally outputs the result of the damaged vehicle condition, combining the injured part and the injury condition, such as: dented car door.

[0091] In the above technical scheme, the solution for vehicle damage assessment in the insurance field based on the prompt-word-guided multimodal large model integrates ViT, SAM, multimodal alignment model, and LLM into a complete set of processes; compared with traditional models that only use CNN-based or transformer-based architectures, this application has well integrated the two modalities of language and image. With the support of the multimodal large model and the language large model LLM, the effect of vehicle damage assessment is more significant.

[0092] Those skilled in the art will appreciate that the present application may be implemented as a system, method or computer program product.

[0093] Therefore, the present disclosure may be specifically implemented in the following forms, namely: it may be completely hardware, it may be completely software (including firmware, resident software, microcode, etc.), or it may be a combination of hardware and software, generally referred to herein as a "circuit", "module" or "system". In addition, in some embodiments, the present application may also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium may contain computer-readable program code.

[0094] Any combination of one or more computer-readable media can be used. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0095] Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art can change, modify, replace and modify the above embodiments within the scope of the present application. On this basis, a variety of replacements and improvements can be made to the present application, all of which fall within the scope of protection of the present application.

Claims

1. A multi-modal large-model vehicle damage assessment method based on prompt words, characterized in that: The following steps are involved: Extract features from the vehicle damage panorama to obtain encoded image features; Based on the encoded image features and prompt words, the damaged part of the vehicle is segmented to obtain a segmentation map of the damaged part of the vehicle; Based on the prompt words and the segmentation map of the damaged parts of the vehicle, a multimodal large model is used to perform multimodal image-text pair information interaction to obtain a large model visual representation; Based on the large model visual representation, a large language model is used to perform vehicle damage assessment to obtain a damaged vehicle damage assessment result.

2. The multi-modal large-model vehicle damage assessment method based on prompt word guidance according to claim 1 is characterized in that: The steps of extracting features from the vehicle damage panorama to obtain encoded image features are as follows: The ViT model is used to extract features from the vehicle damage panorama to obtain the encoded image features.

3. The multi-modal large-model vehicle damage assessment method based on prompt word guidance according to claim 2 is characterized in that: The step of extracting features from the vehicle damage panorama using the ViT model to obtain the encoded image features specifically includes: Performing a block operation on the vehicle damage panorama, and mapping the blocks into vectors using a linear mapping layer to obtain block vectors; After adding the splicing position coding and the classification coding to the block vector, the vector is input into the Transformer Encoder structure to obtain the encoded image features.

4. The multi-modal large-model vehicle damage assessment method based on prompt word guidance according to claim 3 is characterized in that: The steps of segmenting the damaged part of the vehicle based on the encoded image features and prompt words to obtain a segmentation map of the damaged part of the vehicle are specifically as follows: The SAM model is used to segment the damaged part of the vehicle to obtain a segmentation map of the damaged part of the vehicle.

5. The multi-modal large-model vehicle damage assessment method based on prompt word guidance according to claim 4 is characterized in that: The step of using the SAM model to segment the damaged part of the vehicle to obtain a segmentation map of the damaged part of the vehicle specifically includes: The damage to the vehicle is used as a prompt word, input into the prompt encoder, and the prompt word is mapped to the prompt feature space; The encoded image features and the prompt words are fused and input into a mask decoder structure to decode the segmented damaged vehicle part image to obtain a segmentation map of the damaged vehicle part.

6. The multi-modal large-model vehicle damage assessment method based on prompt word guidance according to claim 5 is characterized in that: Based on the prompt words and the segmentation map of the damaged parts of the vehicle, a multimodal large model is used to perform multimodal image-text pair information interaction to obtain a large model visual representation step, specifically: Based on the prompt words and the segmentation map of the damaged parts of the vehicle, a BLIP-2 model is used to perform multimodal information interaction of image and text pairs to obtain the large model visual representation.

7. The multi-modal large-model vehicle damage assessment method based on prompt word guidance according to claim 6 is characterized in that: Based on the prompt words and the segmentation map of the damaged part of the vehicle, the step of performing multimodal information interaction of the image and text pairs using the BLIP-2 model to obtain the visual representation of the large model specifically includes: Mapping the segmentation map of the damaged part of the vehicle to extract a feature map of the corresponding ViT high-level feature; The feature graphs of the prompt words and the ViT high-level features are input into the Q-Former module of the BLIP-2 model to perform multimodal information interaction of image-text pairs, and are fused with the query vector to obtain the visual representation of the large model.

8. The multi-modal large-model vehicle damage assessment method based on prompt word guidance according to claim 7 is characterized in that: The steps of using the large language model to perform vehicle damage assessment based on the large model visual representation to obtain the damaged vehicle damage assessment result specifically include: Write instruction prompts; The instruction prompt words and the large model visual representation are input into the large language model to obtain the damage assessment result of the damaged vehicle.

9. The multi-modal large-model vehicle damage assessment method based on prompt word guidance according to claim 8 is characterized in that: The large language model adopts the Qwen-1.5-72b network structure.

10. A multi-modal large-model vehicle damage assessment system based on prompt words, characterized in that: include: A damaged vehicle panoramic image feature extraction module is used to extract features from the vehicle damage panoramic image to obtain encoded image features; A vehicle damaged part segmentation module is used to segment the vehicle damaged part based on the encoded image features and prompt words to obtain a segmentation map of the vehicle damaged part; A multimodal alignment module, for performing information interaction of multimodal image-text pairs using a multimodal large model based on the prompt word and the segmentation map of the damaged part of the vehicle to obtain a large model visual representation; The large language model vehicle damage assessment module is used to perform vehicle damage assessment based on the large model visual representation and utilize the large language model to obtain a damaged vehicle damage assessment result.

Images