Vehicle damaged picture identification method and related product
By determining the correlation and grouping of multiple images of damaged vehicles, and using a multimodal large language model to process images from multiple angles, the problem of overlooking subtle damage in traditional manual review is solved, and more accurate recognition of images of damaged vehicles is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 太保科技有限公司
- Filing Date
- 2026-01-06
- Publication Date
- 2026-04-17
AI Technical Summary
Traditional manual review methods for identifying damaged vehicle images can easily overlook subtle but important damage points when faced with complex scenarios involving multiple images, affecting the accuracy of the identification results.
By determining the correlation between multiple images of damaged vehicles to be identified, grouping them for identification, using a multimodal large language model to determine the correlation and local magnification relationship between images, and combining image information from different angles and positions for identification.
It improves the accuracy of vehicle damage image recognition, avoids misjudgment based on a single image, and accurately identifies the location and extent of damage to components.
Smart Images
Figure CN121884074A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image recognition, and in particular to a method for recognizing images of damaged vehicles and related products. Background Technology
[0002] In the vehicle insurance claims process, after a car accident, the damaged vehicle usually needs to be taken to a repair shop for repairs. During this process, to ensure that the insurance company can reasonably cover the repair costs, the repair personnel must submit a series of supporting documents to the insurance company, the most crucial of which are photos depicting the vehicle's damage. These photos not only verify the authenticity and severity of the accident but also form the basis for determining the specific damaged parts and the corresponding repair plan.
[0003] Traditional claims review processes rely on manual review, where experienced claims specialists or repair experts examine each image of vehicle damage, using their experience and expertise to determine which parts of the vehicle have suffered what type of damage, such as scratches, dents, or cracks. However, this manual review method has certain drawbacks: when faced with complex scenarios involving multiple images, manual review can overlook some subtle but important damage points, affecting the accuracy of the final identification results.
[0004] Improving the accuracy of vehicle damage image recognition is a technical problem that urgently needs to be solved. Summary of the Invention
[0005] To address the aforementioned issues, this application provides a method and related products for recognizing damaged vehicle images, with the aim of improving the accuracy of such recognition.
[0006] The embodiments of this application disclose the following technical solutions:
[0007] The first aspect of this application provides a method for recognizing vehicle damage images, the method comprising:
[0008] Based on multiple images of damaged vehicles to be identified, the correlation between the multiple images of damaged vehicles to be identified is determined;
[0009] Based on the correlation between the multiple images of damaged vehicles to be identified, the multiple images of damaged vehicles to be identified are grouped to obtain multiple groups of images of damaged vehicles to be identified.
[0010] The damaged parts of each group of images of damaged vehicles to be identified are identified to obtain the identification result of each group of images of damaged vehicles to be identified.
[0011] Optionally, determining the association between multiple images of damaged vehicles to be identified, based on multiple images of damaged vehicles to be identified, includes:
[0012] Using a multimodal large language model, the association between multiple images of damaged vehicles to be identified is determined based on the shooting range of the images of the damaged vehicles to be identified.
[0013] The multimodal large language model is used to determine whether each image of a damaged vehicle to be identified is a magnified partial image of any one of the multiple images of damaged vehicles to be identified.
[0014] Optionally, the step of identifying the damaged components in each set of damaged vehicle images to obtain the identification result for each set of damaged vehicle images includes:
[0015] For any set of images of damaged vehicles to be identified:
[0016] A multimodal large language model is used to determine the damage status of the damaged parts in each image of a damaged vehicle to be identified, as well as the correlation between the damaged parts and other images of damaged vehicles to be identified.
[0017] Optionally, before determining the correlation between the multiple images of damaged vehicles to be identified based on the multiple images of damaged vehicles to be identified, the method further includes:
[0018] Filter the received images. For any given image:
[0019] If the image is related to vehicle damage, it is retained as the image to be identified as vehicle damage; if the image is not related to vehicle damage, it is removed.
[0020] Optionally, after identifying the damaged components in each set of damaged vehicle images to obtain the identification result for each set of damaged vehicle images, the method further includes:
[0021] Based on the recognition results of multiple sets of damaged vehicle images to be identified, duplicate images of the same damaged vehicle component are deduplicated to obtain the target image corresponding to each component.
[0022] A second aspect of this application provides a vehicle damage image recognition device, the device comprising:
[0023] The association determination module is used to determine the association relationship between multiple images of damaged vehicles to be identified based on multiple images of damaged vehicles to be identified.
[0024] The grouping module is used to group the multiple images of damaged vehicles to be identified based on the correlation between them, so as to obtain multiple groups of images of damaged vehicles to be identified.
[0025] The recognition module is used to identify the damaged parts of each group of damaged vehicle images to obtain the recognition result of each group of damaged vehicle images.
[0026] Optionally, the association determination module is used to:
[0027] Using a multimodal large language model, the association between multiple images of damaged vehicles to be identified is determined based on the shooting range of the images of the damaged vehicles to be identified.
[0028] The multimodal large language model is used to determine whether each image of a damaged vehicle to be identified is a magnified partial image of any one of the multiple images of damaged vehicles to be identified.
[0029] Optionally, the device further includes: an image filtering module;
[0030] The image filtering module is used to filter multiple received images. For any given image:
[0031] If the image is related to vehicle damage, it is retained as the image to be identified as vehicle damage; if the image is not related to vehicle damage, it is removed.
[0032] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the vehicle damage image recognition method provided in any implementation of the first aspect.
[0033] The fourth aspect of this application provides a processor for running a computer program that, when running, executes a vehicle damage image recognition method as provided in any implementation of the first aspect.
[0034] Compared with the prior art, this application has the following beneficial effects:
[0035] The vehicle damage image recognition method provided in this application embodiment determines the correlation between multiple images of damaged vehicles to be recognized; based on the correlation between the multiple images of damaged vehicles to be recognized, the images are grouped to obtain multiple groups of damaged vehicle images to be recognized; and the damaged parts of each group of damaged vehicle images are identified to obtain the recognition result of each group of damaged vehicle images.
[0036] By establishing the relationships between multiple images of damaged vehicles to be identified, a more comprehensive understanding of the damage can be gained. This includes information from images taken from different angles and locations, allowing for more accurate identification of the location and extent of damage to components, thus improving the accuracy of vehicle damage image recognition. Grouping images based on these relationships allows for the simultaneous understanding of a group of damaged vehicle images. This enables the identification of detailed images of only partially damaged areas, avoiding misjudgments from relying on single images. This further enhances the accuracy of vehicle damage image recognition by more precisely identifying the location and extent of damage to components. Attached Figure Description
[0037] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0038] Figure 1 A flowchart illustrating a method for recognizing damaged vehicle images provided in this application embodiment;
[0039] Figure 2 A flowchart illustrating yet another method for recognizing damaged vehicle images provided in this application embodiment;
[0040] Figure 3 This is a schematic diagram of the structure of a vehicle damage image recognition device provided in an embodiment of this application. Detailed Implementation
[0041] As described earlier, the traditional claims review process relies on manual review, where experienced claims specialists or repair experts examine each image of vehicle damage, using their personal experience and expertise to determine which parts of the vehicle have suffered what type of damage, such as scratches, dents, or cracks. However, this manual review method has certain drawbacks: when faced with complex scenarios involving multiple images, manual review may overlook some subtle but important damage points, affecting the accuracy of the final identification results.
[0042] In recent years, with the development of computer vision technology and artificial intelligence algorithms, deep learning-based image recognition technology can automatically detect and classify the types of vehicle damage in photos. However, it mainly focuses on locating damaged parts of a vehicle in a single image. First, it identifies what the parts are in the image, and then uses visual positioning algorithms to confirm the damage on the parts.
[0043] However, in real-world scenarios, repair personnel may provide multiple vehicle photos. Many vehicle photos are taken from unusual angles, making it impossible to determine the type of damaged parts based on a single photo. Therefore, it is necessary to combine multiple photos from different perspectives to make a unified judgment.
[0044] In view of the above problems, this application proposes a method and related products for recognizing damaged vehicle images. The method involves determining the correlation between multiple damaged vehicle images to be recognized; grouping the damaged vehicle images to be recognized into multiple groups based on the correlation between the multiple damaged vehicle images to be recognized; and identifying the damaged components in each group of damaged vehicle images to obtain the recognition result for each group of damaged vehicle images.
[0045] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.
[0046] See Figure 1 This figure is a flowchart of a vehicle damage image recognition method provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps:
[0047] S101. Based on multiple images of damaged vehicles to be identified, determine the correlation between the multiple images of damaged vehicles to be identified.
[0048] The correlation is determined based on the shooting range of the image of the damaged vehicle to be identified.
[0049] By determining the correlation between multiple images of damaged vehicles to be identified, a more comprehensive understanding of the damage can be obtained. By comprehensively analyzing image information from different angles and positions, the location and extent of damage to damaged parts can be identified more accurately, thereby improving the accuracy of vehicle damage image recognition.
[0050] S102. Based on the correlation between the multiple images of damaged vehicles to be identified, the multiple images of damaged vehicles to be identified are grouped to obtain multiple groups of images of damaged vehicles to be identified.
[0051] Grouping related images of damaged vehicles ensures that images of the same damaged component are aggregated for analysis, significantly improving the accuracy of damage identification. Furthermore, it reduces redundant calculations and analyses, increasing processing efficiency.
[0052] S103. Identify the damaged parts of each group of damaged vehicle images to obtain the identification result of each group of damaged vehicle images.
[0053] For example, a multimodal large language model is used to perform joint input, overall understanding and cross-image reasoning on a set of vehicle damage images, thereby identifying the damaged vehicle parts and their damage details, and establishing semantic associations between images. This approach avoids misjudging local textures as damage by recognizing each image separately, thus improving the accuracy of vehicle damage image recognition.
[0054] The vehicle damage image recognition method provided in this application embodiment determines the correlation between multiple images of damaged vehicles to be recognized; based on the correlation between the multiple images of damaged vehicles to be recognized, the images are grouped to obtain multiple groups of damaged vehicle images to be recognized; and the damaged parts of each group of damaged vehicle images are identified to obtain the recognition result of each group of damaged vehicle images.
[0055] By establishing the relationships between multiple images of damaged vehicles to be identified, a more comprehensive understanding of the damage can be gained. This includes information from images taken from different angles and locations, allowing for more accurate identification of the location and extent of damage to components, thus improving the accuracy of vehicle damage image recognition. Grouping images based on these relationships and then identifying damaged components within each group avoids situations where a single image cannot identify specific details, leading to incorrect judgments about damaged components. This further enhances the accuracy of vehicle damage image recognition by more precisely identifying the location and extent of damage to components.
[0056] To improve the vehicle damage image recognition method described in the above embodiments, the steps of "filtering multiple received images" and "deriving duplicate images of multiple vehicle damage images corresponding to the same component based on the recognition results of multiple sets of vehicle damage images to be identified, and obtaining the target image corresponding to each component" have been added.
[0057] See Figure 2 This figure is a flowchart of another vehicle damage image recognition method provided in an embodiment of this application. Figure 2 As shown, the method includes the following steps:
[0058] S201. Filter the received images.
[0059] In one feasible implementation:
[0060] Filter the received images. For any given image:
[0061] If the image is related to vehicle damage, it is retained as the image to be identified as vehicle damage; if the image is not related to vehicle damage, it is removed.
[0062] Since the repair shop provides multiple images, including not only those showing vehicle damage but also other types of images such as repair lists and accident scene photos, the received images are first filtered, for example, using image classification algorithms to remove non-vehicle damage images.
[0063] S202. Based on multiple images of damaged vehicles to be identified, determine the correlation between the multiple images of damaged vehicles to be identified.
[0064] In one feasible implementation:
[0065] Using a multimodal large language model, the association between multiple images of damaged vehicles to be identified is determined based on the shooting range of the images of the damaged vehicles to be identified.
[0066] The multimodal large language model is used to determine whether each image of a damaged vehicle to be identified is a magnified partial image of any one of the multiple images of damaged vehicles to be identified.
[0067] For example, multiple images of damaged vehicles to be identified are input into a multimodal large language model. The multimodal large language model returns the correlation between the multiple images of damaged vehicles to be identified. The multimodal large language model determines whether each image of damaged vehicles to be identified is a magnified partial image of any of the multiple images of damaged vehicles to be identified, and outputs whether image i is a magnified partial image of image j or image i is not a magnified partial image of any of the images.
[0068] By determining the correlation between multiple images of damaged vehicles to be identified, a more comprehensive understanding of the damage can be obtained. By comprehensively analyzing image information from different angles and positions, the location and extent of damage to damaged parts can be identified more accurately, thereby improving the accuracy of vehicle damage image recognition.
[0069] S203. Based on the correlation between the multiple images of damaged vehicles to be identified, the multiple images of damaged vehicles to be identified are grouped to obtain multiple groups of images of damaged vehicles to be identified.
[0070] The images of damaged vehicles that are related to each other are grouped together here.
[0071] Grouping related images of damaged vehicles allows a multimodal large language model to understand multiple images at once. This enables the model to identify detailed images of only partially damaged areas based on the relationships between the images. This avoids situations where a single image cannot identify the damaged parts, leading to incorrect judgments. As a result, the model can more accurately identify the location and extent of damage to the damaged parts, thus improving the accuracy of vehicle damage image recognition.
[0072] S204. Identify the damaged parts of each group of damaged vehicle images to obtain the identification result of each group of damaged vehicle images.
[0073] In one feasible implementation:
[0074] For any set of images of damaged vehicles to be identified:
[0075] A multimodal large language model is used to determine the damage status of the damaged parts in each image of a damaged vehicle to be identified, as well as the correlation between the damaged parts and other images of damaged vehicles to be identified.
[0076] Here, a multimodal large language model can be used to identify the damaged components in each set of vehicle damage images. For example, a set of four vehicle damage images is input into the multimodal large language model, and the output of the model is as follows:
[0077] The first picture clearly shows damage to the rear bumper on the right rear side of the vehicle.
[0078] The broken panel in the second image has the same shape as the damage to the rear bumper in the first image. The second image is a magnified view of the damage to the rear bumper near the rear wheel in the first image.
[0079] The third image is a further enlarged view of the second image, showing the repairman disassembling the broken rear bumper to expose the internal damage. Therefore, the third image can also be used to assess the damage to the rear bumper.
[0080] The fourth image is a top view of the rear bumper shown in the first, second, and third images, demonstrating the damage to the clips on the rear bumper.
[0081] By using a multimodal large language model to perform joint input, overall understanding, and cross-graph reasoning on each set of vehicle damage images, the system can identify damaged vehicle parts and their damage details, and establish semantic relationships between images. This avoids the situation where local detail images cannot be identified based on a single image, leading to incorrect judgment of damaged parts in the image, thus improving the accuracy of vehicle damage image recognition.
[0082] S205. Based on the recognition results of multiple sets of damaged vehicle images to be identified, duplicate images of multiple damaged vehicle images corresponding to the same component are deduplicated to obtain the target image corresponding to each component.
[0083] Since each component will have multiple damage images, these images may have the following problems: there may be multiple images taken from similar angles; some images capture the overall damage of the component, but the details of the damage are not clear.
[0084] Here, a multimodal large language model can be used to deduplicate multiple images of damaged vehicles corresponding to the same component, based on the recognition results of multiple sets of images of damaged vehicles to be identified. The target image is obtained based on the image content, shooting angle, and clarity of damage details of the images of the damaged vehicles to be identified. The target image is a representative image that best reflects the damage condition of the component.
[0085] For example, after obtaining the recognition results of the damaged vehicle images in each group of images, part 1 corresponds to Figure a and Figure b. After deduplication, the target image of part 1 is Figure a; part k corresponds to Figure c, Figure d and Figure e. After deduplication, the target images of part k are Figure c and Figure d.
[0086] This application provides another method for recognizing damaged vehicle images, which involves filtering multiple received images; determining the correlation between the multiple damaged vehicle images based on the correlation between them; grouping the multiple damaged vehicle images based on the correlation between them to obtain multiple groups of damaged vehicle images; identifying the damaged components in each group of damaged vehicle images to obtain the recognition result for each group of damaged vehicle images; and deduplicating multiple damaged vehicle images corresponding to the same component based on the recognition results of multiple groups of damaged vehicle images to obtain the target image corresponding to each component.
[0087] Screening images before determining their relationships helps eliminate irrelevant images of vehicle damage. By establishing relationships between multiple images of damaged vehicles to be identified, a more comprehensive understanding of the damage can be gained, considering information from images from different angles and positions. This allows for more accurate identification of the location and extent of damage to components, improving the accuracy of vehicle damage image recognition. Grouping images based on their relationships and identifying damaged components within each group avoids situations where a single image cannot identify details, leading to incorrect identification of damaged components. This further enhances the accuracy of vehicle damage image recognition. After obtaining the recognition results for multiple groups of damaged vehicle images, duplicate images of the same component are removed, and the images within each group are simplified, retaining only the most representative target images, greatly simplifying subsequent review work.
[0088] Based on the vehicle damage image recognition method described in the preceding embodiments, this application also provides a vehicle damage image recognition device. Figure 3 This is a schematic diagram of the device. Figure 3 As shown, the vehicle damage image recognition device includes:
[0089] The association determination module 301 is used to determine the association relationship between multiple images of damaged vehicles to be identified based on multiple images of damaged vehicles to be identified.
[0090] The grouping module 302 is used to group the multiple images of damaged vehicles to be identified based on the correlation between them, so as to obtain multiple groups of images of damaged vehicles to be identified.
[0091] The recognition module 303 is used to identify the damaged parts of each group of damaged vehicle images to obtain the recognition result of each group of damaged vehicle images.
[0092] Optionally, the association determination module is used to:
[0093] Using a multimodal large language model, the association between multiple images of damaged vehicles to be identified is determined based on the shooting range of the images of the damaged vehicles to be identified.
[0094] The multimodal large language model is used to determine whether each image of a damaged vehicle to be identified is a magnified partial image of any one of the multiple images of damaged vehicles to be identified.
[0095] Optionally, the identification module is used for:
[0096] For any set of images of damaged vehicles to be identified:
[0097] A multimodal large language model is used to determine the damage status of the damaged parts in each image of a damaged vehicle to be identified, as well as the correlation between the damaged parts and other images of damaged vehicles to be identified.
[0098] Optionally, the device further includes: an image filtering module;
[0099] The image filtering module is used to filter multiple received images. For any given image:
[0100] If the image is related to vehicle damage, it is retained as the image to be identified as vehicle damage; if the image is not related to vehicle damage, it is removed.
[0101] Optionally, the device further includes: an image deduplication module;
[0102] The image deduplication module is used to deduplicatize multiple images of damaged vehicles corresponding to the same component based on the recognition results of multiple sets of images of damaged vehicles to be identified, so as to obtain the target image corresponding to each component.
[0103] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the vehicle damage image recognition method as described in any of the method embodiments.
[0104] Furthermore, this application embodiment also provides a processor for running a computer program, which executes the vehicle damage image recognition method as described in any of the foregoing method embodiments.
[0105] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0106] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for recognizing damaged vehicle images, characterized in that, include: Based on multiple images of damaged vehicles to be identified, the correlation between the multiple images of damaged vehicles to be identified is determined; Based on the correlation between the multiple images of damaged vehicles to be identified, the multiple images of damaged vehicles to be identified are grouped to obtain multiple groups of images of damaged vehicles to be identified. The damaged parts of each group of images of damaged vehicles to be identified are identified to obtain the identification result of each group of images of damaged vehicles to be identified.
2. The method according to claim 1, characterized in that, The step of determining the correlation between multiple images of damaged vehicles to be identified, based on multiple images of damaged vehicles to be identified, includes: Using a multimodal large language model, the association between multiple images of damaged vehicles to be identified is determined based on the shooting range of the images of the damaged vehicles to be identified. The multimodal large language model is used to determine whether each image of a damaged vehicle to be identified is a magnified partial image of any one of the multiple images of damaged vehicles to be identified.
3. The method according to claim 1, characterized in that, The process of identifying the damaged components in each set of images of damaged vehicles to be identified, and obtaining the identification result for each set of images of damaged vehicles to be identified, includes: For any set of images of damaged vehicles to be identified: A multimodal large language model is used to determine the damage status of the damaged parts in each image of a damaged vehicle to be identified, as well as the correlation between the damaged parts and other images of damaged vehicles to be identified.
4. The method according to claim 1, characterized in that, Before determining the correlation between multiple images of damaged vehicles to be identified, the method further includes: Filter the received images. For any given image: If the image is related to vehicle damage, it is retained as the image to be identified as the vehicle damage image; if the image is not related to vehicle damage, it is removed.
5. The method according to claim 1, characterized in that, After identifying the damaged components in each set of damaged vehicle images to obtain the identification result for each set of damaged vehicle images, the process further includes: Based on the recognition results of multiple sets of damaged vehicle images to be identified, duplicate images of the same damaged vehicle component are deduplicated to obtain the target image corresponding to each component.
6. A vehicle damage image recognition device, characterized in that, include: The association determination module is used to determine the association relationship between multiple images of damaged vehicles to be identified based on multiple images of damaged vehicles to be identified. The grouping module is used to group the multiple images of damaged vehicles to be identified based on the correlation between them, so as to obtain multiple groups of images of damaged vehicles to be identified. The recognition module is used to identify the damaged parts of each group of damaged vehicle images to obtain the recognition result of each group of damaged vehicle images.
7. The apparatus according to claim 6, characterized in that, The association determination module is used for: Using a multimodal large language model, the association between multiple images of damaged vehicles to be identified is determined based on the shooting range of the images of the damaged vehicles to be identified. The multimodal large language model is used to determine whether each image of a damaged vehicle to be identified is a magnified partial image of any one of the multiple images of damaged vehicles to be identified.
8. The apparatus according to claim 6, characterized in that, The device further includes: an image filtering module; The image filtering module is used to filter multiple received images. For any given image: If the image is related to vehicle damage, it is retained as the image to be identified as vehicle damage; if the image is not related to vehicle damage, it is removed.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the vehicle damage image recognition method as described in any one of claims 1-5.
10. A processor, characterized in that, Used to run a computer program, which, when running, executes the vehicle damage image recognition method as described in any one of claims 1-5.