Automatic image annotation method and device, computer equipment and storage medium

Abstract

The invention discloses an automatic image annotation method and device, computer equipment and a storage medium, belongs to the technical field of artificial intelligence, and is applied to processing of vehicle insurance loss assessment images. According to the method, irrelevant information is removed by screening and classifying the original image data. And then, key appearance parts of the vehicle are accurately positioned by using a deep learning segmentation and recognition technology, and specific damaged parts are matched and positioned in combination with the three-dimensional model, so that part-level refined analysis is realized. On the basis, a damage area is extracted through local screenshot, image matching is conducted through a preset standard part damage library, the damage category and the damage degree are automatically recognized, and structured damage information is generated. And finally, integrally labeling and outputting the damage position, category and degree label and the target image set to form a high-quality labeled image set. According to the method and the device, efficient and unified automatic labeling can be carried out on mass traffic accident pictures in a short time, and the picture labeling efficiency and consistency are remarkably improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to an automated image annotation method, apparatus, computer equipment, and storage medium. Background Technology

[0002] In the current insurance industry, especially in the auto insurance claims process, the annotation of car accident image data has become a crucial step in improving the efficiency and accuracy of claims processing. With the frequent occurrence of traffic accidents and the continuous growth in the number of motor vehicles, insurance companies need to process massive amounts of accident scene photos and videos. This visual data is not only used for preliminary assessment of vehicle damage but also serves as an important basis for subsequent damage assessment, liability determination, and claims decisions. To efficiently extract value from this unstructured data, the industry has widely adopted image recognition technologies based on machine learning and deep learning, significantly reducing the workload of manual review and greatly improving the speed of claims processing. However, training an efficient and intelligent recognition model requires high-quality and accurate annotation of car accident images—that is, meticulously marking key information in each image (such as damaged parts, damage type, severity, etc.). Although this process is crucial, the industry currently lacks unified annotation standards and specifications, leading to differences in annotation results between different institutions or teams, hindering data interoperability and collaborative model optimization, and restricting the development of intelligent claims systems.

[0003] In current data annotation practices, the primary problem is the lack and inconsistency of annotation standards. Because a unified annotation standard has not yet been established within the industry, different annotators often interpret the same image content based on personal experience or subjective judgment, leading to inconsistent annotation results and directly affecting the accuracy and generalization ability of model training. Meanwhile, vehicle damage is complex and diverse, encompassing different vehicle models, materials, and manufacturing processes. This places high demands on the professional knowledge of annotators, requiring a certain background in automotive structure and insurance claims assessment to accurately identify and annotate the degree and type of damage. Given the massive amount of accident image data, ensuring that each case receives comprehensive, detailed, and accurate annotation has also become a highly challenging task. Summary of the Invention

[0004] The purpose of this application is to provide an automated image annotation method, apparatus, computer device, and storage medium, which can efficiently and uniformly annotate a large number of car accident images in a short time, significantly improving the efficiency and consistency of image annotation.

[0005] To address the aforementioned technical problems, this application provides an automated image annotation method, employing the following technical solution:

[0006] An automated image annotation method, comprising:

[0007] Acquire pre-collected raw image data, which consists of accident record images related to vehicles;

[0008] The original image data is filtered to obtain the target image set, which is a collection of image data containing vehicle information;

[0009] Identify automotive parts from a target image set and locate key exterior components of the vehicle;

[0010] Based on the key exterior components, locate the damaged parts of the car, and take partial screenshots of the damaged parts to obtain a damage atlas of the parts;

[0011] The component damage atlas is matched with the standard component damage atlas in the preset damaged component library, and component damage information is generated based on the image matching results.

[0012] Based on the component damage atlas and component damage information, the target image set is annotated to obtain the annotated image set.

[0013] To address the aforementioned technical problems, this application also provides an automated image annotation device, which employs the following technical solution:

[0014] An automated image annotation device, comprising:

[0015] The data acquisition module is used to acquire pre-collected raw image data, which consists of accident record images related to vehicles.

[0016] The data filtering module is used to filter the raw image data to obtain the target image set, which is a collection of image data containing vehicle information;

[0017] The component recognition module is used to identify automotive components from a target image set and locate key exterior parts of the vehicle.

[0018] The damaged component identification module is used to locate damaged components of the vehicle based on key appearance components, and to take local screenshots of the damaged components to obtain a component damage atlas.

[0019] The damage information recognition module is used to perform image matching between the component damage image set and the standard component damage image in the preset damaged component library, and generate component damage information based on the image matching result.

[0020] The image annotation module is used to annotate the target image set based on the component damage atlas and component damage information to obtain an annotated image set.

[0021] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0022] A computer device includes a memory and a processor, the memory storing computer-readable instructions, the processor executing the computer-readable instructions to implement the steps of the automated image annotation method as described in any of the preceding claims.

[0023] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0024] A computer-readable storage medium storing computer-readable instructions that, when executed by a processor, implement the steps of the automated image annotation method as described in any one of the preceding descriptions.

[0025] Compared with the prior art, the embodiments of this application have the following main advantages:

[0026] This application discloses an automated image annotation method, apparatus, computer equipment, and storage medium, belonging to the field of artificial intelligence technology, and applied to the processing of vehicle insurance damage assessment images. This application filters and classifies the original image data, removing irrelevant information to ensure that processing focuses on valid images containing the vehicle, reducing redundant computation. Subsequently, deep learning segmentation and recognition technology is used to accurately locate key exterior components of the vehicle, and specific damaged components are located by combining 3D model matching, achieving refined component-level analysis. Based on this, damaged areas are extracted through local screenshots, and image matching is performed using a pre-set standard component damage library to automatically identify damage categories and degrees, generating structured damage information. Finally, the damage location, category, and degree labels are integrated with the target image set for unified annotation output, forming a high-quality annotated image set. This application can achieve efficient and unified automatic annotation of massive amounts of vehicle accident images in a short time, significantly improving image annotation efficiency and consistency. Attached Figure Description

[0027] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 An exemplary system architecture diagram is shown, in which this application can be applied;

[0029] Figure 2A flowchart illustrating one embodiment of the automated image annotation method according to this application is shown;

[0030] Figure 3 A schematic diagram of one embodiment of the automated image annotation apparatus according to this application is shown;

[0031] Figure 4 A schematic diagram of the structure of one embodiment of a computer device according to this application is shown. Detailed Implementation

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0033] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0034] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0035] like Figure 1 As shown, system architecture 100 may include terminal device 101, network 102, and server 103. Terminal device 101 may be a laptop 1011, tablet 1012, or mobile phone 1013. Network 102 is used as a medium to provide a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables.

[0036] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various communication client applications can be installed on terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0037] Terminal device 101 can be various electronic devices with a display screen and support web browsing. In addition to laptops 1011, tablets 1012, or mobile phones 1013, terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, and a desktop computer, etc.

[0038] Server 103 can be a server that provides various services, such as a backend server that provides support for the pages displayed on terminal device 101.

[0039] It should be noted that the automated image annotation method provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the automated image annotation device is generally set in the server / terminal device.

[0040] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative; the system can have any number of terminal devices, networks, and servers depending on implementation needs.

[0041] Continue to refer to Figure 2 A flowchart of an embodiment of the automated image annotation method according to this application is shown. The automated image annotation method includes the following steps:

[0042] S201, acquire pre-collected raw image data, wherein the raw image data is accident record images related to the vehicle;

[0043] Specifically, the system first needs to batch retrieve archived accident scene images and damage assessment images from the insurance company's backend data management system or historical case archive. This process is typically achieved through an API or a dedicated data synchronization module, and requires standardized preprocessing of data formats from different sources. For example, the original images may come from mobile phone photos uploaded by surveyors, accident monitoring frames, supplementary images taken by repair shops, etc., with varying formats (JPEG, PNG, HEIC), and significant differences in resolution and clarity. Therefore, after data acquisition, it is necessary to perform format conversion, resolution standardization, and removal of invalid metadata (such as sensitive GPS information in EXIF ​​data).

[0044] Furthermore, to ensure data integrity, the data capture stage should also associate and store metadata such as case number, timestamp, and geographic location, establishing a mapping index table to facilitate batch retrieval based on cases. Technically, distributed file systems (such as HDFS and Ceph) can be used for image storage management to support concurrent reading and high-throughput processing of large-scale datasets.

[0045] S202, the original image data is filtered to obtain the target image set, wherein the target image set is a set of image data containing vehicle information;

[0046] Specifically, the raw image data not only contains vehicle images but may also contain a large amount of irrelevant content, such as photos of the overall environment of the accident scene, photos of the parties involved in the accident or witnesses, and scanned copies of documents. Therefore, an image filtering model is needed to extract valid vehicle images from the massive dataset. In practice, pre-trained convolutional neural network (CNN) classification models or lightweight architectures (such as MobileNet and EfficientNet) can be used for vehicle detection. Feature vectors are extracted from each image, and multi-class classification is performed to output a "vehicle / non-vehicle" confidence score. To improve the accuracy of filtering, a multi-level filtering strategy is usually adopted. For example, YOLO series object detection models are first used to detect vehicle bounding boxes; if no vehicle boundary is detected, the image is excluded. If a vehicle is detected, a classifier is used to identify the vehicle type, ensuring it is a car and not other means of transportation. In addition, to improve the efficiency of automatic filtering, batch processing and GPU acceleration can be used, combined with TensorRT or ONNX Runtime for model inference acceleration, thereby supporting rapid filtering of tens of millions of images.

[0047] S203, perform automotive component identification on the target image set and locate key exterior components of the vehicle;

[0048] Specifically, the filtered target image set will be used as input, and a deep learning-based part detection and segmentation model will be used for refined recognition. Technically, advanced frameworks such as YOLOv10, Detectron2, or Segment Aspecting (SAM) can be employed to perform object detection and instance segmentation on vehicle images, identifying exterior parts such as the hood, front and rear bumpers, doors, rearview mirrors, windshield, and wheels. During model training, a training set with labeled bounding boxes and masks needs to be prepared in advance to ensure the model maintains high robustness under complex backgrounds, different angles, and different lighting conditions. During the recognition process, multi-scale feature extraction (Feature Pyramid Network, FPN) is first performed on the images. Then, anchor-free detection heads or transformer-based detection heads are used to generate candidate bounding boxes for parts. Finally, non-maximum suppression (NMS) is used to obtain the final localization result. To improve processing speed in production environments, techniques such as model quantization (INT8 / FP16), pruning redundant network layers, and multi-threaded inference can be employed.

[0049] In addition, for special scenarios such as nighttime, rainy days, or backlighting, an image enhancement module can be introduced to defog, enhance brightness, or perform gamma correction on the input image to improve the stability of component recognition.

[0050] S204. Locate the damaged parts of the car based on the key appearance components, and take partial screenshots of the damaged parts to obtain a damage atlas of the parts.

[0051] Specifically, based on the obtained component-level localization results (i.e., the location of key exterior components of the vehicle), the system further executes a damage region extraction algorithm within each detected component region. This process can leverage convolutional neural network-based damage detection models or the fine-grained feature analysis capabilities of visual Transformers to distinguish whether the component surface is intact or has abnormal features such as scratches, dents, and cracks. The model typically performs secondary segmentation of the component at the pixel level, generating a mask for the damaged region. Then, the system crops the original image based on the mask coordinates, generating local screenshots, thereby constructing a component damage atlas.

[0052] To improve the usability of local cropping, the system may automatically extend the edges of the cropped image to preserve the contextual information around the damaged area, while simultaneously resampling the image at high resolution to ensure detail clarity. Technically, this can be achieved by combining OpenCV's image geometric transformation functions (such as warpAffine and resize) and the Pillow library for cropping and enhancement, or by introducing a GPU-accelerated custom CUDA kernel to achieve high-concurrency cropping processing.

[0053] In addition, the system can bind the damaged screenshot with the original image index and store it in a relational database or NoSQL database (such as MongoDB) to form a one-to-one correspondence data structure between the component and the damaged area.

[0054] S205, perform image matching between the component damage atlas and the standard component damage images in the preset damaged component library, and generate component damage information based on the image matching results;

[0055] Specifically, the system internally maintains a standard damaged component library, which consists of component damage examples pre-organized by experts and precisely annotated manually, covering different vehicle models, components, damage types, and levels. For newly generated component damage images, the system compares them with standard samples in the library using an image feature matching algorithm. The matching algorithm can employ Deep Metric Learning, first using a pre-trained image feature extraction network (such as ResNet or Swin Transformer) to map the input image and the sample images in the library into vector spaces, and then calculating cosine similarity or Euclidean distance for matching. To improve the accuracy and robustness of the matching, multimodal information can be combined, such as jointly matching image features with component category and damage location encoding; or using Siamese Network or Triplet Loss to optimize the feature space, making the features of similar damage images more aggregated. After matching is complete, the system will automatically inherit the corresponding label information from the library, such as damage type (scratches, cracks, dents, etc.), severity (mild, moderate, severe) and repair suggestion code, generate structured component damage information and bind it to the screenshot for storage, which can be directly output or used for model training later.

[0056] S206. Based on the component damage atlas and component damage information, the target image set is annotated to obtain an annotated image set.

[0057] Specifically, after completing image matching and damage information generation, the system performs unified annotation processing on the original target image set. The core technology of annotation is to embed the identified component bounding boxes, segmentation masks, and matched damage information into the corresponding annotation file of the image, forming a standardized data format. Commonly used annotation output formats include Pascal VOC (XML), COCO (JSON), or custom JSON / YAML structures. When generating annotations, the system needs to correspond the spatial location, category label, damage type, and damage level of each component one-to-one to ensure that the data structure is clear and traceable. To support AI training, the annotation file records precise coordinate values ​​(pixel level), label numbers, and corresponding text descriptions, and supports version control for subsequent revisions. In technical implementation, batch generation of annotation files can be achieved through automatic annotation toolchains (such as Labellme, CVAT's API interface), or a custom annotation management module can be developed internally to store the results in search engines such as ElasticSearch for easy querying and backtracking. Finally, the system will also perform consistency checks on the labeled data, such as checking whether the bounding boxes are out of bounds, whether the labels are within the predefined category set, and whether the damage level conforms to the business rules, thereby outputting a high-quality set of labeled images.

[0058] Further, the step of filtering the original image data to obtain the target image set specifically includes:

[0059] Target entity recognition is performed on the original image data to obtain the target entity recognition results;

[0060] Based on the target entity recognition results, a pre-trained image classification model is used to classify the original image data to determine whether each image in the original image data contains vehicle information.

[0061] If the image contains vehicle information, it is considered valid image data and is added to a preset image set to obtain the target image set.

[0062] In this embodiment, for the original image data, the system first calls a deep learning-based target entity recognition module (such as using a model like Faster R-CNN, YOLOv10, or DETR) to perform preliminary detection on each image, generating recognition results including category, location, and confidence level. Subsequently, the system inputs the recognition results as filtering conditions into a pre-trained image classification model. This classification model can be a convolutional neural network (such as EfficientNet or MobileNet) trained on a large number of vehicle and non-vehicle images. It performs high-dimensional mapping and probability determination on the features of each image to confirm whether the image truly contains vehicle-related information. The system marks images with the determination result of "containing vehicles" as valid image data and automatically adds them to the target image set according to preset indexing rules. Simultaneously, it records their metadata (such as case number, timestamp, and detection label), establishing an efficient data index table to ensure rapid retrieval, tracking, and updating of these images during processing, thereby forming a high-quality, manageable target image set.

[0063] By following the steps above, irrelevant images are effectively filtered out, ensuring that the target images are concentrated and of high quality, which greatly improves the accuracy and efficiency of subsequent recognition and annotation.

[0064] Furthermore, the steps for identifying automotive parts from the target image set and locating key exterior components of the vehicle specifically include:

[0065] Image segmentation is performed on the effective image data in the target image set to obtain vehicle information;

[0066] Identify vehicle exterior components based on vehicle information and obtain component boundary information of the vehicle exterior components;

[0067] 3D models of vehicle exterior components are simulated and constructed based on component boundary information;

[0068] Match the 3D models of the vehicle's exterior components with the preset 3D model of the whole vehicle;

[0069] Based on the 3D model matching results, the key exterior components of the vehicle are identified, and the key exterior components of the vehicle are located in the whole vehicle 3D model.

[0070] In this embodiment, the system first applies a deep segmentation network (such as Mask R-CNN, DeepLab v3+, or Segment Anything) to each valid image in the target image set to perform pixel-level segmentation of the vehicle region, extracting vehicle information such as body contours, local structures, and color textures. Then, based on the segmented vehicle information, a specially trained component detection algorithm is invoked to identify exterior components such as the hood, front and rear doors, fenders, headlights, rearview mirrors, and windshield, and calculate the bounding boxes or polygon masks of these components in the 2D image. Next, using multi-view matching and depth estimation methods (such as reconstruction algorithms based on Structure-from-Motion or monocular depth estimation), 3D point cloud inference and coordinate fitting are performed on the boundaries of each component to simulate and construct a 3D model of the vehicle's exterior components. The system then performs geometric matching between this 3D component model and a pre-constructed standard full-vehicle 3D model, using the ICP (Iterative ClosestPoint) algorithm or a feature descriptor-based matching method to align the component positions with the coordinate system of the full-vehicle model. Finally, based on the matching results, the precise location of key exterior components in the three-dimensional space of the vehicle is determined, and the corresponding component ID and spatial parameters are recorded.

[0071] Through the above steps, the system can accurately locate key exterior components of a vehicle in three-dimensional space, achieving higher-dimensional component identification and alignment, and providing a coordinate reference for damage analysis.

[0072] Furthermore, the steps of locating damaged automotive components based on key exterior parts and creating partial screenshots of these components to obtain a component damage atlas specifically include:

[0073] Identify damaged vehicle components based on vehicle information;

[0074] Based on the location of key exterior components, the location of damaged components in the vehicle's 3D model is determined to obtain the first location.

[0075] Based on the first location, the location of the damaged vehicle component is identified in each image of the target image set to obtain the second location;

[0076] In each image of the target image set, local cropping of the damaged parts of the car is performed to obtain a preliminary damage image set;

[0077] Based on the second position, the preliminary damage map is stitched together to obtain the component damage map.

[0078] In this embodiment, after completing the 3D localization of key exterior components, the system further combines vehicle information and a historical damage assessment knowledge base to perform fine-grained detection of the key component area using a deep convolutional network or a visual transformer (such as the Swing Transformer) to identify specific damaged components, such as dents on the edge of a door, scratches on the hood surface, or cracks in the rear bumper. The identified damaged components are mapped back to the previously generated 3D model of the entire vehicle. Using the spatial coordinates of the key exterior components, the first position of the damaged component is determined in the 3D coordinate system. The system then projects this 3D position back to each 2D image in the target image set, using feature point matching and projection matrix calculation to obtain the corresponding second position, ensuring consistent localization across different shooting angles. Based on the second position in each image, the system automatically crops out local screenshots containing the damaged area, forming a preliminary damage atlas. To improve data integrity and visual detail, the system uses image stitching and fusion algorithms (such as multi-scale pyramid-based stitching technology or optical flow alignment methods) to spatially align and integrate the preliminary damage atlas, ultimately generating a clear and unified component damage atlas.

[0079] Through the above steps, the system can automatically aggregate damage details from different perspectives to form a unified and complete component damage atlas, providing data support for damage level identification and labeling.

[0080] Furthermore, the standard component damage map includes a first standard image and a second standard image. The first standard image is used to identify the component damage category, and the second standard image is used to identify the component damage degree. The step of matching the component damage map set with the standard component damage maps in the preset damaged component library and generating component damage information based on the image matching result specifically includes:

[0081] Components are simulated and repaired based on component damage atlases to obtain simulated repair images;

[0082] Calculate the similarity between the simulated restored image and the first standard image to obtain the first similarity score;

[0083] The component damage category is determined based on the first similarity.

[0084] Calculate the similarity between the component damage atlas and the second standard image to obtain the second similarity score;

[0085] The degree of component damage is determined based on the second similarity score;

[0086] Damage information for components is obtained based on the type and extent of component damage.

[0087] In this embodiment, after acquiring the component damage atlas, the system first performs simulated repair processing on the damaged areas using a built-in image inpainting module. This module can employ deep generative network-based image inpainting methods (such as GAN-based inpainting, Partial Convolution, or Transformer-based inpainting) to automatically complete missing or damaged areas while preserving local texture and lighting consistency, generating a simulated repair image that closely resembles the original intact component. Subsequently, the system performs feature comparison between this simulated repair image and a first standard image in a preset damaged component library. Similarity calculation uses methods such as multi-layer convolutional feature cosine similarity, perceptual hash matching, or vector distance obtained through metric learning to obtain a first similarity value. This first similarity value is then determined by combining it with a threshold or classification model to identify the component damage category, such as scratches, dents, cracks, or breakage. Next, the system matches the component damage atlas with a second standard image. Similarity calculation is also based on deep feature similarity or multi-scale structural similarity (MS-SSIM) to quantify the matching degree of damage at different levels (mild, moderate, severe) to obtain a second similarity value. By fusing the two similarity results and inferring rules, the final output is the comprehensive damage information of the component, including a clear damage category label and the corresponding damage level.

[0088] Through the above steps, the system can accurately identify the type and severity of component damage, achieve multi-dimensional damage information extraction, and improve the professionalism and consistency of image annotation.

[0089] Furthermore, the step of annotating the target image set based on the component damage atlas and component damage information to obtain the annotated image set specifically includes:

[0090] Identify the boundary information of damaged automotive components in the target image set, and mark the location of the damaged components based on the boundary information to obtain damage location labels;

[0091] The damage location labels are associated and mapped with the component damage category and the component damage degree, respectively;

[0092] Based on the association mapping results, damage category labels and damage severity labels are generated;

[0093] The target image set is labeled using damage category labels, damage location labels, and damage severity labels to obtain a labeled image set.

[0094] In this embodiment, after acquiring the component damage atlas and corresponding damage information, the system performs further spatial-level annotation processing on the target image set. First, based on the previously identified key vehicle exterior components and their corresponding damage locations, pixel-level edge detection and polygon contour fitting techniques (such as binary masks based on Mask R-CNN output or GrabCut optimization) are used to accurately extract the boundary information of the damaged components in each image, thereby determining the coordinate range of each damaged area in the image. Next, the system associates the damage location label with the previously obtained component damage category and damage severity information, establishing an index relationship between damage location and damage semantics in the database to ensure rapid matching of the corresponding damage metadata during retrieval and application. Subsequently, the system automatically generates a composite annotation data structure containing "damage category labels" (such as scratches, dents, breaks, etc.) and "damage severity labels" (such as mild, moderate, severe) based on the association mapping results. Finally, the system overlays the damage category labels, damage location labels, and damage severity labels onto the original target image through image rendering and metadata writing, forming a standardized and structured annotated image set.

[0095] Through the above steps, the system achieves multi-dimensional and accurate annotation of damaged vehicle parts, generating a high-quality annotated image set, which facilitates model training and data sharing.

[0096] Furthermore, the step of associating damage location labels with component damage categories and component damage degrees specifically includes:

[0097] Based on a pre-established mapping rule base, the damage location labels are initially matched with the component damage categories to obtain the first association result;

[0098] For the first association result, if the matched component category is consistent with the preset category library, the first association result is retained;

[0099] Based on the mapping rule base, the damage location labels are initially matched with the damage degree of the component to obtain the second association result;

[0100] For the second association result, if the degree of damage to the matched component is consistent with the preset damage degree library, the second association result is retained;

[0101] Obtain the first association result and the second association result to get the association mapping result.

[0102] In this embodiment, after obtaining the damage location label, the system calls an internally pre-established mapping rule base. This rule base is constructed from historical damage assessment data, manual review experience, and standardized component classification specifications, containing rich mapping relationships between component categories and damage levels. First, the system performs semantic and spatial feature matching on the damage location label based on this rule base. By calculating the correspondence between the damage location coordinates and the known component structural locations, a preliminary component category matching result, i.e., the first association result, is generated. Subsequently, the system compares and verifies the component categories in the first association result with a preset standard category library. If the matching result does not exist in the standard library or is inconsistent with the specifications, it will be automatically filtered or marked as requiring manual review. Only categories that conform to the standard library definition are retained. Next, the system uses the same matching mechanism to associate and map the damage location label with the damage level. Combining features such as pixel density of the damaged area, texture damage degree, and edge continuity, it compares this with the damage level template in the rule base to obtain the second association result. A consistency check is performed with the preset damage level library, and only mapping results consistent with the standard level are retained. Finally, the system merges the first and second association results, which have undergone two verifications, to form the final association mapping result.

[0103] Through the above steps, the system can ensure that the correspondence between location, category, and degree is accurate and reliable, significantly improving the consistency and usability of the labeled data.

[0104] In the above embodiments, this application discloses an automated image annotation method, belonging to the field of artificial intelligence technology, applied to the processing of vehicle insurance damage assessment images. This application filters and classifies the original image data, removing irrelevant information to ensure that processing focuses on valid images containing the vehicle, reducing redundant computation. Subsequently, deep learning segmentation and recognition technology is used to accurately locate key exterior components of the vehicle, and specific damaged components are located by combining 3D model matching, achieving refined component-level analysis. Based on this, damaged areas are extracted through local screenshots, and image matching is performed using a preset standard component damage library to automatically identify damage categories and degrees, generating structured damage information. Finally, the damage location, category, and degree labels are integrated with the target image set for unified annotation output, forming a high-quality annotated image set. This application can achieve efficient and unified automatic annotation of massive amounts of vehicle accident images in a short time, significantly improving image annotation efficiency and consistency.

[0105] In this embodiment, the automated image annotation method runs on an electronic device (e.g., Figure 1The server shown can receive instructions or acquire data via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G connections, Wi-Fi connections, Bluetooth connections, Wi-Fi connections, Zigbee connections, UWB (ultra-wideband) connections, and other currently known or future wireless connection methods.

[0106] It should be emphasized that, in order to further ensure the privacy and security of the aforementioned original image information, the original image information can also be stored in a blockchain node.

[0107] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0108] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0109] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0110] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0111] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0112] Further reference Figure 3 As a response to the above Figure 2 The present application provides an embodiment of an automated image annotation device to implement the method shown. This device embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0113] like Figure 3 As shown, the automated image annotation device 300 described in this embodiment includes:

[0114] The data acquisition module 301 is used to acquire pre-collected raw image data, wherein the raw image data is accident record images related to vehicles;

[0115] The data filtering module 302 is used to filter the original image data to obtain a target image set, wherein the target image set is a collection of image data containing vehicle information;

[0116] The component recognition module 303 is used to identify automotive components from a target image set and locate key exterior parts of the vehicle.

[0117] The damaged component identification module 304 is used to locate the damaged components of the car based on the key appearance components, and to take local screenshots of the damaged components to obtain a component damage atlas.

[0118] The damage information recognition module 305 is used to perform image matching between the component damage image set and the standard component damage image in the preset damaged component library, and generate component damage information based on the image matching result.

[0119] The image annotation module 306 is used to annotate the target image set according to the component damage atlas and component damage information to obtain an annotated image set.

[0120] Furthermore, the data filtering module 302 specifically includes:

[0121] The entity recognition unit is used to perform target entity recognition on the original image data to obtain the target entity recognition result.

[0122] The image classification unit is used to classify the original image data based on the target entity recognition result using a pre-trained image classification model, and to determine whether each image in the original image data contains vehicle information.

[0123] The data filtering unit is used to determine that if the image contains vehicle information, it is valid image data and adds the valid image data to a preset image set to obtain the target image set.

[0124] Furthermore, the component identification module 303 specifically includes:

[0125] The image segmentation unit is used to segment the effective image data in the target image set to obtain vehicle information;

[0126] The first boundary recognition unit is used to recognize vehicle exterior components based on vehicle information and obtain component boundary information of the vehicle exterior components.

[0127] 3D model building unit, used to simulate and build 3D models of vehicle exterior components based on component boundary information;

[0128] The 3D model matching unit is used to match the 3D models of vehicle exterior components with the preset 3D models of the whole vehicle.

[0129] The component identification unit is used to determine the key exterior components of the vehicle based on the 3D model matching results, and to locate the key exterior components of the vehicle in the whole vehicle 3D model.

[0130] Furthermore, the damaged component identification module 304 specifically includes:

[0131] Damaged component identification unit, used to identify damaged components of a vehicle based on vehicle information;

[0132] The first position unit is used to locate the position of the damaged parts of the vehicle in the 3D model of the whole vehicle based on the position of the key appearance parts, and obtain the first position;

[0133] The second position unit is used to identify the location of the damaged vehicle component in each image of the target image set based on the first position, and obtain the second position.

[0134] The local screenshot unit is used to perform local screenshots of the damaged parts of the car in each image of the target image set, so as to obtain a preliminary damage image set.

[0135] The atlas splicing unit is used to splice the preliminary damage atlas according to the second position to obtain the component damage atlas.

[0136] Furthermore, the standard component damage diagram includes a first standard image and a second standard image. The first standard image is used to identify the component damage category, and the second standard image is used to identify the component damage degree. The damage information identification module 305 specifically includes:

[0137] The simulation repair unit is used to simulate the repair of components based on component damage atlases to obtain simulated repair images;

[0138] The first similarity calculation unit is used to calculate the similarity between the simulated restored image and the first standard image to obtain the first similarity.

[0139] Damage category identification unit, used to determine the damage category of the component based on a first similarity;

[0140] The second similarity calculation unit is used to calculate the similarity between the component damage atlas and the second standard image to obtain the second similarity.

[0141] A damage degree identification unit is used to determine the degree of component damage based on a second similarity.

[0142] The damage information aggregation unit is used to obtain component damage information based on the component damage category and the degree of component damage.

[0143] Furthermore, the image annotation module 306 specifically includes:

[0144] The second boundary recognition unit is used to recognize the boundary information of the damaged parts of the car in the target image set, and to mark the location of the damaged parts according to the boundary information of the damaged parts to obtain the damage location label.

[0145] The association mapping unit is used to associate and map the damage location label with the component damage category and the component damage degree, respectively.

[0146] The label generation unit is used to generate damage category labels and damage severity labels based on the association mapping results;

[0147] The image annotation unit is used to annotate the target image set using damage category labels, damage location labels, and damage degree labels to obtain an annotated image set.

[0148] Furthermore, the association mapping unit specifically includes:

[0149] The first association subunit is used to perform preliminary matching between damage location labels and component damage categories based on a pre-established mapping rule base to obtain the first association result;

[0150] The damage category determination subunit is used to retain the first association result if the matched component category is consistent with the preset category library.

[0151] The second association subunit is used to perform a preliminary match between the damage location label and the damage degree of the component based on the mapping rule base to obtain the second association result;

[0152] The damage degree judgment subunit is used to determine the second association result. If the damage degree of the matched component is consistent with the preset damage degree library, the second association result is retained.

[0153] The association mapping subunit is used to obtain the first association result and the second association result to obtain the association mapping result.

[0154] In the above embodiments, this application discloses an automated image annotation device, belonging to the field of artificial intelligence technology, applied to the processing of vehicle insurance damage assessment images. This application filters and classifies the original image data, removing irrelevant information to ensure that processing focuses on valid images that actually contain the vehicle, reducing redundant computation. Subsequently, deep learning segmentation and recognition technology is used to accurately locate key exterior components of the vehicle, and specific damaged components are located by combining 3D model matching, achieving refined component-level analysis. Based on this, damaged areas are extracted through local screenshots, and image matching is performed using a preset standard component damage library to automatically identify damage categories and degrees, generating structured damage information. Finally, the damage location, category, and degree labels are integrated with the target image set for unified annotation output, forming a high-quality annotated image set. This application can achieve efficient and unified automatic annotation of massive amounts of car accident images in a short time, significantly improving image annotation efficiency and consistency.

[0155] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0156] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with memory 41, processor 42, and network interface 43 is shown in the figure; however, it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0157] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0158] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for automated image annotation methods. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0159] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions for the automated image annotation method.

[0160] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0161] This application also provides an implementation method, namely, a computer device including a memory and a processor. The memory stores computer-readable instructions, and the processor, when executing the computer-readable instructions, implements the steps of the automated image annotation method described above, that is, implements:

[0162] An automated image annotation method, comprising:

[0163] Acquire pre-collected raw image data, which consists of accident record images related to vehicles;

[0164] The original image data is filtered to obtain the target image set, which is a collection of image data containing vehicle information;

[0165] Identify automotive parts from a target image set and locate key exterior components of the vehicle;

[0166] Based on the key exterior components, locate the damaged parts of the car, and take partial screenshots of the damaged parts to obtain a damage atlas of the parts;

[0167] The component damage atlas is matched with the standard component damage atlas in the preset damaged component library, and component damage information is generated based on the image matching results.

[0168] Based on the component damage atlas and component damage information, the target image set is annotated to obtain the annotated image set.

[0169] This application also provides another embodiment, namely, a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the automated image annotation method described above, i.e., to implement:

[0170] An automated image annotation method, comprising:

[0171] Acquire pre-collected raw image data, which consists of accident record images related to vehicles;

[0172] The original image data is filtered to obtain the target image set, which is a collection of image data containing vehicle information;

[0173] Identify automotive parts from a target image set and locate key exterior components of the vehicle;

[0174] Based on the key exterior components, locate the damaged parts of the car, and take partial screenshots of the damaged parts to obtain a damage atlas of the parts;

[0175] The component damage atlas is matched with the standard component damage atlas in the preset damaged component library, and component damage information is generated based on the image matching results.

[0176] Based on the component damage atlas and component damage information, the target image set is annotated to obtain the annotated image set.

[0177] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0178] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0179] It should be noted that the software tools or components not belonging to this company that appear in the various embodiments of this application are merely illustrative examples and do not represent actual use.

[0180] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. An automated image annotation method, characterized in that, include: Acquire pre-collected raw image data, wherein the raw image data is accident record images related to vehicles; The original image data is filtered to obtain a target image set, wherein the target image set is a collection of image data containing vehicle information; The target image set is used to identify automotive parts and locate key exterior components of the vehicle. Based on the key exterior components, locate the damaged parts of the vehicle, and take partial screenshots of the damaged parts to obtain a damage atlas of the parts; The component damage atlas is matched with the standard component damage atlas in the preset damaged component library, and component damage information is generated based on the image matching results. Based on the component damage atlas and the component damage information, the target image set is annotated to obtain an annotated image set.

2. The automated image annotation method as described in claim 1, characterized in that, The step of filtering the original image data to obtain the target image set specifically includes: The original image data is subjected to target entity recognition to obtain the target entity recognition result; Based on the target entity recognition result, a pre-trained image classification model is used to classify the original image data to determine whether each image in the original image data contains the vehicle information; If the vehicle information is included, it is determined to be valid image data, and the valid image data is added to a preset image set to obtain the target image set.

3. The automated image annotation method as described in claim 2, characterized in that, The step of identifying automotive parts from the target image set and locating key exterior components of the vehicle specifically includes: Image segmentation is performed on the valid image data in the target image set to obtain vehicle information; Based on the vehicle information, identify vehicle exterior components and obtain component boundary information of the vehicle exterior components; Based on the component boundary information, a 3D model of the vehicle exterior component is simulated and constructed; The 3D model of the vehicle exterior component is matched with the preset 3D model of the whole vehicle; The key exterior components of the vehicle are determined based on the 3D model matching results, and the key exterior components of the vehicle are located in the whole vehicle 3D model.

4. The automated image annotation method as described in claim 3, characterized in that, The step of locating damaged automotive components based on the key exterior components and taking partial screenshots of the damaged components to obtain a component damage atlas specifically includes: Based on the vehicle information, identify damaged vehicle components; Based on the location of the key appearance components, the location of the damaged components of the vehicle is located in the 3D model of the vehicle to obtain the first location; Based on the first location, the location of the damaged vehicle component is identified in each image of the target image set to obtain the second location; In each image of the target image set, a partial crop of the damaged vehicle component is taken; a preliminary damage image set is obtained. Based on the second location, the preliminary damage map is stitched together to obtain the component damage map.

5. The automated image annotation method as described in claim 1, characterized in that, The standard component damage map includes a first standard image and a second standard image. The first standard image is used to identify the component damage category, and the second standard image is used to identify the component damage degree. The step of matching the component damage map set with the standard component damage maps in a preset damaged component library and generating component damage information based on the image matching result specifically includes: Based on the component damage atlas, simulated repair of the component is performed to obtain simulated repair images; Calculate the similarity between the simulated restored image and the first standard image to obtain a first similarity score; The component damage category is determined based on the first similarity. Calculate the similarity between the component damage atlas and the second standard image to obtain the second similarity. The degree of component damage is determined based on the second similarity. The component damage information is obtained based on the component damage category and the component damage degree.

6. The automated image annotation method as described in claim 5, characterized in that, The step of annotating the target image set according to the component damage atlas and the component damage information to obtain an annotated image set specifically includes: Identify the boundary information of damaged vehicle components in the target image set, and mark the location of the damaged components based on the boundary information to obtain damage location labels; The damage location labels are associated and mapped with the component damage category and the component damage degree, respectively; Based on the association mapping results, damage category labels and damage severity labels are generated; The target image set is labeled using the damage category label, the damage location label, and the damage severity label to obtain a labeled image set.

7. The automated image annotation method as described in claim 6, characterized in that, The step of associating and mapping the damage location labels with the component damage category and the component damage degree specifically includes: Based on a pre-established mapping rule library, the damage location label is initially matched with the component damage category to obtain a first association result; If the matched component category matches the preset category library, the first association result is retained. Based on the mapping rule base, the damage location label is initially matched with the damage degree of the component to obtain a second association result; If the degree of damage to the matched component matches the preset damage degree library, then the second association result is retained. Obtain the first association result and the second association result to obtain the association mapping result.

8. An automated image annotation device, characterized in that, include: The data acquisition module is used to acquire pre-collected raw image data, wherein the raw image data is accident record images related to vehicles; The data filtering module is used to filter the original image data to obtain a target image set, wherein the target image set is a collection of image data containing vehicle information; The component identification module is used to identify automotive components from the target image set and locate key exterior parts of the vehicle. The damaged component identification module is used to locate the damaged components of the vehicle based on the key appearance components, and to take local screenshots of the damaged components to obtain a component damage atlas. The damage information recognition module is used to perform image matching between the component damage image set and the standard component damage image in the preset damaged component library, and generate component damage information based on the image matching result. The image annotation module is used to annotate the target image set according to the component damage atlas and the component damage information to obtain an annotated image set.

9. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the automated image annotation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the automated image annotation method as described in any one of claims 1 to 7.

Images