An automatic identification method of vehicle submerged parts based on image
By constructing an image processing model, automatically segmenting the water surface and the vehicle, detecting the external components of the vehicle, and identifying the flooded parts, the problem of automatic judgment of the vehicle flooded parts in the prior art is solved, and efficient and economical identification of the vehicle flooded parts is achieved.
Patent Information
- Application Number
- CN202510013412.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-06
- Publication Date
- 2025-05-13
- Estimated Expiration
- 2045-01-06
AI Technical Summary
The prior art is difficult to automatically determine the vehicle's flooded parts, resulting in damage to vehicles and trapped people, and is expensive, making it difficult to apply to large-scale and large-scale vehicle flooded parts identification tasks.
By constructing a water surface and vehicle instance segmentation model, a vehicle external component object detection model and a vehicle submerged part discrimination model, the water surface and vehicle are automatically divided using image data, detecting vehicle external components, and determining the vehicle submerged part based on confidence.
It realizes automatic discrimination of vehicle flooded parts, reduces equipment costs, and is suitable for large-scale and large-scale vehicle flooded parts identification application scenarios, reducing losses and harms.
Smart Images
Figure CN119399479B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer vision technology, and relates to flood monitoring based on images, deep learning and machine learning, and specifically to an image-based automatic identification method for a vehicle submerged location. Background Art
[0002] Severe urban floods often cause a large number of vehicles to be submerged in water, causing damage to vehicles and trapping people. The identification of submerged vehicle parts is of great reference value for emergency rescue, evasive measures and vehicle damage assessment. At present, the identification of submerged vehicle parts relies on the visual interpretation of observers. This method is time-consuming, labor-intensive, and costly, and is difficult to apply to large-scale and large-scale vehicle submerged part identification task scenarios. Therefore, it is of great significance to develop an automatic identification method for submerged vehicle parts.
[0003] Video surveillance cameras in every corner of the city and widely used smartphone cameras can record various dynamics of the city in the form of images, including various dynamics in urban flood scenes. Images of urban flood scenes contain information about the submerged parts of vehicles, providing data conditions for automatic identification of submerged parts of vehicles. At present, there is still a lack of automatic identification methods of submerged parts of vehicles based on images, which limits the promotion and application of vehicle submerged part identification, and is very unfavorable to reduce and reduce the harm of urban floods. Summary of the invention
[0004] In view of the shortcomings of the existing methods, the present invention proposes an image-based automatic identification method for the submerged part of a vehicle, which segments the water surface and the vehicle from the image, selects the underwater vehicle according to the water surface mask and the vehicle mask, performs target detection on the external parts of the vehicle in the mask area range image of the underwater vehicle, calculates the target detection confidence of various external parts of the vehicle, and identifies the submerged part of the vehicle according to the target detection confidence, thereby realizing automatic identification of the submerged part of the vehicle based on the image.
[0005] An automatic identification method of vehicle submerged parts based on an image, the specific steps are as follows:
[0006] Step 1: Build a water surface and vehicle instance segmentation model
[0007] A batch of urban flood images containing water surfaces and vehicles are collected, and the images are data enhanced to expand the image scale; the water surface and vehicles in the images are instance segmented and labeled to generate a water surface and vehicle instance segmentation and annotation dataset; the water surface and vehicle instance segmentation and annotation dataset is used to train a deep learning-based instance segmentation pre-training model to obtain a water surface and vehicle instance segmentation model.
[0008] Step 2: Build a vehicle external parts target detection model
[0009] Collect a batch of images containing vehicle external parts, perform data enhancement on the images, and expand the image scale; perform target detection and annotation on the vehicle external parts in the images to generate a vehicle external parts target detection and annotation dataset; use the vehicle external parts target detection and annotation dataset to train a deep learning-based target detection pre-training model to obtain a vehicle external parts target detection model.
[0010] Step 3: Construct a vehicle submerged location discrimination model
[0011] Another batch of urban flood images containing water surfaces and vehicles are collected, and vehicles that are not in the water are cropped out from the images to ensure that all vehicles in all images are in the water. The images are then data enhanced to expand the image size.
[0012] The water surface and vehicle instance segmentation model is used to segment the vehicle from the image and obtain the vehicle mask; the image and the vehicle mask are bitwise ANDed to extract the vehicle mask area range image.
[0013] The vehicle external parts target detection model is used to detect the vehicle external parts from the vehicle mask area range image, and the target detection confidence of each type of external parts is calculated as the sample feature. The submerged part of the vehicle is determined by manual visual interpretation as the sample label to generate a feature-label dataset of the vehicle submerged part.
[0014] The feature-label dataset of the vehicle submerged parts is used to train the machine learning classification model to obtain the vehicle submerged parts discrimination model.
[0015] Step 4: Read urban flood image data
[0016] The urban flood image data is read from the urban flood scene video captured by a video surveillance camera or a smartphone camera.
[0017] Step 5: Instance segmentation of water surface and vehicle
[0018] The water surface and vehicle instance segmentation model is used to segment the water surface and vehicles from the urban flood image to obtain the water surface mask and vehicle mask.
[0019] Step 6: Use the water surface mask and vehicle mask to select the underwater vehicle
[0020] For each vehicle, get the bounding box of its mask. If the lower left corner or lower right corner of the bounding box is within the water surface mask area, it means that the vehicle is in the water, that is, it is an underwater vehicle. Otherwise, it means that the vehicle is not in the water. Get the underwater vehicle mask.
[0021] Step 7: Get the underwater vehicle mask area range image
[0022] The urban flood image and the underwater vehicle mask are bitwise ANDed to extract the underwater vehicle mask area range image.
[0023] Step 8: Target detection of vehicle external parts
[0024] The vehicle external parts target detection model is used to detect the vehicle's external parts from the underwater vehicle mask area range image, and the target detection confidence of various types of vehicle external parts is calculated.
[0025] Step 9: Identify the submerged part of the vehicle
[0026] The target detection confidence of various external parts of the underwater vehicle is input into the vehicle submerged part discrimination model, and the submerged part of the underwater vehicle is obtained from the model output, thereby realizing automatic discrimination of the vehicle submerged part based on images.
[0027] The present invention has the following beneficial effects:
[0028] The present invention can make full use of the surveillance cameras deployed in the city or the videos uploaded on social media as the image data source, greatly reducing the equipment cost for identifying the submerged part of the vehicle. The present invention realizes the automatic identification of the submerged part of the vehicle, overcomes the disadvantages of manual visual identification, and can be applied to the large-scale and large-scale application scenarios of vehicle submerged part identification in the city, which helps the relevant departments and car owners to timely and comprehensively understand the submerged status of the vehicle, provide reference for emergency rescue, vehicle transfer and vehicle damage assessment, and reduce and lower the losses. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 A flowchart of an automatic identification method of a submerged vehicle location based on an image;
[0030] Figure 2 Schematic diagram of water surface and vehicle instance segmentation;
[0031] Figure 3 A schematic diagram for obtaining an image of the mask area of a vehicle in water;
[0032] Figure 4 A schematic diagram of target detection for vehicle external parts;
[0033] Figure 5 Schematic diagram for identifying the submerged part of a vehicle. DETAILED DESCRIPTION
[0034] The present invention will be further explained below with reference to the accompanying drawings;
[0035] like Figure 1 As shown, a method for automatically distinguishing the submerged part of a vehicle based on an image specifically comprises the following steps:
[0036] Step 1: Build a water surface and vehicle instance segmentation model
[0037] s1.1. Collect a batch of urban flood images with different weather conditions, different image quality, and containing different types of vehicles and water surfaces, and perform data enhancement on the images, including horizontal flipping, color conversion, scaling, and noise injection, to expand the scale of urban flood images.
[0038] s1.2. Use the X-Anylabeling tool to annotate the contours of the water surface and vehicles in the urban flood image in a polygonal manner, save the annotated data in YOLO format, and generate a water surface and vehicle instance segmentation and annotation dataset in the urban flood scene.
[0039] s1.3. Use the water surface and vehicle instance segmentation annotation dataset to train the instance segmentation pre-training model of YOLOv11 to obtain the water surface and vehicle instance segmentation model in urban flood scenarios.
[0040] Step 2: Build a vehicle external parts target detection model
[0041] s2.1. Collect a batch of images with different weather conditions, different image qualities, and different types of vehicle external parts, and perform data enhancement on the images, including horizontal flipping, color conversion, scaling, and noise injection, to expand the image scale.
[0042] s2.2. Use the X-Anylabeling annotation tool to annotate the bounding boxes of each external part of the vehicle from the above image in a rectangular manner, save the annotation data in YOLO format, and generate a vehicle external part target detection annotation dataset. The vehicle external parts are divided into 26 categories, including: headlights, front fog lights, rear taillights, rear bumper lights, front windshields, rear windshields, windows, engine covers, front fenders, rear fenders, front bumpers, rear bumpers, front license plates, rear license plates, exterior mirrors, door handles, exhaust ports, tires, wheels, sunroofs, roofs, trunk lids, front doors, rear doors, center grilles, and lower grilles of front bumpers.
[0043] s2.3. Use the vehicle external parts target detection annotation dataset to train the YOLOv11 target detection pre-training model to obtain the vehicle external parts target detection model.
[0044] Step 3: Construct a vehicle submerged location discrimination model
[0045] s3.1. Collect another batch of urban flood images with different weather conditions, different image quality, and containing different types of vehicles and water surfaces. Crop the vehicles that are not in the water from the images to ensure that all vehicles in all images are in the water. Perform data enhancement on the images, including horizontal flipping, color conversion, scaling, and noise injection, to expand the scale of urban flood images.
[0046] s3.2. Use the water surface and vehicle instance segmentation model to segment each vehicle from the image, obtain the mask of each vehicle, perform a bitwise AND operation between the image and each vehicle mask, and extract the image of the mask area range of each vehicle.
[0047] s3.3. For each vehicle, the vehicle external component target detection model is used to detect the external components of the vehicle from the vehicle mask area range image, and the target detection confidence of each external component output by the target detection model is obtained.
[0048] For each external component category of the vehicle, if no external component of this category is detected, the target detection confidence of this category of external components is set to 0; if only one external component of this category is detected, the target detection confidence of this external component is used as the target detection confidence of this category of external components; if two or more external components of this category are detected, the average target detection confidence of all detected external components of this category is used as the target detection confidence of this category of external components.
[0049] The submerged parts of the vehicle are designated by manual visual interpretation, and the submerged parts of the vehicle specifically include: tires, the area from the top of the tires to the bottom of the windows, windows, roof, front windshield, engine cover, headlights / grille, front bumper, front fog lights, below the front fog lights, rear windshield, top of the trunk lid, rear taillights, rear bumper, rear bumper lights, rear exhaust pipe, and below the rear exhaust pipe.
[0050] s3.4. For each vehicle, the object detection confidence of the 26 types of external parts of the vehicle obtained in s3.3 is sorted based on the fixed order of the external parts categories, and the sorted confidence is used as a 26-dimensional sample feature, and the submerged part of the vehicle specified in s3.3 is used as a sample label. The sample features and sample labels are used to generate a feature-label dataset of the submerged parts of the vehicle.
[0051] s3.5. Use the feature-label dataset of the vehicle submerged location to train the random forest classification model and obtain a vehicle submerged location discrimination model based on random forest.
[0052] Step 4: Read urban flood image data
[0053] The urban flood image data is read from the urban flood scene video captured by a video surveillance camera or a smartphone camera.
[0054] Step 5: Instance segmentation of water surface and vehicle
[0055] The water surface and vehicle instance segmentation model is used to segment the water surface and vehicles from the urban flood image to obtain the water surface mask and vehicle mask, as shown in Figure 2 shown.
[0056] Step 6: Use the water surface mask and vehicle mask to select the underwater vehicle
[0057] For each vehicle, get the bounding box of its mask. If the lower left corner or lower right corner of the bounding box is within the water surface mask area, it means that the vehicle is in the water, that is, it is a submerged vehicle. Otherwise, it means that the vehicle is not in the water. Get the mask of the submerged vehicle.
[0058] Step 7: Get the mask area range image of the vehicle in the water
[0059] For each vehicle in the water, the read urban flood image is bitwise ANDed with the mask of the vehicle in the water to extract the mask area range image of the vehicle in the water, such as Figure 3 shown.
[0060] Step 8: Target detection of vehicle external parts
[0061] For each underwater vehicle, the mask area range image of the underwater vehicle is input into the vehicle external component target detection model, and the external components in the mask area range image of the vehicle are detected, such as Figure 4 As shown in FIG. 1 , the confidence level of each type of external component target detection in the vehicle mask area range image is calculated as the discriminant feature of the submerged part of the vehicle in the water.
[0062] Step 9: Identify the submerged part of the vehicle
[0063] For each vehicle in the water, the discriminant features of the submerged part of the vehicle are input into the vehicle submerged part discrimination model, and the submerged part of the vehicle is obtained from the model output, such as Figure 5 shown.
[0064] The present invention is not limited to the above specific implementations, and various modifications and changes can be made without departing from the technical essence of the present invention. Any modification, equivalent replacement or improvement based on the technical principle of the present invention shall be regarded as the protection scope of the present invention.
Claims
1. An automatic identification method of vehicle submerged parts based on images, characterized in that: The specific steps include: Step 1: Generate a water surface and vehicle instance segmentation annotation dataset and build a water surface and vehicle instance segmentation model; Step 2: Generate a vehicle external parts target detection annotation dataset and build a vehicle external parts target detection model; Step 3: Generate a feature-label dataset of the submerged vehicle parts and build a vehicle submerged part discrimination model. The specific method is as follows: s3.
1. Collect another batch of urban flood images with different weather conditions, different image quality, and different types of vehicles and water surfaces. Crop the vehicles that are not in the water from the images to ensure that all vehicles in all images are in the water. Perform data enhancement on the images to expand the scale of urban flood images. s3.
2. Use the water surface and vehicle instance segmentation model to segment vehicles from the urban flood image and obtain the vehicle mask; perform a bitwise AND operation on the urban flood image and the vehicle mask to extract the vehicle mask area range image; s3.
3. Use the vehicle external parts target detection model to detect the vehicle external parts from the vehicle mask area range image, calculate the target detection confidence of each type of external parts as the sample feature, use manual visual interpretation to determine the submerged part of the vehicle as the sample label, and generate a feature-label dataset of the vehicle submerged part; s3.
4. Use the feature-label dataset of the submerged vehicle parts to train the machine learning classification model to obtain the vehicle submerged part discrimination model; Step 4: reading an urban flood image from a video of an urban flood scene captured by a video surveillance camera or a smartphone camera; Step 5: Use the water surface and vehicle instance segmentation model to segment the water surface and vehicles from the urban flood image to obtain the water surface mask and the vehicle mask; Step 6: Select the underwater vehicle according to the water surface mask and the vehicle mask, and obtain the underwater vehicle mask; Step 7: Using the underwater vehicle mask, extract the underwater vehicle mask area range image from the urban flood image; Step 8: Detect the external parts of the underwater vehicle from the underwater vehicle mask area range image using the vehicle external parts target detection model, and calculate the target detection confidence of various external parts of the underwater vehicle; Step 9: Input the target detection confidence of various external parts of the vehicle in the water into the vehicle submerged part discrimination model, obtain the submerged part of the vehicle in the water from the model output, and realize the automatic discrimination of the vehicle submerged part based on the image.
2. The method for automatically distinguishing the submerged part of a vehicle based on an image as claimed in claim 1, characterized in that: The method of generating water surface and vehicle instance segmentation annotation datasets and constructing water surface and vehicle instance segmentation models is as follows: s1.
1. Collect a batch of urban flood images with different weather conditions, different image quality, and containing different types of vehicles and water surfaces, and perform data enhancement processing on the images to expand the scale of urban flood images; the data enhancement processing includes horizontal flipping, color conversion, scaling, and noise injection; s1.
2. Perform instance segmentation and annotation on water surface and vehicles in urban flood images to generate water surface and vehicle instance segmentation and annotation datasets; s1.
3. Use the water surface and vehicle instance segmentation annotation dataset to train the deep learning-based instance segmentation pre-training model to obtain the water surface and vehicle instance segmentation model.
3. The method for automatically identifying the submerged part of a vehicle based on an image as claimed in claim 1, characterized in that: The method of generating a vehicle external parts target detection annotation dataset and building a vehicle external parts target detection model is as follows: s2.
1. Collect a batch of images of different weather conditions, different image qualities, and different types of vehicle exterior parts, and perform data enhancement on the images to expand the image scale of vehicle exterior parts; s2.2, performing target detection and annotation on the vehicle external parts in the image of the vehicle external parts, and generating a vehicle external parts target detection and annotation dataset; s2.
3. Use the vehicle external parts target detection annotation dataset to train the deep learning-based target detection pre-training model to obtain the vehicle external parts target detection model.
4. The method for automatically distinguishing the submerged position of a vehicle based on an image as claimed in claim 1 or 3, characterized in that: The vehicle's external parts include: headlights, front fog lights, rear taillights, rear bumper lights, front windshield, rear windshield, windows, engine cover, front fenders, rear fenders, front bumper, rear bumper, front license plate, rear license plate, exterior mirrors, door handles, exhaust ports, tires, wheels, sunroof, roof, trunk lid, front doors, rear doors, center grille and front bumper lower grille.
5. The method for automatically distinguishing the submerged part of a vehicle based on an image as claimed in claim 2 or 3, characterized in that: The deep learning-based instance segmentation pre-training model is an instance segmentation pre-training model of YOLOv11, and the deep learning-based target detection pre-training model is a target detection pre-training model of YOLOv11.
6. The method for automatically distinguishing the submerged part of a vehicle based on an image as claimed in claim 1, characterized in that: The method for calculating the target detection confidence of each type of external parts is as follows: for each external part category, if no external parts are detected in this category, the target detection confidence of this type of external parts is set to 0; if only one external part is detected in this category, the target detection confidence of this external part is used as the target detection confidence of this type of external parts; if two or more external parts are detected in this category, the average of the target detection confidences of all detected external parts in this category is used as the target detection confidence of this type of external parts.
7. The method for automatically identifying the submerged part of a vehicle based on an image as claimed in claim 1, characterized in that: The submerged parts of the vehicle include: tires, the area from the top of the tires to the bottom of the windows, windows, roof, front windshield, engine cover, headlights / grille, front bumper, front fog lights, below the front fog lights, rear windshield, top of the trunk lid, rear taillights, rear bumper, rear bumper lights, rear exhaust pipe, and below the rear exhaust pipe.
8. The method for automatically identifying the submerged part of a vehicle based on an image as claimed in claim 1, characterized in that: The machine learning classification model is a random forest model.
9. The method for automatically distinguishing the submerged part of a vehicle based on an image as claimed in claim 1, characterized in that: The method for selecting underwater vehicles based on the water surface mask and the vehicle mask is as follows: for each vehicle, obtain the bounding box of its mask. If the lower left corner point or the lower right corner point of the bounding box is within the water surface mask area, it means that the vehicle is in the water, that is, it is an underwater vehicle. Otherwise, it means that the vehicle is not in the water.