A target damage assessment method based on image recognition

Abstract

This invention discloses a target damage assessment method based on image recognition. Using continuous or single-frame video images, it can determine the damage status of functional components of a target and assign a damage level, offering simplicity and efficiency. It provides a method for defining target damage levels, forming multi-level definitions based on the functional components of the assessment object (target), enabling quantification of the degree of damage assessment. This method does not rely on image comparison before and after target assessment but rather on pre-existing knowledge and definitions, resulting in greater accuracy and speed. With the expansion of the knowledge base, this method exhibits excellent scalability and can be applied to assess the damage level of other targets. Operators can participate in the assessment through human-computer interaction, facilitating a faster assessment process and improving accuracy.

Description

Technical Field

[0001] This invention belongs to the field of optoelectronic imaging system technology and relates to a target damage assessment method based on image recognition. Background Technology

[0002] Target damage assessment is a crucial task in disasters such as earthquakes, floods, landslides, strong winds, fires, and multi-vehicle collisions, as well as in warfare. The degree of damage to a target significantly impacts subsequent operations. How to quickly approach the damaged object (target site) or its vicinity, promptly obtain damage information, and determine the extent of damage is a matter of great concern to government departments, rescue agencies, insurance companies, and combat troops.

[0003] Unmanned aerial vehicles (UAVs), manned aircraft, reconnaissance vehicles, or overhead remote sensing satellites can quickly reach damaged areas and capture videos or images, providing rapid data for assessing the level of damage. Utilizing these videos and images to quickly evaluate the extent of damage to the photographed objects (targets) is of paramount value in supporting subsequent action decisions.

[0004] Application No. 201811407266.2 proposes a method, apparatus, equipment, and computer-readable storage medium for assessing post-earthquake damage to physical targets. The method first extracts the vector contours of the buildings to be assessed before and after the earthquake. Based on the vector contour data and their corresponding remote sensing images, preset feature parameters are calculated to form a target feature parameter set. This target feature parameter set is then input into a damage assessment model to obtain the building damage assessment result. This method is used to determine the damage level of buildings after an earthquake. It requires using machine learning algorithms to train the feature parameters of multiple building samples to construct a damage assessment model, but the model's structure is not described.

[0005] Application No. 201410510164.9 proposes a method for assessing the physical damage of buildings based on remote sensing multispectral images. First, preprocessing multi-temporal multispectral remote sensing images before and after a disaster is performed. Then, a binary mask image is obtained using a random walk-based change detection method. Next, the pre-disaster image is subtracted using the binary mask image to detect and extract target buildings. Finally, the physical damage assessment results are given. This method does not describe the definition of damage levels or assessment results.

[0006] Application No. 202011226859.6 proposes a method, computer equipment, and computer-readable storage medium for assessing vehicle apparent damage. The method first normalizes the size of the input vehicle image. Then, it performs grayscale conversion in two paths: one path directly converts the image to grayscale to obtain a grayscale image containing both the target and background; the other path first uses a target semantic segmentation model to perform pixel-level segmentation to extract the target region, and then converts the segmented image to grayscale to obtain a grayscale image containing only the target. Next, a feature extraction model is constructed based on a convolutional neural network to extract high-dimensional features from the two grayscale images (the grayscale image containing the target and background, and the grayscale image containing only the target), obtaining high-dimensional convolutional features of the target. These two high-dimensional convolutional features are then fused. Finally, a fully connected network is used to perform regression calculations on the final fused target features, outputting a continuous value in the range of 0 to 1 as the assessment value for the degree of apparent damage to the vehicle target. This method does not require prior image registration or comparison, but it does not define the level and meaning of vehicle apparent damage.

[0007] Application No. 202011453012.1 proposes a human-machine collaborative autonomous level damage assessment method. First, it acquires images of the target before and after damage, along with preset simulation parameters, involving image registration and correction, target segmentation, and other processing. Next, it performs model simulation on the target's damage effectiveness under explosives, calculating a first damage assessment result. Then, it detects image changes in the images before and after damage, constructing a difference image. A second damage assessment result is calculated based on this difference image, and the change result is superimposed on the target's pre-damage image to receive external input auxiliary assessment results. Finally, it fuses the first, second, and auxiliary assessment results to obtain the final damage measurement result. This method combines image processing and manual judgment, and requires a target damage effectiveness model, but this model is not described.

[0008] Application No. 202110198215.9 proposes a ground target cloud model damage assessment method based on visible light image processing. The method obtains target damage images, including pre-attack images, post-attack images, binary images of the changed region, a target mask, and target coordinates. Image cropping is performed to create four cropped images. These cropped images are then overlaid. The gray-level co-occurrence matrix is ​​calculated, along with the rate of change of texture and geometric features. Finally, based on a cloud model containing expected, entropy, and hyperentropy digital features, the target damage effect is assessed, involving four levels (severe damage, moderate damage, minor damage, and no damage), for battlefield decision support. This method relies on the described "cloud model" but does not describe the actual architecture of the model.

[0009] In order to quickly and hierarchically determine the damage or destruction of an assessment object (target) and provide a level determination, it is necessary to develop a machine-automated assessment method for the damage level of a target based on video images. Summary of the Invention

[0010] (I) Purpose of the Invention

[0011] The purpose of this invention is to provide a target damage assessment method based on image recognition. This method can automatically determine the functional integrity and degree of functional loss (damage level) of the assessment object (target) by acquiring video images of the target at the time of the assessment, without relying on before-and-after image comparisons. This provides rapid information support for corresponding decision-making. By adding a database of targets and their functions, it has the ability to expand the assessment of damage to multiple targets.

[0012] (II) Technical Solution

[0013] To address the aforementioned technical problems, this invention provides a target damage assessment method based on image recognition. The method establishes a correspondence between the appearance features and functions of the target to be assessed, forming a knowledge-based related database. Targets of interest are classified using video images. Then, based on the functions and physical characteristics that a target type should possess, the method determines its functional integrity and the extent of damage, thereby automatically determining the target's damage level. The method includes the following steps:

[0014] Step 1: Define the target composition and function correspondence matrix, determine the correspondence of a certain function that will be lost when a certain component is damaged, and define the damage level corresponding to the loss of the function based on the importance of the function to the target.

[0015] Step 2: Obtain video images (continuous images or single images) of the target to be determined through sensors;

[0016] Step 3: Process the video image containing the target, and detect the target based on the database and recognition algorithm; or the operator can specify the target whose level needs to be determined.

[0017] Step 4: The software automatically determines the category of the target and obtains its corresponding component features and function list.

[0018] Step 5: Based on the actual external features and motion features of the target in the video image, automatically detect the components of its loss, thereby determining its functional loss list;

[0019] Step 6: Based on the list of functional losses, automatically determine the damage level and output the judgment result.

[0020] (III) Beneficial Effects

[0021] The target damage assessment method based on image recognition provided by the above technical solution has the following beneficial effects:

[0022] First, it can determine the damage to a target and give its damage level through continuous or single video images, which is simple and efficient.

[0023] Second, a method for defining the damage level of a target is given, which can quantify the degree of damage assessment;

[0024] Third, it does not rely on image comparison before and after target evaluation, but relies on prior knowledge and definitions to make this judgment, which is more accurate and faster;

[0025] Fourth, as the knowledge base increases, this method has good scalability and can be used to determine the degree of damage to other targets;

[0026] Fifth, operators can participate in the judgment through human-computer interaction, which facilitates faster judgment and improves accuracy. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the target damage assessment process based on image recognition.

[0028] Figure 2 This is a schematic diagram of the information processing relationships for target damage assessment based on image recognition. Detailed Implementation

[0029] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0030] Example 1

[0031] Taking the post-damage assessment of a flatbed semi-trailer truck carrying a container as an example, such as Figure 1 , 2 As shown, the target damage assessment method based on image recognition in this embodiment includes the following steps:

[0032] Step 1: Define the composition and function correspondence matrix of a flatbed trailer truck loaded with a container. Its main functional parts are the engine (providing power), the cab (providing driver support and protection), the cab tires, and the trailer body (providing cargo support). Determine the corresponding function lost when a certain component is damaged. Based on the importance of the function to the target, define the damage level corresponding to its loss.

[0033] The lower part of the cab houses the engine, which is the power source and has a crucial function. The middle part of the cab houses the driver's seat and the area for controlling the vehicle, also with a crucial function. The upper shell of the cab serves as a carrier for secondary functions, providing driver protection and rain shelter. The wheels of the cab and the trailer are carriers of important functions, essential for the vehicle's mobility. Damage to individual tires on the trailer does not affect the vehicle's mobility and constitutes a general function. The container loaded on the trailer is a carrier of general functions (involving cargo damage, not vehicle damage).

[0034] The damage level of the target is defined as five levels: complete damage if the important functions of the cab and trailer are lost; severe damage if the important functions of the cab or trailer are lost; moderate damage if individual tires, cab, or container body are lost; minor damage if individual tires, cab, or container body are lost; and no damage if there is virtually no significant equipment loss. These definitions are stored for easy access by the target damage level determination system software.

[0035] Step 2: Obtain video images (continuous images or single images) of the cab-over trailer target to be identified using sensors;

[0036] It can be visible light video images obtained from drones, ground monitoring cameras, etc., or infrared thermal imaging video images, or multispectral sensor video images, or target images obtained from SAR radar.

[0037] Step 3: Process the video images containing cab-over trailer truck targets, and detect the cab-over trailer trucks from the images based on the database and recognition algorithm;

[0038] The operator can also specify the cab-over trailer truck target in the video image to be classified.

[0039] Step 4: Automatically determine the category of the flatbed trailer truck target and obtain the corresponding list of its required components, such as the cab and trailer, as well as its mobility, load-bearing capacity, and protection functions.

[0040] It can also generate a human-computer interaction interface, allowing operators to interpret target attributes and categories, and accurately define the target category, its composition, and functions.

[0041] Step 5: Based on the actual external features and motion characteristics of the cab-over trailer target in the video image, such as whether the lower part of the cab is damaged—determine whether important functions have been lost, whether only a small part of the upper part of the cab is dented—determine whether general functions have been lost, etc.; based on the damage changes of the components detected by image processing, retrieve its functional matrix and automatically determine the list of functional losses of the cab-over trailer.

[0042] It can also generate a human-machine interface for operators to interpret, and further accurately define the loss portion of the flat-head trailer truck target;

[0043] Step 6: Based on the list of functional losses, the software automatically determines the damage level of the object being evaluated and outputs the judgment result.

[0044] Example 2

[0045] Taking the damage assessment of a main battle tank after being hit as an example, the target damage assessment method based on image recognition in this embodiment includes the following steps:

[0046] Step 1: Define the function correspondence matrix of the main battle tank. Its main functional parts are the engine compartment at the rear of the vehicle (providing power for mobility), the track running system on both sides of the lower part of the vehicle (providing the execution of mobility), the turret on the roof (providing firepower), and the hull armor (providing crew protection). Determine the corresponding function lost when a certain component is damaged. Based on the importance of the function to the target, define the damage level corresponding to its loss.

[0047] The engine compartment is located at the rear of the vehicle, which is the power source for enabling mobility and is of great importance; the hull is the area where the crew sits and works, and is the source of armor protection and is of great importance; the roof turret is the source of firepower and is of great importance; the tracked running system is the carrier of important functions and is the condition for enabling mobility and is of great importance.

[0048] The damage level of a target is defined as five levels: complete destruction (gun barrel damage (loss of firing capability, no longer a threat), engine compartment damage or track damage (loss of mobility), and front armor cracking (loss of protection); severe destruction (gun barrel damage, engine compartment damage, track damage, or front armor cracking, resulting in the loss of two of the three functions); moderate destruction (gun barrel damage, engine compartment damage, track damage, or front armor cracking, resulting in the loss of one of the three functions); minor destruction (other localized damage, such as radio antenna damage, external protective railing damage, or small-area damage to the armor plating); and no damage (no significant equipment loss). These definitions are stored for easy retrieval by the target damage level determination system software.

[0049] Step 2: Obtain video images (continuous images or single images) of the main battle tank target to be identified through sensors;

[0050] It can be visible light video images obtained from drones, ground monitoring cameras, etc., or infrared thermal imaging video images, or multispectral sensor video images, or target images obtained from SAR radar.

[0051] Step 3: Process the video images containing main battle tank targets, and detect the main battle tank targets from the images based on the database and recognition algorithm;

[0052] The operator can also specify the main battle tank target in the video image to be classified.

[0053] Step 4: Automatically determine the category of the main battle tank target and obtain its corresponding list of components, mobility, attack, protection and other functions;

[0054] It can also generate a human-computer interaction interface, allowing operators to interpret target attributes and categories, and accurately define the target category, its composition, and functions.

[0055] Step 5: Based on the actual external features and motion characteristics of the main battle tank target in the video image, such as whether the rear engine compartment is damaged—determine whether important functions have been lost, whether only a small part of the armor plating is lost—determine whether general functions have been lost, etc.; based on the damage changes of the components detected by image processing, retrieve their functional matrix and automatically determine the list of functional losses of the main battle tank.

[0056] It can also generate a human-computer interaction interface, which can be interpreted by the operator to further accurately define the loss of the main battle tank target;

[0057] Step 6: Based on the list of functional losses, the software automatically determines the target damage level of the object being evaluated and outputs the judgment result.

[0058] Example 3

[0059] Taking the damage assessment of a main battle tank after being hit as an example, the target damage assessment method based on image recognition in this embodiment includes the following steps:

[0060] Step 1: Define the function correspondence matrix of the main battle tank. Its main functional parts are the engine compartment at the rear of the vehicle (providing power for mobility), the track running system on both sides of the lower part of the vehicle (providing the execution of mobility), the turret on the roof (providing firepower), and the hull armor (providing crew protection). Determine the corresponding function lost when a certain component is damaged. Based on the importance of the function to the target, define the damage level corresponding to its loss.

[0061] The engine compartment is located at the rear of the vehicle, which is the power source for enabling mobility and is of great importance; the hull is the area where the crew sits and works, and is the source of armor protection and is of great importance; the roof turret is the source of firepower and is of great importance; the tracked running system is the carrier of important functions and is the condition for enabling mobility and is of great importance.

[0062] The damage level of the target is defined as seven levels. The damage level for all significant functional losses (damage to the engine compartment at the rear of the vehicle, damage to the tracked mobility system, damage to the gun barrel, and damage to the vehicle's armor) is "destroyed"; the damage level for multiple significant functional losses (damage to the engine compartment at the rear of the vehicle and damage to the tracked mobility system) is "disabled"; the damage level for individual significant functional losses (damage to the engine compartment at the rear of the vehicle, or damage to the tracked mobility system, or damage to the gun barrel, or damage to the vehicle's armor) is "severely damaged"; the damage level for minor functional losses (damage to the radio antenna, partial damage to the armor plating, damage to the gunner's sight, etc.) is "damaged"; the damage level for multiple general functional losses (damage to the external protective guardrail, damage to the gunner's sight, etc.) is "disrupted"; the damage level for a small number of general functional losses is "impeded"; and the damage level for virtually no functional losses is "no damage". These definitions are stored for easy retrieval by the target damage level determination system software.

[0063] Step 2: Obtain video images (continuous images or single images) of the main battle tank target to be identified through sensors;

[0064] It can be visible light video images obtained from drones, ground monitoring cameras, etc., or infrared thermal imaging video images, or multispectral sensor video images, or target images obtained from SAR radar.

[0065] Step 3: Process the video images containing main battle tank targets, and detect the main battle tank targets from the images based on the database and recognition algorithm;

[0066] The operator can also specify the main battle tank target in the video image to be classified.

[0067] Step 4: Automatically determine the category of the main battle tank target and obtain its corresponding list of components, mobility, attack, protection and other functions;

[0068] It can also generate a human-computer interaction interface, allowing operators to interpret target attributes and categories, and accurately define the target category, its composition, and functions.

[0069] Step 5: Based on the actual external features and motion characteristics of the main battle tank target in the video image, determine its functional loss; based on the damage changes of the components detected by image processing, retrieve its functional matrix and automatically determine the functional loss list of the main battle tank.

[0070] It can also generate a human-computer interaction interface, which can be interpreted by the operator to further accurately define the loss of the main battle tank target;

[0071] Step 6: Based on the list of functional losses, the software automatically determines the target damage level of the object being evaluated and outputs the judgment result.

[0072] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A target damage assessment method based on image recognition, characterized in that, Includes the following steps: Step 1: Define the target composition and function correspondence matrix, determine the correspondence of a certain function that will be lost when a certain component is damaged, and define the damage level corresponding to the loss of the function based on the importance of the function to the target. Step 2: Obtain video images of the target to be determined through sensors. The video images can be continuous images or single images. Step 3: Process the video image containing the target, and detect the target based on the database and recognition algorithm; or the operator can specify the target whose level needs to be determined. Step 4: Determine the category of the target and obtain its corresponding component features and function list; Step 5: Based on the actual external features and motion features of the target in the video image, detect the components of its loss, thereby determining its functional loss list; Step 6: Based on the list of functional losses, determine the damage level and output the determination result; In step 1, the damage level is defined using five levels: complete destruction for loss of all important functions, severe destruction for loss of one or more important functions, moderate destruction for loss of minor important functions, slight destruction for loss of general functions, and no damage for loss of virtually no functions; or, the damage level is defined using seven levels: destruction for loss of all important functions, paralysis for loss of multiple important functions, severe damage for loss of individual important functions, damage for loss of minor important functions, disruption for loss of multiple general functions, interference for loss of a few general functions, and no damage for loss of virtually no functions. In step 5, a human-computer interaction interface is generated, which is interpreted by the operator to accurately define the target loss portion; When the target is a cab-over semi-trailer truck loaded with a container, the damage level is defined as Level 5. The damage level for loss of important functions of the cab and the trailer is complete damage; the damage level for loss of important functions of the cab or the trailer is severe damage; the damage level for partial damage to individual tires of the trailer, the cab body, or the container body is moderate damage; the damage level for partial damage to individual tires of the trailer, the cab body, or the container body is minor damage; and the damage level for no significant equipment damage is no damage. When the target is a main battle tank, the damage level is defined as Level 5: Complete destruction (gun barrel damage, engine compartment damage, track damage, and front armor cracking); Severe destruction (gun barrel damage, engine compartment damage, track damage, or front armor cracking, resulting in the loss of two of the three main functions); Moderate destruction (gun barrel damage, engine compartment damage, track damage, or front armor cracking, resulting in the loss of one of the three main functions); Minor destruction (radio antenna damage, external protective railing damage, and minor damage to the armor plating); No damage (no significant equipment loss). When the target is a main battle tank, the damage level is defined as Level 7. The damage level is "Destroyed" if all important functions are lost, including damage to the engine compartment, track system, gun barrel, and hull armor at the rear of the vehicle. The damage level is "Disabled" if multiple important functions are lost, including damage to the engine compartment and track system at the rear of the vehicle. The damage level is "Severely Damaged" if only one important function is lost, including damage to the engine compartment, track system, gun barrel, or hull armor at the rear of the vehicle. The damage level is "Damaged" if secondary important functions are lost, including damage to the radio antenna, partial armor plating, and gunner's sight. The damage level is "Harassed" if multiple general functions are lost, including damage to the external protective guardrail and commander's sight. The damage level is "Interference" if a small amount of general functions are lost. The damage level is "Undamaged" if there is virtually no functional loss.

2. The target damage assessment method based on image recognition as described in claim 1, characterized in that, In step 2, the video image of the target is obtained by a visible light camera, or by infrared thermal imaging, or by a multispectral sensor, or by SAR radar.

3. The target damage assessment method based on image recognition as described in claim 2, characterized in that, In step 4, a human-computer interaction interface is generated, which is interpreted by the operator to accurately define the category of the target.

4. An application of a target damage assessment method based on image recognition according to any one of claims 1-3 in the field of optoelectronic imaging system technology.

Images