Remote sensing image partial occlusion aircraft target detection method, device, equipment and medium

Abstract

This application relates to a method, apparatus, device, and medium for detecting partially occluded aircraft targets in remote sensing images. The method extracts features from the remote sensing aircraft image under test using a feature extraction network based on a multi-task deep learning detection network. Then, an overall detection head predicts the overall bounding box and its confidence level of the aircraft; a component detection head predicts the bounding box and its confidence level of each key component of the aircraft; a dynamic mapping relationship is established between component visibility and the overall confidence level of the target; for multiple detected components, their spatial relative positions are analyzed and matched with a preset aircraft structure template to infer the complete aircraft target, effectively merging duplicate detections and eliminating erroneous detections. This method significantly improves the robustness and accuracy of detection under occlusion conditions and effectively suppresses false component detections.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, and relates to a method, device, equipment and medium for detecting partially occluded aircraft targets in remote sensing images. Background Technology

[0002] With the rapid development of remote sensing technology, high-resolution satellite and aerial imagery have become important means of Earth observation. Aircraft, as important strategic targets, have always been a research hotspot in remote sensing applications for their automatic detection and identification technology. However, in real-world scenarios, aircraft targets are often obscured by hangars, trees, clouds, etc., resulting in incomplete targets and posing a significant challenge to traditional target detection algorithms.

[0003] Existing deep learning object detection methods (such as YOLO and Faster R-CNN) typically learn and predict objects as a whole. When an object is severely occluded, its features are not fully extracted, which can easily lead to low confidence scores and thus false negatives. On the other hand, detectors that rely solely on overall appearance are prone to false positives when faced with objects with similar local textures (such as building roofs or other vehicles).

[0004] Some existing research attempts to introduce component detection or relation reasoning, but most of them have the following shortcomings: (1) Failed to quantify the visibility of components into a learnable, dynamic confidence adjustment mechanism; (2) Lack of a post-processing procedure that utilizes rigorous prior knowledge of aircraft structure for reasoning; (3) During the training process, no structural constraints between the whole and the parts were introduced at the loss function level, which may result in discrete and false part detection results that are inconsistent with the spatial logic of the whole target.

[0005] Therefore, there is an urgent need for a detection method that can effectively deal with occlusion while maintaining high accuracy and high recall. Summary of the Invention

[0006] To address the problems existing in the above-mentioned traditional methods, this invention proposes a method, apparatus, equipment, and medium for detecting partially occluded aircraft targets in remote sensing images, which can improve the robustness and accuracy of detection under occlusion conditions and effectively suppress the detection of false parts.

[0007] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions: On the one hand, a method for detecting partially occluded aircraft targets in remote sensing images is provided, including the following steps: The remote sensing image of the aircraft to be detected is input into a pre-trained multi-task deep learning detection network to obtain the set of detection boxes and initial confidence scores for the overall aircraft target and its components. The detection network includes a feature extraction network, an overall detection head, and a component detection head. The overall detection head is used to predict the overall bounding box of the aircraft and its confidence score. The component detection head is used to predict the bounding box and its confidence score for each key component of the aircraft.

[0008] The initial confidence level of each overall aircraft target and the initial confidence level of each component falling within its overall detection frame are dynamically fused to obtain the final confidence level of the overall aircraft target. All detected components are clustered using a distance-based clustering algorithm to obtain multiple component clusters.

[0009] Calculate the spatial layout of components within each component cluster, and perform similarity matching between the spatial layout and a preset aircraft structure template to obtain the similarity matching degree.

[0010] If the similarity matching degree exceeds the preset threshold, it is determined that there is an aircraft target. Based on the center and range of the component cluster, an optimized aircraft bounding box is generated, and the confidence of the aircraft target is calculated. Otherwise, the component cluster is discarded. This process continues until all component clusters have been traversed, and the final detection result of the aircraft target is output.

[0011] In one embodiment, the training process of a multi-task deep learning detection network includes: Acquire a dataset of remote sensing aircraft images, and annotate an overall bounding box and the bounding boxes of its five key components for each aircraft to obtain a training sample set; the visible parts of the occluded components should also be annotated.

[0012] Construct a total loss function; the total loss function includes: standard detection loss for the whole and components, regularization loss, and structural constraint loss, the structural constraint loss being used to force the whole and components, and components to maintain the correct spatial relationship.

[0013] Based on the training sample set and the total loss function, the task-oriented deep learning detection network is trained to obtain the trained task-oriented deep learning detection network.

[0014] In one embodiment, the total loss function is:

[0015] in: It is the total loss function. and These are the standard inspection losses for the whole system and individual components; It is the regularization loss. It is structural constraint loss. , , , There are four weight parameters.

[0016] In one embodiment, the structural constraint loss is:

[0017]

[0018]

[0019] in, It is a whole-component constraint. It is a component-to-component constraint. It is the distance from the i-th component to the target center point in the prediction result. For the predicted results, It is the distance from the i-th component in the true value to the center point of the target. It is the first i The component and the first j The distance between the center point of the predicted bounding box of each component For this is the first i The first component and the first j The actual distance between the center points of each component's labeled dimensions This is a true label.

[0020] In one embodiment, the five key components include: the nose, wings, left engine, right engine, and tail.

[0021] In one embodiment, the initial confidence level of each overall aircraft target and the initial confidence level of each component falling within its overall detection frame are dynamically fused to obtain the final confidence level of the overall aircraft target:

[0022] in, The final confidence level for the overall aircraft objective. The goal The initial confidence level, It fell into The first one in the box Confidence level of each component It is the first i The bounding box of the first component and the first j The overlap of the overall bounding boxes of the aircraft targets. It is a hyperparameter that adjusts the contribution weight of components. It is the summation symbol. The first i The bounding box of the first component and the first j The overall frame of the aircraft target.

[0023] In one embodiment, the multi-task deep learning detection network is, but is not limited to, based on... or An improved model.

[0024] On the other hand, a remote sensing image partially occluded aircraft target detection device is also provided, including: The preliminary detection module is used to input the remote sensing aircraft image to be detected into a trained multi-task deep learning detection network to obtain the set of detection boxes and initial confidence scores for the overall aircraft target and its components. The detection network includes a feature extraction network, an overall detection head, and a component detection head. The overall detection head is used to predict the overall bounding box of the aircraft and its confidence score. The component detection head is used to predict the bounding box and its confidence score for each key component of the aircraft.

[0025] The overall aircraft target confidence fusion module is used to dynamically fuse the initial confidence of each overall aircraft target and the initial confidence of each component falling within its overall detection frame to obtain the final confidence of the overall aircraft target. The structural constraint reasoning module is used to cluster all detected components using a distance-based clustering algorithm to obtain multiple component clusters; calculate the spatial layout of components within each component cluster, and perform similarity matching between the spatial layout and a preset aircraft structure template to obtain the similarity matching degree; if the similarity matching degree of a component exceeds a preset threshold, it is determined that there is an aircraft target, and an optimized aircraft bounding box is generated based on the center and range of the component cluster, and the confidence degree of the aircraft target is calculated; otherwise, the component cluster is discarded, until all component clusters have been traversed, and the final detection result of the aircraft target is output.

[0026] On the other hand, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-mentioned remote sensing image partially occluded aircraft target detection method.

[0027] Furthermore, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the above-mentioned remote sensing image partial occlusion aircraft target detection methods.

[0028] One of the above technical solutions has the following advantages and beneficial effects: The aforementioned method, apparatus, equipment, and medium for detecting partially occluded aircraft targets in remote sensing images involve extracting features from the remote sensing aircraft image under test using a feature extraction network based on a multi-task deep learning detection network. Then, an overall detection head predicts the overall bounding box and its confidence level of the aircraft; a component detection head predicts the bounding box and its confidence level of each key component of the aircraft; a dynamic mapping relationship is established between component visibility and the overall confidence level of the target; for multiple detected components, their spatial relative positions are analyzed and matched with a preset aircraft structure template to infer the complete aircraft target, effectively merging duplicate detections and eliminating erroneous detections. This method significantly improves the robustness and accuracy of detection under occlusion conditions and effectively suppresses false component detections. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart illustrating a method for detecting partially occluded aircraft targets in remotely sensed images, as shown in one embodiment. Figure 2 This is a schematic diagram of the overall process of a remote sensing image partially occluded aircraft target detection method in one embodiment; Figure 3 In one embodiment, the distance between the component and the center point Distance between components Schematic diagram; Figure 4 This is a schematic diagram of the effect of dynamic confidence fusion and structural constraint reasoning in one embodiment, where (a) is the direct reasoning result of the existing method and (b) is the reasoning result of the present method. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0033] It should be noted that, in this document, the reference to "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The presentation of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will understand that the embodiments described herein can be combined with other embodiments. The term "and / or" as used herein refers to any combination of one or more of the associated listed items, and all possible combinations, including such combinations.

[0034] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0035] In one embodiment, such as Figure 1 As shown, a method for detecting partially occluded aircraft targets in remote sensing images is provided, which may include the following processing steps 100 to 104: Step 100: Input the remote sensing aircraft image to be detected into the trained multi-task deep learning detection network to obtain the set of detection boxes and initial confidence scores of the overall aircraft target and components; the detection network includes a feature extraction network, an overall detection head and a component detection head; the overall detection head is used to predict the overall bounding box of the aircraft and its confidence score; the component detection head is used to predict the bounding box of each key component of the aircraft and its confidence score.

[0036] Specifically, construct a multi-task deep learning detection network (such as based on...). or An improved model). This network has a shared backbone feature extraction network ( ), and then divided into two detection heads ( ),in Overall detection head: used to predict the overall bounding box of an aircraft. and its confidence level Component inspection head: used to predict the bounding boxes of the above five aircraft components. and its confidence level ( i =1,2,...,5). The key components include: nose, wings, left engine, right engine, and tail.

[0037] Step 101: Dynamically fuse the initial confidence of each overall aircraft target and the initial confidence of each component falling within its overall detection frame to obtain the final confidence of the overall aircraft target; Specifically, dynamic confidence. During the inference phase, the final confidence level of the target (…). The result is no longer determined solely by the overall detection branch, but by its own confidence level. ) and the confidence level of all detected components ( The confidence level is determined jointly by all parties. It is evident that the more components and the higher their quality, the higher the final confidence level, and vice versa.

[0038] Step 102: Cluster all detected components using a distance-based clustering algorithm to obtain multiple component clusters.

[0039] Specifically, component clustering: a distance-based clustering algorithm (such as DBSCAN) is used to cluster all detected components. Clustering is performed to group spatially adjacent components together, making a preliminary judgment that they may belong to the same aircraft.

[0040] Step 103: Calculate the spatial layout of the components within each component cluster, and perform similarity matching between the spatial layout and the preset aircraft structure template to obtain the similarity matching degree.

[0041] Specifically, template matching involves calculating the spatial layout of internal components for each component cluster (e.g., wings should be on both sides of the fuselage, and tail fins should be at the rear of the fuselage). This layout is then matched against a pre-defined aircraft structure template (a prior matrix that defines the relative angles, distances, and size ratios between components).

[0042] Step 104: If the similarity matching degree exceeds the preset threshold, it is determined that there is an aircraft target. Based on the center and range of the component cluster, an optimized aircraft bounding box is generated, and the confidence of the aircraft target is calculated; otherwise, the component cluster is discarded until all component clusters are traversed, and the final detection result of the aircraft target is output.

[0043] Specifically, structural constraint post-processing is performed. For multiple detected components, their spatial relative positions are analyzed and matched with a preset aircraft structure template to infer the complete aircraft target, effectively merging duplicate detections and eliminating erroneous detections.

[0044] Target generation and verification: If the layout and template matching degree of a component cluster exceeds the threshold... If a target is detected at a certain location, it is considered to be an aircraft target. An optimized aircraft bounding box is generated based on the center and extent of the component cluster. The confidence level of this target can be determined by the average or highest confidence level of the components within the cluster. This step effectively "reverse-engineers" aircraft that are severely occluded and not detected by the overall detection head from the components, and merges overall targets that have been mistakenly split into multiple targets due to occlusion.

[0045] The aforementioned method for detecting partially occluded aircraft targets in remote sensing images involves extracting features from the remote sensing aircraft image under test using a feature extraction network based on a multi-task deep learning detection network. Then, an overall detection head predicts the overall bounding box and its confidence score of the aircraft; a component detection head predicts the bounding box and its confidence score of each key component of the aircraft; a dynamic mapping relationship is established between component visibility and the overall confidence score of the target; for multiple detected components, their spatial relative positions are analyzed and matched with a preset aircraft structure template to infer the complete aircraft target, effectively merging duplicate detections and eliminating erroneous detections. This method significantly improves the robustness and accuracy of detection under occlusion conditions and effectively suppresses false component detections.

[0046] In one embodiment, the training process of the multi-task deep learning detection network in step 100 includes: Step 200: Obtain the remote sensing aircraft image dataset and label each aircraft with an overall bounding box and the bounding boxes of its five key components to obtain the training sample set; the visible parts of the occluded components should also be labeled.

[0047] Step 201: Construct the total loss function; the total loss function includes: standard detection loss for the whole and components, regularization loss, and structural constraint loss. The structural constraint loss is used to force the whole and components, and components to maintain the correct spatial relationship.

[0048] Specifically, structural constraint loss: In the loss function during the training phase, a structural constraint loss term based on the spatial relationship between the whole and its components is introduced, which forces the network to learn a component layout that conforms to physical laws and avoids false and discrete component detection.

[0049] Step 202: Train the task-specific deep learning detection network based on the training sample set and the total loss function to obtain the trained task-specific deep learning detection network.

[0050] In one embodiment, the total loss function in step 201 is:

[0051] in: It is the total loss function. and These are the standard inspection losses for the whole system and individual components; It is the regularization loss. This is the structural constraint loss, which is calculated by comparing the predicted and actual values ​​in the spatial geometric relationships between the whole and its components, and between components. , , , There are four weight parameters.

[0052] In one embodiment, the structural constraint loss in step 201 is:

[0053]

[0054]

[0055] in, It is a whole-component constraint. It is a component-to-component constraint. It is the first in the prediction results i The distance from each component to the target center point For the predicted results, It is the first truth value. i The distance from each component to the target center point It is the first i The first component and the first j The distance between the center point of the predicted bounding box of each component For this is the first i The first component and the first j The actual distance between the center points of each component's labeled dimensions This is a true label.

[0056] Specifically, (Global-Component Constraints): For each real aircraft target, calculate the distance between the center point of its global bounding box and the center point of each real component bounding box. The prediction of the overall bounding box and the center point of the corresponding component bounding box are constrained to be close to each other.

[0057] (Component-Component Constraint): For any two components belonging to the same real aircraft (such as the left wing and fuselage), calculate the distance between the center points of their predicted bounding boxes. Distance from the center point of the actual annotation The differences.

[0058] Structural constraint losses are used to enforce correct spatial relationships between the whole and its components, and between components themselves. A diagram illustrating structural loss calculations is shown below. Figure 3 As shown, this structural loss term effectively penalizes false detections that are far from the overall target or whose relative positions between components are abnormal, making the component detection results more cohesive and consistent with the physical structure.

[0059] In one embodiment, the five key components in step 200 include: the nose, wings, left engine, right engine, and tail.

[0060] In one embodiment, step 101 includes: dynamically fusing the initial confidence level of each overall aircraft target and the initial confidence level of each component falling within its overall detection frame to obtain the final confidence level of the overall aircraft target:

[0061] in, The final confidence level for the overall aircraft objective. The goal The initial confidence level, It fell into The first one in the box Confidence level of each component It is the first i The bounding box of the first component and the first j The overlap of the overall bounding boxes of the aircraft targets. It is a hyperparameter that adjusts the contribution weight of components. It is the summation symbol. The first i The bounding box of the first component and the first j The overall frame of the aircraft target.

[0062] Specifically, forward inference: The image to be detected is input into the trained model to obtain a preliminary set of overall detection boxes. and component detection box set .

[0063] Dynamic confidence fusion: For each initially detected overall aircraft target Its final confidence level Calculate using the following formula:

[0064] in: The goal The initial confidence level, It fell into The first one in the box Confidence level of each component Calculate the overlap between the component bounding box and the overall bounding box for weight allocation; It is a hyperparameter that adjusts the contribution weight of components. Summing all components that fall within its frame.

[0065] The final confidence calculation formula achieves the following: the more components exposed (the more terms to sum), the higher the confidence of the component itself, and the better the component's position matches the whole (the higher the IOU), the higher the final confidence. The higher the value, the better. In one embodiment, the detection network of the multi-task deep learning in step 100 is, but is not limited to, based on... or An improved model.

[0066] In one specific embodiment, with To improve the baseline model, MAR20 was used as the experimental dataset. use ,exist After that, copy and modify. This forms two independent detection heads, which predict separately. The overall target of the class (aircraft as one class) and The target of the class's components. During training, the following is used: Optimizer, initial learning rate set to The loss weights are set as follows: During reasoning, Set as Template matching threshold Set as The aircraft structural template is obtained by calculating the average of the relative positions and dimensions between all correctly labeled aircraft components in the training set. Structural constraint loss. Initially, a smaller weight is assigned, which is gradually increased as training progresses to stabilize the learning process. A comparison chart of the effects of dynamic confidence fusion and structural constraint inference is presented. Figure 4 As shown, where Figure 4 In (a), the result is the direct inference result of the existing method. Figure 4 (b) is the result of the reasoning in this method.

[0067] It should be understood that, although the above process Figure 1 The steps in the diagram are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are executed; they can be performed in other orders. Furthermore, the above process... Figure 1 At least some of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0068] In one embodiment, a remote sensing image partially occluded aircraft target detection device is also provided, the device comprising: The preliminary detection module is used to input the remote sensing aircraft image to be detected into a trained multi-task deep learning detection network to obtain the set of detection boxes and initial confidence scores for the overall aircraft target and components. The detection network includes a feature extraction network, an overall detection head, and a component detection head. The overall detection head is used to predict the overall bounding box of the aircraft and its confidence score. The component detection head is used to predict the bounding box of each key component of the aircraft and its confidence score.

[0069] The overall aircraft target confidence fusion module is used to dynamically fuse the initial confidence of each overall aircraft target and the initial confidence of each component falling within its overall detection frame to obtain the final confidence of the overall aircraft target. The structural constraint reasoning module is used to cluster all detected components using a distance-based clustering algorithm to obtain multiple component clusters. It calculates the spatial layout of the components within each component cluster and performs similarity matching between the spatial layout and a preset aircraft structure template to obtain the similarity matching degree. If the similarity matching degree of a component exceeds a preset threshold, it determines that there is an aircraft target. Based on the center and range of the component cluster, it generates an optimized aircraft bounding box and calculates the confidence degree of the aircraft target. Otherwise, it discards the component cluster. This process continues until all component clusters have been traversed, and the final aircraft target detection result is output.

[0070] In one embodiment, the training process of the multi-task deep learning detection network in the preliminary detection module includes: acquiring a remote sensing aircraft image dataset and labeling each aircraft with an overall bounding box and the bounding boxes of its five key components to obtain a training sample set; wherein, the visible parts of occluded components are also labeled; constructing a total loss function; the total loss function includes: the standard detection loss for the whole and components, the regularization loss, and the structural constraint loss, the structural constraint loss being used to force the whole and components, and the components to maintain the correct spatial relationship; and training the task deep learning detection network based on the training sample set and the total loss function to obtain the trained task deep learning detection network.

[0071] In one embodiment, the total loss function in the preliminary detection module is:

[0072] in: It is the total loss function. and These are the standard inspection losses for the whole system and individual components; It is the regularization loss. It is structural constraint loss. , , , There are four weight parameters.

[0073] In one embodiment, the structural constraint loss in the preliminary detection module is:

[0074]

[0075]

[0076] in, It is a whole-component constraint. It is a component-to-component constraint. It is the first in the prediction results i The distance from each component to the target center point For the predicted results, It is the first truth value. i The distance from each component to the target center point It is the first i The first component and the first j The distance between the center point of the predicted bounding box of each component For this is the first i The first component and the first j The actual distance between the center points of each component's labeled dimensions This is a true label.

[0077] In one embodiment, the five key components of the preliminary detection module include: the nose, wings, left engine, right engine, and tail.

[0078] In one embodiment, step 101 includes: dynamically fusing the initial confidence level of each overall aircraft target and the initial confidence level of each component falling within its overall detection frame to obtain the final confidence level of the overall aircraft target:

[0079] in, The final confidence level for the overall aircraft objective. The goal The initial confidence level, It fell into The first one in the box Confidence level of each component It is the first i The bounding box of the first component and the first j The overlap of the overall bounding boxes of the aircraft targets. It is a hyperparameter that adjusts the contribution weight of components. It is the summation symbol. The first i The bounding box of the first component and the first j The overall frame of the aircraft target.

[0080] In one embodiment, the detection network in the preliminary detection module, based on multi-task deep learning, is, but not limited to, a detection network based on... or An improved model.

[0081] It is understood that for a detailed explanation of the device for detecting partially occluded aircraft targets in remote sensing images, please refer to the corresponding explanations of the various embodiments of the method for detecting partially occluded aircraft targets in remote sensing images above, and will not be repeated here. Each module in the aforementioned device for detecting partially occluded aircraft targets in remote sensing images can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in hardware or independently of a device with data processing capabilities, or stored in software in the memory of the aforementioned device, so that the processor can call and execute the operations corresponding to each module. The aforementioned device can be, but is not limited to, various types of data processing computer devices already existing in the art.

[0082] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0083] It is understood that, in addition to the memory and processor mentioned above, the computer equipment described above also includes other hardware and software components not listed in this specification. The specific components can be determined according to the model of the image processing computer in different application scenarios, and will not be listed and described in detail in this specification.

[0084] In one embodiment, when the processor executes the computer program, it can also implement the steps or sub-steps added in the various embodiments of the remote sensing image partially occluded aircraft target detection method.

[0085] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus DRAM (RDRAM), and interface DRAM (DRDRAM), etc.

[0086] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0087] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and all such modifications and improvements fall within the scope of protection of this application.

Claims

1. A method for detecting partially occluded aircraft targets in remote sensing images, characterized in that, Including the following steps: The remote sensing image of the aircraft to be detected is input into a pre-trained multi-task deep learning detection network to obtain the set of detection boxes and initial confidence scores for the overall aircraft target and its components. The detection network includes a feature extraction network, an overall detection head, and a component detection head. The overall detection head is used to predict the overall bounding box of the aircraft and its confidence score. The component detection head is used to predict the bounding box of each key component of the aircraft and its confidence score. The initial confidence level of each overall aircraft target and the initial confidence level of each component falling within its overall detection frame are dynamically fused to obtain the final confidence level of the overall aircraft target. All detected components are clustered using a distance-based clustering algorithm to obtain multiple component clusters; Calculate the spatial layout of components within each component cluster, and perform similarity matching between the spatial layout and a preset aircraft structure template to obtain the similarity matching degree; If the similarity matching degree exceeds the preset threshold, it is determined that there is an aircraft target. Based on the center and range of the component cluster, an optimized aircraft bounding box is generated, and the confidence degree of the aircraft target is calculated. Otherwise, discard the component cluster until all component clusters have been traversed, and output the final detection result of the aircraft target.

2. The method for detecting partially occluded aircraft targets in remote sensing images according to claim 1, characterized in that, The training process of a multi-task deep learning detection network includes: Acquire a dataset of remote sensing aircraft images, and annotate an overall bounding box and the bounding boxes of its five key components for each aircraft to obtain a training sample set; the visible parts of the occluded components should also be annotated. Construct a total loss function; the total loss function includes: standard detection loss for the whole and components, regularization loss, and structural constraint loss, the structural constraint loss being used to force the whole and components, and components to maintain the correct spatial relationship; Based on the training sample set and the total loss function, the task-oriented deep learning detection network is trained to obtain the trained task-oriented deep learning detection network.

3. The method for detecting partially occluded aircraft targets in remote sensing images according to claim 2, characterized in that, The total loss function is: in: It is the total loss function. and These are the standard inspection losses for the whole system and individual components; It is the regularization loss. It is structural constraint loss. , , , There are four weight parameters.

4. The method for detecting partially occluded aircraft targets in remote sensing images according to claim 3, characterized in that, The structural constraint loss is: in, It is a whole-component constraint. It is a component-to-component constraint. It is the first in the prediction results i The distance from each component to the target center point For the predicted results, It is the first truth value. i The distance from each component to the target center point It is the first i The first component and the first j The distance between the center point of the predicted bounding box of each component For this is the first i The first component and the first j The actual distance between the center points of each component's labeled dimensions This is a true label.

5. The method for detecting partially occluded aircraft targets in remote sensing images according to claim 2, characterized in that, The five key components include: the nose, wings, left engine, right engine, and tail.

6. The method for detecting partially occluded aircraft targets in remote sensing images according to claim 1, characterized in that, The initial confidence scores of each overall aircraft target and the initial confidence scores of each component falling within its overall detection frame are dynamically fused to obtain the final confidence score of the overall aircraft target: in, The final confidence level for the overall aircraft objective. The goal The initial confidence level, It fell into The first one in the box Confidence level of each component It is the first i The bounding box of the first component and the first j The overlap of the overall bounding boxes of the aircraft targets. It is a hyperparameter that adjusts the contribution weight of components. It is the summation symbol. The first i The bounding box of the first component and the first j The overall frame of the aircraft target.

7. The method for detecting partially occluded aircraft targets in remote sensing images according to claim 1, characterized in that, Multi-task deep learning detection networks are, but are not limited to, those based on... or An improved model.

8. A device for detecting partially occluded aircraft targets in remote sensing images, characterized in that, include: The preliminary detection module is used to input the remote sensing aircraft image to be detected into a trained multi-task deep learning detection network to obtain the set of detection boxes and initial confidence scores for the overall aircraft target and its components. The detection network includes a feature extraction network, an overall detection head, and a component detection head. The overall detection head is used to predict the overall bounding box of the aircraft and its confidence score. The component detection head is used to predict the bounding box of each key component of the aircraft and its confidence score. The overall aircraft target confidence fusion module is used to dynamically fuse the initial confidence of each overall aircraft target and the initial confidence of each component falling within its overall detection frame to obtain the final confidence of the overall aircraft target. The structural constraint reasoning module is used to cluster all detected components using a distance-based clustering algorithm to obtain multiple component clusters; calculate the spatial layout of components within each component cluster; and perform similarity matching between the spatial layout and a preset aircraft structure template to obtain the similarity matching degree. If the similarity matching degree of a component exceeds a preset threshold, it is determined that there is an aircraft target. Based on the center and range of the component cluster, an optimized aircraft bounding box is generated, and the confidence of the aircraft target is calculated. Otherwise, discard the component cluster until all component clusters have been traversed, and output the final detection result of the aircraft target.

9. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the remote sensing image partially occluded aircraft target detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the remote sensing image partially occluded aircraft target detection method according to any one of claims 1 to 7.

Images