Evolutionary and inherited methods for salient object detection in high-resolution UAV images

Abstract

This invention provides an evolutionary and inherited method for detecting salient objects in high-resolution drone imagery, belonging to the field of image processing technology. The method comprises an evolutionary phase and an inheritance phase. The evolutionary phase uses evolutionary mechanisms at both the supervisory and feature levels to achieve detail-preserving and target-integrity salient object localization in low-resolution images. The inheritance phase utilizes shallow high-resolution features to supplement and enhance inherited features in a lightweight manner, generating a final high-resolution salient prediction. The invention achieves higher-precision salient object detection in high-resolution drone imagery.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an evolutionary and inherited method for detecting salient targets in high-resolution unmanned aerial vehicle images. Background Art

[0002] The statements in this section merely provide background art related to the present invention and do not necessarily constitute prior art.

[0003] In recent years, thanks to the continuous development of deep learning, salient object detection (SOD) has made great progress. However, previous SOD models and datasets have mainly focused on regular life images limited to conventional resolutions (such as 400×400). Compared with regular life low-resolution images, the main difference between UAV high-resolution images is that UAV images provide higher pixel density and details, can capture wide geographical areas from the air, and are suitable for professional fields such as geographic information systems and environmental monitoring.

[0004] Salient object detection (SOD) is inspired by the human visual system and aims to locate the most attractive objects and segment them pixel-by-pixel in a given image. As an upstream task, it has been widely used in a wide range of computer vision tasks, including image / video segmentation, video compression, and thumbnail creation. In recent years, particularly in the era of deep learning, the SOD task has undergone significant development, leading to the formation of a full-scenario, multi-source research system. With the continuous advancement and upgrading of hardware devices, high-resolution images and videos have become ubiquitous in our lives. The task of high-resolution salient object detection (HRSOD) has emerged. Furthermore, with the popularization and advancement of drone technology, its applications have expanded to various fields, including urban planning, agricultural monitoring, environmental protection, disaster response, and military reconnaissance. Drones offer a new perspective on images, capable of capturing high-resolution images of vast areas from the air. These images provide rich visual information suitable for detailed analysis. Unlike traditional normal-resolution salient object detection tasks, high-resolution salient object detection in drone imagery faces two key challenges. First, from a data source perspective, high-resolution imagery can more accurately and vividly capture fine details and small objects. This ability to perceive these complex elements distinguishes it from standard SOD tasks. Achieving high-quality detection in these areas is a worthy question. Second, because the number of pixels is dozens of times greater than in a normal image, the computational complexity of directly processing high-resolution drone imagery is unimaginable.

[0005] Some researchers have conducted research in the field of high-resolution salient object detection and proposed a two-stage solution. In the first stage, salient objects are located from a macroscopic perspective by processing the low-resolution images obtained by downsampling. Then, in the second stage, the high-resolution images are locally cropped and fine-tuned, and the results of the two stages are finally fused and spliced ​​to form a high-resolution salient map. At the same time, some researchers have adopted a two-stage network and reduced the amount of computation through image cropping. In addition, they have also incorporated the ternary map technology and uncertain region loss function commonly used in the field of image cutouts into the network. Some researchers have adopted an end-to-end lightweight network structure and a dual-branch network architecture to process details and global context information respectively. In order to expand the network's receptive field and reduce the amount of computation, they have made extensive use of dilated convolution and depthwise separable convolution techniques. Some researchers have also used two different types of backbone networks to extract features from images of different resolutions to capture more complementary information.

[0006] In the process of realizing the present invention, the inventors found that the prior art has at least the following shortcomings and deficiencies:

[0007] (1) In the two-stage "block-splicing" model, global features and local features are processed separately, which may cause repeated feature extraction. At the same time, the blocking and splicing operations are likely to produce inconsistencies in the results, which will also reduce the overall reasoning speed of the model; (2) In the end-to-end model structure, it is not necessary to use the original high-resolution image to extract global context information, which will increase the additional computational burden; (3) Existing models often lack sufficient attention to small targets and detailed areas that are unique to high-resolution drone images. They are the key to high-resolution tasks and the key to improving the subsequent application scenarios of high-resolution drone images. Summary of the Invention

[0008] To address the shortcomings of the existing technology, the present invention provides an evolutionary and inherited method for salient target detection in high-resolution UAV imagery. In the evolutionary stage, evolutionary mechanisms at the supervision and feature levels are used to achieve detail-preserving and target-intact salient target localization on low-resolution images. In the inheritance stage, shallow high-resolution features are used to supplement and enhance the inherited features in a lightweight manner to generate the final high-resolution saliency prediction.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] In a first aspect, the present invention provides an evolutionary and inherited method for salient object detection in high-resolution drone images.

[0011] An evolutionary and inherited method for salient object detection in high-resolution UAV imagery includes the following steps:

[0012] The high-resolution UAV image is downsampled to obtain a downsampled image. The downsampled image is sequentially passed through multiple coding layers, each coding layer corresponds to a coding feature. Feature evolution is performed based on the low-resolution saliency map and decoding features output by any decoding layer to obtain the decoding features output by the previous decoding layer.

[0013] The first encoding features corresponding to the first encoding layer and the second decoding features corresponding to the second decoding layer are upsampled respectively. The upsampled first encoding features are updated according to the high-resolution UAV image to obtain enhanced high-resolution encoder features. The final salient object detection results are obtained based on the enhanced high-resolution encoder features and the upsampled second decoding features.

[0014] As a further limitation of the first aspect of the present invention, updating the upsampled first encoding feature according to the high-resolution drone image to obtain an enhanced high-resolution encoder feature includes:

[0015] Extract high-resolution features from high-resolution drone images, combine them with the first saliency map output by the first-layer decoder to enhance salient attributes, perform pixel-by-pixel multiplication on the Sobel value of the saliency enhancement result and the upsampled first encoded feature, and then add the pixel-by-pixel multiplication result to the upsampled first encoded feature to obtain enhanced high-resolution encoder features.

[0016] Among them, significant attribute enhancements are made, including:

[0017] Obtaining a saliency boundary region and a background boundary region according to the first saliency map after upsampling;

[0018] According to the salient boundary area, the salient prototype is extracted from the input features through mask average pooling;

[0019] According to the background boundary area, the background prototype is extracted from the input features through mask average pooling;

[0020] According to the feature extraction salient prototype, feature extraction background prototype and input features, the features after salient attribute enhancement are obtained.

[0021] As a further limitation of the first aspect of the present invention, performing feature evolution based on the low-resolution saliency map and decoding features output by any decoding layer to obtain decoding features output by the previous decoding layer includes:

[0022] Fuse the decoding features output by any decoding layer with the encoding features of the previous layer to obtain fused features;

[0023] The fusion feature is multiplied by the low-resolution saliency map of the current layer and then added to the fusion feature to obtain the decoding feature output by the decoding layer of the previous layer.

[0024] As a further limitation of the first aspect of the present invention, at the last layer of the decoder, the supervision target is the saliency map output by the decoding layer of the last layer; at the i-th layer of the decoder, the supervision target is the saliency map output by the decoding layer of the i-th layer; and so on until the first layer of the decoder, the supervision target is the saliency map output by the decoding layer of the i-th layer;

[0025] The supervised truth map corresponding to the high-resolution saliency map is downsampled to obtain the supervised truth map of the first layer, and the supervised truth map of the i-1th layer is subjected to a maximum pooling operation to obtain the supervised truth map of the i-th layer, where i is a positive integer greater than 1.

[0026] As a further limitation of the first aspect of the present invention, the evolutionary change constraint is used so that the difference between two adjacent layers is only reflected in the area to be evolved, including:

[0027] The encoding process gradually extracts high-level features from shallow features, i gradually increases from 1 to 5, and the decoding process advances from high-level to low-level, i gradually decreases from 5 to 1. In the decoding process, the previous layer of the i-th layer is the i+1-th layer;

[0028] Use the saliency map output by the previous decoding step to suppress the noise in the background area of ​​the current layer feature and the higher layer feature. Then, multiply the corresponding positions of the two layers of features after suppressing the background noise to obtain the internal shared features.

[0029] The higher-level features are subtracted from the common features at the pixel level to obtain the difference area features. The difference supervision of the corresponding level evolution labels is used to constrain the difference area features to only be reflected on the target boundary contour.

[0030] As a further limitation of the first aspect of the present invention, the salient object detection results are evaluated by boundary-detail perceived mean absolute error, where the boundary-detail perceived mean absolute error is calculated based on the boundary region mask, the detail region mask, the error weight, the saliency mask, and the background mask.

[0031] In a second aspect, the present invention provides an evolved and inherited high-resolution drone image salient object detection system.

[0032] An evolved and inherited high-resolution salient object detection system for UAV imagery, including:

[0033] The evolution module is configured to: downsample the high-resolution drone image to obtain a downsampled image, sequentially pass the downsampled image through multiple coding layers, each coding layer corresponds to a coding feature, perform feature evolution based on the low-resolution saliency map and decoding features output by any decoding layer, and obtain the decoding features output by the previous decoding layer;

[0034] The inheritance module is configured to: upsample the first encoding features corresponding to the first encoding layer and the second decoding features corresponding to the second decoding layer, update the upsampled first encoding features according to the high-resolution UAV image to obtain enhanced high-resolution encoder features, and obtain the final salient object detection results based on the enhanced high-resolution encoder features and the upsampled second decoding features.

[0035] As a further limitation of the second aspect of the present invention, in the inheritance module, updating the upsampled first encoding feature according to the high-resolution drone image to obtain an enhanced high-resolution encoder feature includes:

[0036] Extract high-resolution features from high-resolution drone images, combine them with the first saliency map output by the first-layer decoder to enhance salient attributes, perform pixel-by-pixel multiplication on the Sobel value of the saliency enhancement result and the upsampled first encoded feature, and then add the pixel-by-pixel multiplication result to the upsampled first encoded feature to obtain enhanced high-resolution encoder features.

[0037] Among them, significant attribute enhancements are made, including:

[0038] Obtaining a saliency boundary region and a background boundary region according to the first saliency map after upsampling;

[0039] According to the salient boundary area, the salient prototype is extracted from the input features through mask average pooling;

[0040] According to the background boundary area, the background prototype is extracted from the input features through mask average pooling;

[0041] According to the feature extraction salient prototype, feature extraction background prototype and input features, the features after salient attribute enhancement are obtained.

[0042] In a third aspect, the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the evolved and inherited high-resolution drone image salient target detection method as described in the first aspect of the present invention.

[0043] In a fourth aspect, the present invention provides an electronic device comprising a memory, a processor, and a program stored in the memory and runnable on the processor. When the processor executes the program, the steps of the evolved and inherited high-resolution drone image salient target detection method as described in the first aspect of the present invention are implemented.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. From the perspectives of evolution and inheritance, this paper proposes a two-stage evolution-inheritance network (ESNet) to achieve the task of salient object detection in high-resolution UAV imagery. In the evolution stage, evolutionary mechanisms at the supervision and feature levels are used to achieve detail-preserving and target-intact salient object localization in low-resolution images. In the inheritance stage, shallow high-resolution features are used to supplement and enhance the inherited features in a lightweight manner, achieving more accurate salient object detection in high-resolution UAV imagery.

[0046] 2. This paper innovatively proposes an evolutionary and inherited strategy for salient target detection in high-resolution UAV images, designs a new evaluation metric, boundary-detail perception mean absolute error, and realizes accurate evaluation of detail detection quality in high-resolution UAV images. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0048] Figure 1 A schematic diagram of the process flow of the evolved and inherited high-resolution drone image salient object detection method provided in Example 1 of the present invention;

[0049] Figure 2 Schematic diagram of a visualization example of high-resolution salient object detection provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0050] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0051] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0052] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0053] Example 1:

[0054] like Figure 1 As shown, embodiment 1 of the present invention provides an evolutionary and inherited high-resolution drone image salient target detection method, and proposes a two-stage evolution-inheritance network (ESNet), including a low-resolution salient target localization model (LLM) in the evolutionary stage and a high-resolution refinement model (HRM) in the inheritance stage. The evolutionary stage realizes detail-preserving salient target localization on low-resolution images through evolutionary mechanisms at the supervision level and feature level; the inheritance stage uses shallow high-resolution features to supplement and enhance the inherited features in a lightweight manner to generate the final high-resolution salient prediction. In addition, the present invention also designs a new evaluation index boundary-detail perception mean absolute error (MAE BD ) to evaluate the quality of detail detection in high-resolution UAV imagery.

[0055] Specifically, the process includes the following:

[0056] S1: Evolutionary stage (low-resolution salient object localization model).

[0057] As the beginning of the network, the evolution stage lays the foundation for the subsequent refinement process, focusing on the accurate and complete positioning of salient targets. At the same time, in order to adapt to the requirements of high-quality description of details in high-resolution scenes, it is also important to retain detailed areas in the detection results. In order to achieve these goals, the LLM of the present invention adopts an encoder-decoder structure and combines the evolution mechanism of supervision and features. On the one hand, according to the rule from coarse to fine, the present invention upgrades the side supervision in the traditional SOD task to adapt to the high-resolution drone image SOD task, and proposes the idea of ​​progressive constraint of supervision evolution, providing different supervision labels for each layer. The core operation of this part is to gradually expand the original supervision labels through the maximum pooling operation to obtain different degrees of supervision information; on the other hand, in order to further model the relationship between features of different layers, the present invention designs an evolutionary structure in the feature dimension. The present invention focuses on the areas to be evolved corresponding to the changes between supervision labels, and then applies evolution difference loss between the features of adjacent layers to strengthen the network's attention to the evolution difference areas.

[0058] Specifically, high-resolution drone images are first Downsample to Then I l The pre-trained feature extraction backbone network is fed with multi-level encoding features. Then, an evolution-based decoder is used to generate a series of complete and accurate low-resolution saliency maps while preserving details, laying the foundation for subsequent detail restoration.

[0059] I l =Downsample(I h ) (1);

[0060]

[0061]

[0062] Among them, Downsample(·) represents the downsampling operation, Encoder(·) represents the pre-trained feature extraction backbone network, and Decoder evo (·) represents the evolution-based feature decoder, They represent the encoding features, decoding features and corresponding level saliency map output of the i-th layer at low resolution.

[0063] S1.1: Supervised evolution.

[0064] Considering that as the network deepens, the boundary information in the network's high-level features gradually becomes blurred due to operations such as pooling and convolution, if traditional side-supervision methods use a ground-truth map with clear boundaries to strictly constrain the output of high-level features, it may have a negative impact on network learning and accumulate errors during the layer-by-layer decoding process. In addition, detailed areas such as butterfly antennae account for a small proportion of the ground-truth map. If they are ignored in the high-level features, it will be difficult to recover these details in the final output after multi-layer fusion and decoding.

[0065] The present invention designs a series of evolutionary labels so that the network has different focuses at each layer and follows certain evolutionary rules to achieve more efficient and targeted learning. The evolutionary rules are as follows: at the fifth layer of the decoder, the supervision target is the coarsest saliency map; as the layer descends, the supervision saliency map gradually becomes more refined until the first layer of the decoder, where the supervision target becomes the original saliency ground truth map. Specifically, the present invention achieves the gradual dilution of boundary information in the supervision ground truth map of high-level features by performing maximum pooling operations layer by layer. The specific formula for the operation is as follows:

[0066]

[0067] Among them, G h represents the high-resolution ground-truth saliency map, and Maxpool(·) represents the maximum pooling operation. This evolutionary strategy increases the proportion of detailed areas in the supervised ground-truth map, allowing small objects and detailed areas unique to high-resolution drone imagery to be prioritized in the top-level features. This allows them to be retained in the final output, making them potentially repairable during the second stage of detail restoration.

[0068] S1.2: Characteristic evolution.

[0069] Simply strong supervision of the features at each layer of the network cannot fully enable the network to grasp the essence of evolution. Therefore, the present invention also designs an evolutionary structure of feature dimensions, which allows the network to more naturally complete the evolutionary learning process from coarse to fine. Under this structure, the saliency map output by the previous decoding layer can be used as a spatial attention map to process the features of the current layer:

[0070]

[0071]

[0072] Among them, ↑ represents the upsampling operation, i∈{1, 2, 3, 4}, the Fusion(·) operation represents the feature fusion module proposed by Zhao et al. (derived from Complementary trilateral decoder for fast and accurate salient object detection, in Proc. ACM MM, 2021, pp. 4967–4975), and SA(·) represents the attention operation in the spatial dimension. By implementing supervised evolution, the present invention expands the coverage of high-level features in detecting salient targets, that is, captures as many salient targets in the scene as possible. Therefore, in this case, the attention operation in the spatial dimension can effectively suppress background noise while avoiding concerns about omission or damage to salient targets.

[0073] However, this invention also needs to consider the following issue: the evolutionary directions of the two layers of features are different during the evolution process. In other words, they may occur within the object's main body or in the boundary details. Furthermore, it is well known that high-level features contain rich semantic information and are insensitive to information such as texture and color. Therefore, they have good internal consistency across various targets. In fact, during the decoding process, each evolution operation only needs to correct the boundary areas of the result.

[0074] Therefore, the present invention also proposes an evolution change constraint to ensure that the difference between two adjacent layers is only reflected in the area to be evolved, so as to maintain the good internal consistency of the high-level features. The noise in the background area of ​​two adjacent layers of features is suppressed, and then the two layers of features after the background noise is suppressed are multiplied element by element. The pixel-level multiplication operation can make the similar areas of feature representation obtain high response values, thereby obtaining internal shared features. Finally, the higher-level features are subtracted from the shared features at the pixel level to obtain the difference area features. Subsequently, through the difference supervision of the corresponding level evolution label, these differences are constrained to be reflected as much as possible on the target boundary contour, that is, the area to be evolved:

[0075]

[0076]

[0077] Among them, Represents the evolution graph of the i-th layer, conv is a convolutional layer with a kernel size of 3×3, thus, using The binary cross entropy loss between the evolution graph and this supervision is calculated as supervision.

[0078] S2: Inheritance stage (high-resolution refined model).

[0079] In the first stage, the present invention achieves detail-preserving salient object localization in low-resolution images through an evolutionary mechanism. However, due to image resolution limitations, detail representation capabilities remain insufficient. Therefore, the inheritance stage aims to achieve high-quality detail refinement while reducing computational cost. It reuses features generated in the first stage to supplement and correct details, obtaining high-quality, high-resolution saliency maps in a lightweight manner. The present invention also designs a salient attribute enhancement module (SFEM) to globally model and enhance features obtained from the encoding layer, using the saliency map from the first stage as a guide.

[0080] Specifically, the present invention inherits the up-sampled coding features obtained in the first stage and decoder features As basic features, shallow features extracted from high-resolution images are used to update features, generating enhanced high-resolution encoder features

[0081]

[0082]

[0083] Among them, SFEM(·) is the salient attribute enhancement module, Res(·) represents the coding layer residual block used to extract high-resolution features, represents the upsampled saliency map generated in the first stage, and then, the feature and is fed into the final decoder layer, which shares the same structure as the LLM and generates the final saliency prediction S out :

[0084]

[0085] S2.1: Salient attribute enhancement module.

[0086] During low-level feature extraction, especially in high-resolution images processed using conventional scale convolution, the receptive field will be more limited and insufficient to obtain global context information. Therefore, the present invention designs SFEM to reuse the saliency map generated in the first stage for feature salient attribute enhancement and global modeling.

[0087] First, the present invention determines the significant boundary area based on the prediction results of the first stage and background boundary area

[0088]

[0089]

[0090] Among them, Maxpool(·) and Minpool(·) represent maximum pooling and minimum pooling respectively. Then, according to the determined area, the input feature F is averaged by mask pooling. in Extract salient prototypes and background prototypes:

[0091]

[0092]

[0093] Where MAP(·) represents the mask average pooling operation, Represents pixel-by-pixel multiplication. Subsequently, the global information is introduced into the local feature extraction process through the prototype-based self-attention mechanism to obtain the enhanced feature F out :

[0094] K=V=concat(P S , P B ) (16);

[0095] Q=F in (17);

[0096]

[0097] Among them, d k represents the dimension of the feature, concat(·) is a spatial parallel operation, ∪ is a union operation, and softmax(·) represents the softmax activation function.

[0098] S2.2: Boundary-Detail Perception Mean Absolute Error

[0099] Since the detail areas such as butterfly antennae are relatively small in the whole image, the detection quality of these areas has little impact on traditional indicators (such as MAE). However, these areas are the key to distinguishing HRSOD tasks from ordinary SOD, so this paper proposes a new evaluation indicator boundary-detail perception mean absolute error (MAE) BD , the specific calculation method is as follows:

[0100]

[0101]

[0102] Among them, M B and M D Represent the boundary area mask and detail area mask respectively, θ is used to balance the weight of the error in the two areas, M B It can be obtained by dilating the saliency map boundary. In order to calculate M D , the present invention first divides the saliency mask G into a series of local patches (e.g., 80×80 pixels in size), and then calculates the ratio of the object perimeter to the area in each local patch. If the ratio is greater than a threshold (e.g., 0.25), the local patch is regarded as a detail area. Finally, M is obtained by dilating these local patches. D .

[0103] The method of the present invention achieves competitive performance on multiple public benchmark datasets. Figure 2 This paper presents a visualization example of the technology presented in this paper. From top to bottom, it shows a high-resolution image, a ground-truth map of high-resolution salient object detection, and a saliency map predicted by this method. The results demonstrate that the method achieves superior visual quality in many challenging scenarios, with outstanding performance in detail quality and object integrity. Furthermore, the method achieves real-time inference speed, surpassing existing methods.

[0104] Example 2:

[0105] Embodiment 2 of the present invention provides an evolved and inherited high-resolution drone image salient object detection system, comprising:

[0106] The evolution module is configured to: downsample the high-resolution drone image to obtain a downsampled image, sequentially pass the downsampled image through multiple coding layers, each coding layer corresponds to a coding feature, perform feature evolution based on the low-resolution saliency map and decoding features output by any decoding layer, and obtain the decoding features output by the next decoding layer;

[0107] The inheritance module is configured to: upsample the first encoding features corresponding to the first encoding layer and the second decoding features corresponding to the second decoding layer, update the upsampled first encoding features according to the high-resolution UAV image to obtain enhanced high-resolution encoder features, and obtain the final salient object detection results based on the enhanced high-resolution encoder features and the upsampled second decoding features.

[0108] The working methods of the evolution module and the inheritance module are the same as those provided in Example 1 and will not be repeated here.

[0109] Example 3:

[0110] Embodiment 3 of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the evolved and inherited high-resolution drone image salient target detection method as described in Embodiment 1 of the present invention.

[0111] Example 4:

[0112] Embodiment 4 of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and runnable on the processor. When the processor executes the program, it implements the steps of the evolved and inherited high-resolution drone image salient target detection method as described in Embodiment 1 of the present invention.

[0113] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0114] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0117] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0118] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. An evolutionary and inherited method for salient object detection in high-resolution UAV images, characterized by: The following processes are included: The high-resolution UAV image is downsampled to obtain a downsampled image. The downsampled image is sequentially passed through multiple coding layers, each coding layer corresponds to a coding feature. Feature evolution is performed based on the low-resolution saliency map and decoding features output by any decoding layer to obtain the decoding features output by the next decoding layer. Based on the low-resolution saliency map and decoding features output by any decoding layer, feature evolution is performed to obtain the decoding features output by the next decoding layer, including: Fuse the decoding features output by any decoding layer with the encoding features of the previous layer to obtain fused features; ； in, Represents low resolution The fusion features of the layers, Represents low resolution The encoded features of the layer, Represents low resolution +1 layer of decoded features, Represents an upsampling operation; Multiply the fusion feature with the low-resolution saliency map of the current layer and then add it to the fusion feature to obtain the decoding feature output by the decoding layer of the previous layer; ； in, Represents low resolution The decoded features of the layer, Represents low resolution The fusion features of the layers, represents the upsampling operation, Represents low resolution +1 layer decoder output saliency map, ; The first encoding features corresponding to the first encoding layer and the second decoding features corresponding to the second decoding layer are upsampled respectively. The upsampled first encoding features are updated according to the high-resolution UAV image to obtain enhanced high-resolution encoder features. The final salient object detection results are obtained based on the enhanced high-resolution encoder features and the upsampled second decoding features.

2. The method for high-resolution drone image salient object detection based on the evolution and inheritance of claim 1, wherein: The upsampled first encoder features are updated based on the high-resolution UAV image to obtain enhanced high-resolution encoder features, including: Extract high-resolution features from high-resolution drone images, combine them with the first saliency map output by the first-layer decoder to enhance salient attributes, perform pixel-by-pixel multiplication on the Sobel value of the saliency enhancement result and the upsampled first encoded feature, and then add the pixel-by-pixel multiplication result to the upsampled first encoded feature to obtain enhanced high-resolution encoder features. Among them, significant attribute enhancements are made, including: Obtaining a saliency boundary region and a background boundary region according to the first saliency map after upsampling; According to the salient boundary area, the salient prototype is extracted from the input features through mask average pooling; According to the background boundary area, the background prototype is extracted from the input features through mask average pooling; According to the feature extraction salient prototype, feature extraction background prototype and input features, the features after salient attribute enhancement are obtained.

3. The method for high-resolution drone image salient object detection based on evolution and inheritance as claimed in claim 1, wherein: At the last layer of the decoder, the supervision target is the saliency map output by the decoding layer of the last layer. At the i-th layer of the decoder, the supervision target is the saliency map output by the decoding layer of the i-th layer. Until the first layer of the decoder, the supervision target is the saliency map output by the decoding layer of the first layer. The supervised truth map corresponding to the high-resolution saliency map is downsampled to obtain the supervised truth map of the first layer, and the supervised truth map of the i-1th layer is subjected to a maximum pooling operation to obtain the supervised truth map of the i-th layer, where i is a positive integer greater than 1.

4. The method for high-resolution drone image salient object detection based on evolution and inheritance as claimed in claim 1, wherein: By constraining evolutionary changes, the difference between two adjacent layers is only reflected in the area to be evolved, including: The encoding process gradually extracts high-level features from shallow features, i gradually increases from 1 to 5, and the decoding process advances from high-level to low-level, i gradually decreases from 5 to 1. In the decoding process, the previous layer of the i-th layer is the i+1-th layer; Use the saliency map output by the previous decoding step , decode the features of the current layer and higher-level decoding features The noise in the background area is suppressed, and then the two layers of features after suppressing the background noise are multiplied by the corresponding positions to obtain the internal common features; Decoding features at a higher level The difference region features are obtained by pixel-level subtraction from the common features, and the difference supervision of the corresponding hierarchical evolution labels is used to constrain the difference region features to only be reflected on the target boundary contour.

5. The method for high-resolution drone image salient object detection based on evolution and inheritance as claimed in claim 1, wherein: The boundary-detail-aware mean absolute error (MAE) is used to evaluate the salient object detection results. The MAE is calculated based on the boundary region mask, detail region mask, error weight, saliency mask, and background mask.

6. An evolutionary and inherited high-resolution drone image salient object detection system, characterized by: include: The evolution module is configured to: downsample the high-resolution drone image to obtain a downsampled image, sequentially pass the downsampled image through multiple coding layers, each coding layer corresponds to a coding feature, perform feature evolution based on the low-resolution saliency map and decoding features output by any decoding layer, and obtain the decoding features output by the previous decoding layer; Based on the low-resolution saliency map and decoding features output by any decoding layer, feature evolution is performed to obtain the decoding features output by the next decoding layer, including: Fuse the decoding features output by any decoding layer with the encoding features of the previous layer to obtain fused features; ； in, Represents low resolution The fusion features of the layers, Represents low resolution The encoded features of the layer, Represents low resolution +1 layer of decoded features, Represents an upsampling operation; Multiply the fusion feature with the low-resolution saliency map of the current layer and then add it to the fusion feature to obtain the decoding feature output by the decoding layer of the previous layer; ； in, Represents low resolution The decoded features of the layer, Represents low resolution The fusion features of the layers, represents the upsampling operation, Represents low resolution +1 layer decoder output saliency map, ; The inheritance module is configured to: upsample the first encoding features corresponding to the first encoding layer and the second decoding features corresponding to the second decoding layer, update the upsampled first encoding features according to the high-resolution UAV image to obtain enhanced high-resolution encoder features, and obtain the final salient object detection results based on the enhanced high-resolution encoder features and the upsampled second decoding features.

7. The evolved and inherited high-resolution drone image salient object detection system according to claim 6, characterized in that: In the inheritance module, the upsampled first encoding features are updated according to the high-resolution UAV image to obtain enhanced high-resolution encoder features, including: Extract high-resolution features from high-resolution drone images, combine them with the first saliency map output by the first-layer decoder to enhance salient attributes, perform pixel-by-pixel multiplication on the Sobel value of the saliency enhancement result and the upsampled first encoded feature, and then add the pixel-by-pixel multiplication result to the upsampled first encoded feature to obtain enhanced high-resolution encoder features. Among them, significant attribute enhancements are made, including: Obtaining a saliency boundary region and a background boundary region according to the first saliency map after upsampling; According to the salient boundary area, the salient prototype is extracted from the input features through mask average pooling; According to the background boundary area, the background prototype is extracted from the input features through mask average pooling; According to the feature extraction salient prototype, feature extraction background prototype and input features, the features after salient attribute enhancement are obtained.

8. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps of the evolved and inherited high-resolution drone image salient target detection method as described in any one of claims 1 to 5 are implemented.

9. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the steps in the evolved and inherited high-resolution drone image salient target detection method as described in any one of claims 1-5.

Images