A pixel-level satellite remote sensing small target recognition method and system
By performing segmentation, initial screening, morphological detection, and fine screening on satellite remote sensing images, and combining RESNET and Darknet models, the problems of large satellite remote sensing image size and insufficient computing resources are solved, achieving efficient and accurate identification of small satellite remote sensing targets under low computing power.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-09
- Publication Date
- 2026-03-17
AI Technical Summary
Existing satellite remote sensing images have large imaging sizes, requiring enormous computing resources for real-time identification, and the identification accuracy is low. Limited by the hardware environment of satellite payloads, it is difficult to achieve efficient satellite remote sensing identification of small targets.
A pixel-level satellite remote sensing small target recognition method is adopted. Through steps such as image segmentation, initial screening, target morphology detection, regional target recognition and fine screening, the method utilizes the RESNET classification network, the Darknet model and the sliding window small target recognition algorithm to reduce the amount of data and optimize computing resources, thereby achieving efficient recognition.
This paper presents a highly reliable satellite remote sensing image recognition algorithm with low computational cost, which can achieve real-time and accurate recognition of small satellite remote sensing targets in low-Earth orbit satellite environments, meeting the needs of real-time monitoring.
Smart Images

Figure CN116630819B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of image processing and aerospace video target recognition technology, and in particular to a pixel-level satellite remote sensing small target recognition method and system. Background Technology
[0002] Object recognition involves identifying all objects of interest in an image and determining their category and location. The current mainstream technology trend is object recognition based on deep learning algorithms. This primarily focuses on semantic understanding of the target image, extracting edge features through convolutional integral networks. As the network deepens, it gradually extracts deeper target features, detecting the target location by understanding the image's semantics.
[0003] The main challenge of existing spaceborne target recognition algorithms is the small size of the targets. Generally speaking, low-orbit satellites are 1,000 to 5,000 km above the ground. Due to the extremely long distance between the satellite and the ground, the target of interest is only 2×2 pixels to 8×8 pixels in the satellite camera image, and the target is relatively blurry with a low signal-to-noise ratio, resulting in very low detection accuracy for such targets.
[0004] Secondly, satellites use a pushbroom method for each imaging process, resulting in extremely large image sizes. They typically generate over 500 1K images per second, thus requiring enormous computational resources for real-time identification. Furthermore, the limited satellite payload prevents the deployment of large-scale artificial intelligence algorithm hardware environments, which also impacts detection speed. Summary of the Invention
[0005] Based on the above analysis, the embodiments of the present invention aim to provide a pixel-level satellite remote sensing small target identification method and system to solve the problems of large imaging size of satellite remote sensing images, huge computational resources required for real-time identification, and low identification accuracy in the prior art.
[0006] On one hand, embodiments of the present invention provide a pixel-level satellite remote sensing small target identification method, including:
[0007] Acquire satellite remote sensing images and divide the images into several blocks;
[0008] The segmented blocks are initially screened to obtain background blocks and target blocks;
[0009] Target morphology detection is performed on the background blocks to retain the background blocks that contain small targets to be identified for further screening;
[0010] Regional target recognition is performed on the target block to obtain a regional target image;
[0011] The regional target image and the background block to be screened are screened using a sliding window-based small target recognition algorithm to identify small targets.
[0012] Furthermore, satellite remote sensing images are obtained and divided into several blocks, including:
[0013] Receive video streams from satellite cameras in line scan mode via the CXP interface;
[0014] Decode the satellite remote sensing image data from the video stream according to the CXP protocol;
[0015] The image data is stored in memory. After storing several rows, the data is read and cut into several rows of image streams. The resulting image stream is adaptively divided into N×N blocks.
[0016] Furthermore, the segmented blocks are initially screened to obtain background blocks and target blocks, including:
[0017] The segmented blocks are fed into the RESNET classification network's recognition algorithm for initial screening. Output 0 is considered background, and output 1 is considered foreground, resulting in background blocks and target blocks. In this process, multiple images of the regional target are pre-acquired and labeled as category 1, and multiple images including the background are acquired and labeled as category 0. The RESNET classification network is then trained.
[0018] Furthermore, target morphological detection is performed on the background blocks to retain the background blocks to be further screened that contain the small targets to be identified, including:
[0019] Background blocks are compared with fixed-storage typical images using pixel matching to exclude background areas with low similarity. The fixed-storage typical images refer to images obtained by storing several images of small targets to be identified on satellites. During the comparison, no distinction is made between real targets and target-like objects; both are treated as background blocks to be further screened.
[0020] Random noise interference is filtered out by a low-pass filter, and the background block that has been successfully matched and filtered out of random noise interference is taken as the background block to be finely screened.
[0021] Furthermore, regional target recognition is performed on the target block to obtain a regional target image, including:
[0022] The resolution of the target block is normalized to the standard aspect ratio of the display, and the normalized image block is downsampled.
[0023] The downsampled target blocks are then used to perform regional target recognition using a pre-trained regional target recognition module. First, the Darknet model is trained using the publicly available dataset Visdrone. Then, the bounding boxes and types of regional targets are manually labeled on the satellite camera data and fed into the Darknet module for further training, resulting in the regional target recognition module. When the trained Darknet module performs poorly, targeted learning is performed again on the poorly performing test cases to optimize the regional target recognition module.
[0024] If the target is successfully identified, the category and candidate box are output; otherwise, image enhancement processing is performed on the downsampled target block and target identification is performed on the next downsampled target block.
[0025] The original image is cropped based on the output category and candidate box location information to obtain a regional target image.
[0026] Furthermore, downsampling is performed on the normalized image blocks, including:
[0027] The downsampling factor is determined by the ratio of the input image pixel size (w / N, h / N) to the pixel size of the regional target.
[0028] Dw={6+2e^[w / (N*warea)]} / {1+e^[w / (N*warea)]}
[0029] Dh={6+2e^[h / (N*harea)]} / {1+e^[h / (N*harea)]}
[0030] D = max(Dw, Dh)
[0031] Where Dw and Dh are intermediate calculation results of the downsampling ratios in the width and height directions, respectively. The larger of Dw and Dh is taken as the actual downsampling ratio D in the width and height directions. w and h are the pixel sizes of the original image, N is the number of regions segmented in the width or height direction by the above adaptive segmentation, and warea and harea are the average pixel sizes of the regional targets.
[0032] Furthermore, if target recognition is successful, the category and candidate bounding box are output; otherwise, image enhancement processing is performed on the downsampled target block, and target recognition is performed on the next downsampled target block, including:
[0033] If the target recognition module fails to recognize the target due to unclear image, then request the import of the image enhancement module. Use the image enhancement module to adjust the brightness and contrast of the unclear image and then re-input it into the regional target recognition module for recognition.
[0034] Furthermore, the regional target image and the background block to be finely screened are further refined using a sliding window-based small target recognition algorithm to identify small targets, including:
[0035] Multiple images to be detected are obtained by using the sliding window method to extract the regional target image and the background block to be screened.
[0036] Each of the above images to be detected is processed through a residual structure;
[0037] The residual structure is that each image to be detected is sequentially passed through (1×1,n / 4), (3×3,n / 4), and (1×1,n) convolutional kernels to output semantic information; among them, an activation operation is required after each convolution to differentiate the information of that convolution.
[0038] The image to be detected is added to the output semantic information result to obtain the first layer feature map;
[0039] Repeat the above process M times to obtain the feature map of the Mth layer;
[0040] The Mth layer feature map is fed into the RPN network. The RPN network extracts the abstract features of the Mth layer feature map and compares them with several known, fixed-size anchor boxes to generate the focal region and output the M+1th layer feature map. The M+1th layer feature map is used to distinguish the foreground region and the background region to obtain the position of the small target.
[0041] The feature map of the (M+1)th layer is classified using a fully connected layer and a softmax layer to determine the category of small objects.
[0042] Here, (3×3, n / 4) represents n / 4 3×3 convolution kernels. Each 3×3 convolution is used to extract features of different dimensions of the image, and the 1×1 convolution is used to control the number of dimensions of the image.
[0043] Furthermore, after detecting the location coordinates of the small target, the output recognition result with the target bounding box and coordinate information are fused into the original image as the final detection result.
[0044] On the other hand, embodiments of the present invention provide a pixel-level satellite remote sensing small target identification system, comprising:
[0045] The image segmentation module is used to segment the acquired satellite remote sensing image into several blocks;
[0046] The initial screening module is used to perform initial screening on the segmented blocks to obtain background blocks and target blocks.
[0047] The target morphology detection module is used to perform target morphology detection on the background block and retain the background block to be further screened that contains the small target to be identified.
[0048] A regional target recognition module is used to perform regional target recognition on the target block to obtain a regional target image;
[0049] The fine screening module is used to fine screen the regional target image and the background block to be fine screened using a sliding window-based small target recognition algorithm to identify small targets.
[0050] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:
[0051] 1. This invention provides a low-computing-power satellite remote sensing image recognition algorithm by processing satellite camera data into image streams and then adaptively dividing the resulting image streams into N×N blocks, initially screening and discarding background blocks without targets, downsampling the data entering the target blocks to further reduce the data volume, and discarding areas without targets after identifying regional target blocks.
[0052] 2. This invention provides a highly reliable satellite remote sensing image recognition algorithm by utilizing a series of processing methods including initial screening, regional target identification, and fine screening.
[0053] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description
[0054] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.
[0055] Figure 1 This is a flowchart illustrating a pixel-level satellite remote sensing small target identification method according to an embodiment of the present invention.
[0056] Figure 2 This is a flowchart illustrating the regional target recognition process of a pixel-level satellite remote sensing small target recognition method according to an embodiment of the present invention.
[0057] Figure 3 This is a residual structure used in the fine screening module of a pixel-level satellite remote sensing small target identification method according to an embodiment of the present invention. Detailed Implementation
[0058] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.
[0059] A specific embodiment of the present invention discloses a pixel-level satellite remote sensing small target identification method, such as... Figure 1 As shown, the method includes the following steps:
[0060] Step S1: Acquire satellite remote sensing images and divide the images into several blocks. This specifically includes steps S11-S12.
[0061] Step S11: The satellite camera transmits the video stream through the CXP interface in a line scan mode, and decodes it according to the CXP protocol to extract the image line data.
[0062] Step S12: Store the decoded and extracted row image data into memory. After storing a certain number of rows, read the data from memory at the specified address. First, cut it into an image stream. The resulting image stream is adaptively divided into N×N regions, and several blocks are sent to the initial screening module in sequence.
[0063] Each segmented image stream region is w pixels long and h pixels wide, and the resulting blocks are w / N pixels long and h / N pixels wide, where w and h are the length and width of the segmented image in pixels, respectively. This is Figure 1 Image preprocessing module.
[0064] Step S2: Perform initial screening on the segmented blocks to obtain background blocks and target blocks.
[0065] Specifically, the segmented blocks are fed into a trained RESNET classification network. An output of 0 is considered a background block, and an output of 1 is identified as a foreground block, i.e., a target block. Here, a target block refers to a block that is considered to potentially contain a target after excluding background blocks.
[0066] The initial screening module is designed to quickly perform real-time screening of the input image stream. After screening, valuable target blocks (such as ports, airports, parking lots, etc.) are sent to the regional target recognition module for secondary screening, while worthless blocks (such as backgrounds such as clouds, land, and ocean) are sent to the morphological detection module.
[0067] In this process, multiple images of regional targets such as ports, airports, and parking lots are obtained in advance and labeled as category 1. Multiple images of backgrounds such as clouds, land, and ocean are obtained and labeled as category 0. The RESNET classification network is then trained to obtain the trained RESNET classification network.
[0068] Step S3: Perform target morphological detection on the background blocks and retain the background blocks containing the small targets to be identified for further screening. Specifically, this includes steps S31-S32.
[0069] Step S31: Compare the background blocks obtained in step 2 with the fixed-stored typical images pixel by pixel to exclude background areas with low similarity.
[0070] Assume the scene model of the target can be described as:
[0071] f(x,y,k)=f r (x,y,k)+f B (x,y,k)+f I (x,y,k)+f o (x,y,k)
[0072] Where (x, y, k) represents the x and y coordinates of the target in k image layers, f r Let (x,y,k) be the target, and f B (x,y,k) is the background, f I (x,y,k) represents random noise interference, f o (x,y,k) represents similar target interference.
[0073] Fixed-storage typical images refer to images obtained by storing several acquired images of small targets to be identified on a satellite. Examples include vehicles, airplanes, and ships. The background block to be detected is compared pixel-by-pixel with the fixed-storage typical images, and images with low similarity are excluded. B The region (x, y, k) contains the real target, noise interference similar to the target (camera dirt, etc.), and other objects similar to the target (such as birds, trees, etc.). In this step, the real target f... r (x,y,k) and class target f o (x,y,k) are not distinguished.
[0074] The comparison process is as follows: the background block is obtained by sliding window method and the detection area is scaled to the same size as the fixed storage of typical images, then converted into grayscale images. After the background is filtered out by global threshold segmentation method, the Euclidean distance of the corresponding pixels is calculated. If the result is less than a fixed value, the match is considered successful.
[0075] Step S32: Filter out random noise interference using a low-pass filter.
[0076] noise f I (x, y, k) are randomly distributed in the image space and can be approximated as following a Gaussian distribution, while the image background f B(x, y, k) exhibits strong correlation in the image space, and from a frequency domain perspective, it lies in the low-frequency region, while noise resides in the high-frequency region. Therefore, a low-pass filter is used to remove f. I (x,y,k).
[0077] Step S31: Match successful and filter out f I After (x,y,k), the background block is sent to the fine screening module as the background block to be finely screened to execute step S5. If the matching fails, the block is discarded.
[0078] Step S4: Perform regional target recognition on the target block to obtain a regional target image. This specifically includes steps S41-S44.
[0079] Step S41: Normalize the resolution of the target block to the standard aspect ratio of the display, and downsample the normalized image block.
[0080] This step is used to perform regional target identification on the blocks identified as containing targets in step S2. Regional target identification utilizes a large target fast identification network based on downsampling. Its input is the area, or target block, initially identified by the initial screening module as containing regional targets. The identified targets are relatively large regional targets, mainly airports, ports, parking lots, etc. Due to the large target imaging size, the input image stream can be downsampled at a significant rate, reducing processing latency.
[0081] The pixel size of the regional target is in the range of 150×150 pixels to 500×500 pixels. The downsampling ratio is determined by the ratio of the input image pixel size (w / N, h / N) to the pixel size of the regional target.
[0082] Dw={6+2e^[w / (N*warea)]} / {1+e^[w / (N*warea)]}
[0083] Dh={6+2e^[h / (N*harea)]} / {1+e^[h / (N*harea)]}
[0084] D = max(Dw, Dh)
[0085] Where Dw and Dh are intermediate calculation results of the downsampling ratio in the width and height directions, the larger of Dw and Dh is taken as the actual downsampling ratio D in the width and height directions, w and h are the pixel size of the original image, N is the number of segments in the above adaptive segmentation, and warea and harea are the average pixel size of the regional target, which can be obtained by statistically analyzing the target size through some collected images.
[0086] Generally speaking, the downsampling factor can be controlled between 2 and 4 times, that is, the input size is (w / 4N, h / 4N) to (w / 2N, h / 2N).
[0087] Step S42: Perform regional target recognition on the downsampled target block using the trained regional target recognition module.
[0088] The regional target recognition module is built based on the open-source Darknet model, a primary detection network that can directly output target type and location based on input. The overall training process is as follows:
[0089] First, the dataset visdrone was used to train it for 300 epochs to give it general recognition capabilities for common targets and basic real-world scene understanding capabilities.
[0090] Then, the bounding boxes and types of regional targets in the shooting data transmitted back by the satellite camera are manually labeled and sent into the module for further training for 300 epochs, so that it can adapt to the imaging characteristics of specific satellite cameras.
[0091] The trained network is then integrated into the overall detection network for testing (i.e., connecting the output of the initial screening module to the input of the fine screening module). The detection performance is observed manually, and test cases with poor performance are subjected to 100 epochs of targeted learning.
[0092] Where epoch refers to the round.
[0093] Step S43: If the regional target recognition is successful, output the category and candidate box of the regional target; otherwise, perform image enhancement processing on the downsampled target block and perform regional target recognition on the next downsampled target block.
[0094] Step S44: Crop the original image according to the output category and candidate box position information to obtain the regional target image.
[0095] Step S5: The regional target image and the background block to be screened are screened using a sliding window-based small target recognition algorithm to identify small targets.
[0096] The fine screening module extracts multiple images to be detected (pixel size w1×h1, n is the number of channels) from the regional target image and the background block to be screened using the sliding window method.
[0097] The images to be detected are sequentially processed through convolutional kernels of (1×1, n / 4), (3×3, n / 4), and (1×1, n), as follows: Figure 3As shown, (3×3, n / 4) represents n / 4 3×3 convolutional kernels. Each 3×3 convolution is used to extract features from different dimensions of the image. 1×1 convolutions are used to control the number of dimensions of the image to better accommodate the features extracted by the 3×3 kernels (each dimension contains feature information at different levels, such as edges, colors, and textures). After each convolution, an activation (ReLU) operation is required to differentiate the information from that convolution. Simultaneously, the input image to be detected is directly connected from the input to the output and added to the semantic information result after the three convolutional differentiation operations to obtain the first layer feature map. By fully fusing the output and input, the output feature map may lose some input information. The above process is repeated M times to obtain the Mth layer feature map.
[0098] After obtaining the Mth layer feature map, it is fed into the RPN network. The RPN network extracts the abstract features of the Mth layer feature map and compares them with several known, fixed-size anchor boxes to generate the focal region and output the M+1th layer feature map. The M+1th layer feature map mainly distinguishes the original image into the foreground region (the target region of interest) and the background region. At this point, the position of the small target has been determined.
[0099] Finally, the feature map of the (M+1)th layer is classified through a fully connected layer and a softmax layer to determine the category of small objects.
[0100] The overall process employs a deep model (M+1 layers), requiring significant computational resources. However, because the images to be detected are regional target images and the output images of background blocks to be refined are extracted using the sliding window method, the image pixel size is relatively small, thus enabling high speed. Its main functions include detecting aircraft inside airports, ships inside ports, and large vehicles inside parking lots, with target pixel sizes ranging from 2×2 to 8×8 pixels.
[0101] Step S6: After obtaining the position coordinates of the small target through the above network algorithm, the output recognition result with the target box and coordinate information are fused into the original image as the final detection result.
[0102] Compared with existing technologies, this embodiment provides a pixel-level satellite remote sensing small target recognition method, which solves the problems that due to the long distance between low-orbit satellites and the ground, the target of interest is only imaged at the pixel level in the satellite camera, the target is relatively blurry, the signal-to-noise ratio is low, and the target recognition algorithm capability is limited by hardware conditions in most spaceborne scenarios. It is a satellite remote sensing image recognition algorithm with low computing power and high reliability.
[0103] On the other hand, another embodiment of the present invention provides a pixel-level satellite remote sensing small target identification system, comprising:
[0104] The image segmentation module is used to segment the acquired satellite remote sensing image into several blocks;
[0105] The initial screening module is used to perform initial screening on the segmented blocks to obtain background blocks and target blocks.
[0106] The target morphology detection module is used to perform target morphology detection on the background block and retain the background block to be further screened that contains the small target to be identified.
[0107] A regional target recognition module is used to perform regional target recognition on the target block to obtain a regional target image;
[0108] The fine screening module is used to fine screen the regional target image and the background block to be fine screened using a sliding window-based small target recognition algorithm to identify small targets.
[0109] A commercial remote sensing satellite uses this algorithm to achieve real-time monitoring of remote sensing targets:
[0110] The onboard camera continuously acquires data to be identified within a 15×70km radius for approximately 8–10 seconds. The video stream is transmitted via the CXP interface using a line scan method. It first undergoes preprocessing operations such as algorithm stitching, decoding, and enhancement to generate a 1280×1280 pixel image stream, an 8-bit grayscale image, at 500 frames per second.
[0111] The initial screening module divides the image stream into 8×8 regions, each region being 160×160 pixels in size, using a ResNet34 network;
[0112] The regional target recognition module downsamples the target data output by the initial screening module by 4 times to obtain 40×40 pixels;
[0113] The input size of the fine screening module is the detection result of the regional target recognition and screening module, and the output result is 2×2 to 8×8 pixels.
[0114] The hardware resources used for inference are eight Huawei Atles200 chips.
[0115] Example execution results:
[0116] Airport aircraft inspection: detection rate better than 90%, false alarm rate 25%; Ship inspection: length over 50m, detection rate better than 90%, false alarm rate 25%; Large vehicle inspection: detection rate better than 80%, false alarm rate 25%.
[0117] A single frame takes 1.8ms, which meets the real-time monitoring requirement of 2ms (500 frames / s).
[0118] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.
[0119] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.
Claims
1. A pixel-level satellite remote sensing small target identification method, characterized in that, The method comprises the following steps: acquire satellite remote sensing images, and divide the images into a plurality of blocks; perform preliminary screening on the divided blocks to obtain background blocks and target blocks; perform target morphological detection on the background blocks to retain background blocks containing small targets to be identified, comprising: perform pixel matching comparison between the background blocks and fixedly stored typical images to exclude background areas with low similarity; wherein the fixedly stored typical images refer to images obtained by storing a plurality of collected images of small targets to be identified on a satellite; during the comparison, real targets and similar targets are not distinguished, and are all regarded as background blocks to be finely screened; filter out random noise interference through a low-pass filter, and take the background blocks after successful matching and filtering out of random noise interference as the background blocks to be finely screened; perform regional target identification on the target blocks to obtain regional target images; perform fine screening on the regional target images and the background blocks to be finely screened using a small target identification algorithm based on a sliding window to identify small targets.
2. The method according to claim 1, wherein, Obtain satellite remote sensing images, and divide the images into a plurality of blocks, comprising: receive a video stream obtained by a satellite camera in a line scanning manner through a CXP interface; decode the video stream according to a CXP protocol to extract satellite remote sensing image data from the video stream; store the image data in a memory, start reading the data and cut the data into a picture stream after storing a plurality of lines, and adaptively divide the obtained picture stream into N×N blocks; wherein N is the number of regions divided in the width or height direction in the adaptive division.
3. The method according to claim 1 or 2, characterized in that, Perform preliminary screening on the divided blocks to obtain background blocks and target blocks, comprising: perform preliminary screening on the divided blocks by inputting the blocks into an identification algorithm of a RESNET classification network, output 0 as background, and output 1 as foreground to obtain background blocks and target blocks; wherein a plurality of pictures of regional targets are obtained in advance, and are labeled as class 1, a plurality of pictures including background are obtained, and are labeled as class 0, and the RESNET classification network is trained.
4. The method according to claim 2, wherein, Perform regional target identification on the target blocks to obtain regional target images, comprising: normalize the resolution of the target blocks to a standard aspect ratio of a display, and downsample the normalized image blocks; perform regional target identification on the downsampled target blocks using a trained regional target identification module; wherein a darknet model is first trained using a public data set visdrone; then, the shooting data returned by the satellite camera is manually labeled with the bounding boxes and types of regional targets, and is input into the darknet module for further training to obtain the regional target identification module; when the trained darknet module has poor detection effect, targeted learning is performed again for test cases with poor effect to optimize the regional target identification module; if the target identification is successful, output the class and the candidate box; otherwise, perform image enhancement processing on the downsampled target blocks and perform target identification on the next downsampled target blocks; cut the original image according to the output class and candidate box position information to obtain a regional target image.
5. The method according to claim 4, wherein, Downsample the normalized image blocks, comprising: The down-sampling rate is determined by the ratio of the pixel size of the input image (w / N, h / N) to the pixel size of the regional target; Dw = {6 + 2e^[w / (N*warea)]} / {1 + e^[w / (N*warea)]} Dh = {6 + 2e^[h / (N*harea)]} / {1 + e^[h / (N*harea)]} D = max(Dw, Dh) Wherein, Dw, Dh are the intermediate calculation results of the down-sampling rate in the width and height directions, and the larger one of Dw and Dh is taken as the actual down-sampling rate D in the width and height directions, and w, h are the pixel size of the original image, and warea, harea are the average pixel size of the regional target.
6. The method according to claim 4, wherein, If the target recognition is successful, the category and the candidate box are outputted; Otherwise, the image enhancement processing is performed on the down-sampled target block, and the target recognition is performed on the next down-sampled target block, including: If the target recognition fails in the regional target recognition module, it is caused by the unclear image, and the image enhancement module is applied to adjust the brightness and contrast of the unclear image and then input the image into the regional target recognition module for recognition.
7. The method according to claim 1, wherein, The regional target image and the background block to be screened are screened by using a small target recognition algorithm based on a sliding window, and a small target is recognized, including: The regional target image and the background block to be screened are extracted by using a sliding window method to obtain a plurality of detection pictures; Each detection picture is subjected to a residual structure; The residual structure is that each detection picture is sequentially subjected to (1x1, n / 4), (3x3, n / 4), and (1x1, n) convolution kernel to output semantic information; wherein, the information of each convolution is differentiated by an activation operation. The detection picture and the output semantic information are added to obtain a first layer feature map; The above process is repeated M times to obtain an Mth layer feature map; The Mth layer feature map is input into an RPN network, the RPN network compares the abstract features of the Mth layer feature map with a plurality of known and fixed size anchor boxes to generate a focus region, and outputs an M+1th layer feature map, the M+1th layer feature map is used to distinguish the foreground region and the background region, and the position of the small target is obtained; The M+1th layer feature map is classified by using a full connection layer and a softmax layer to determine the category of the small target. Wherein, (3x3, n / 4) represents n / 4 3x3 convolution kernels, each 3x3 convolution is used to extract the features of images in different dimensions, and the 1x1 convolution is used to control the number of dimensions of the image.
8. The method according to claim 7, wherein, Further comprising: After the position coordinates of the small target are detected, the recognition result with the target box and the coordinate information are fused into the original image as the final detection result.
9. A pixel-level satellite remote sensing small target identification system, characterized in that, Including: An image segmentation module is configured to segment a plurality of blocks based on the acquired satellite remote sensing image; An initial screening module is configured to perform initial screening on the segmented blocks to obtain background blocks and target blocks. If the target recognition fails in the regional target recognition module, it is caused by the unclear image, and the image enhancement module is applied to adjust the brightness and contrast of the unclear image and then input the image into the regional target recognition module for recognition. The regional target image and the background block to be screened are screened by using a small target recognition algorithm based on a sliding window, and a small target is recognized, including: The regional target image and the background block to be screened are extracted by using a sliding window method to obtain a plurality of detection pictures; Each detection picture is subjected to a residual structure; The residual structure is that each detection picture is sequentially subjected to (1x1, n / 4), (3x3, n / 4), and (1x1, n) convolution kernel to output semantic information; wherein, the information of each convolution is differentiated by an activation operation. The detection picture and the output semantic information are added to obtain a first layer feature map; The above process is repeated M times to obtain an Mth layer feature map; The Mth layer feature map is input into an RPN network, the RPN network compares the abstract features of the Mth layer feature map with a plurality of known and fixed size anchor boxes to generate a focus region, and outputs an M+1th layer feature map, the M+1th layer feature map is used to distinguish the foreground region and the background region, and the position of the small target is obtained; The M+1th layer feature map is classified by using a full connection layer and a softmax layer to determine the category of the small target. Wherein, (3x3, n / 4) represents n / 4 3x3 convolution kernels, each 3x3 convolution is used to extract the features of images in different dimensions, and the 1x1 convolution is used to control the number of dimensions of the image. Further comprising: After the position coordinates of the small target are detected, the recognition result with the target box and the coordinate information are fused into the original image as the final detection result. Including: An image segmentation module is configured to segment a plurality of blocks based on the acquired satellite remote sensing image; The target morphological detection module is configured to perform target morphological detection on the background blocks, and retain to-be-precisely-screened background blocks containing small targets to be recognized, including: performing pixel matching comparison between the background blocks and fixedly stored typical images, and excluding background regions with low similarity; wherein the fixedly stored typical images refer to images obtained by storing a plurality of images of the small targets to be recognized on the satellite; in the comparison, real targets and similar targets are not distinguished, and are all regarded as to-be-precisely-screened background blocks; random noise interference is filtered out through a low-pass filter, and the background blocks that are matched successfully and have the random noise interference filtered out are taken as the to-be-precisely-screened background blocks; The regional target recognition module is configured to perform regional target recognition on the target blocks, and obtain a regional target image. The precise screening module is configured to perform precise screening on the regional target image and the to-be-precisely-screened background blocks by using a small target recognition algorithm based on a sliding window, and recognize small targets.
Citation Information
Patent Citations
Remote sensing image target detection method based on rotating region generation network
CN112395975A
Remote sensing image target extraction method based on deep neural network
CN112712500A
Large-breadth SAR image ship target detection and identification method based on fine segmentation
CN113409325A