A method and system for crowd recognition in remote sensing satellite images taking into account spatial attention

Abstract

The present invention relates to a method and system for crowd recognition in remote sensing satellite images that takes spatial attention into account. The present invention constructs a SACRSI network, and inputs a positioning map obtained by interpreting and processing remote sensing satellite images, and a remote sensing satellite image set into the network to obtain first-order and second-order positioning heads after processing. The focal loss function is used to calculate the error between the positioning map and the first-order and second-order positioning heads, and first-order and second-order loss information is obtained to optimize the SACRSI network. The second-order positioning head output by the trained SACRSI network is then used, and a local maximum detection strategy is adopted to convert the second-order positioning head into the position coordinates of each person, thereby completing crowd recognition for remote sensing satellite images. Compared with the existing technology, the present invention has the advantages of high positioning accuracy, can effectively complete crowd detection of remote sensing images, strong generalization ability, and a wide range of applications.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image recognition, and in particular to a method and system for recognizing people in remote sensing satellite images taking into account spatial attention. Background Art

[0002] In recent years, the increased frequency of crowd activity in public spaces such as commercial centers, squares, and transportation hubs has fueled economic prosperity. However, this activity also poses certain public safety risks, such as traffic congestion, stampedes, social security incidents, and public health crises, threatening people's lives and property. Therefore, developing methods that can accurately identify the location of people in specific locations—crowd detection—is crucial for mitigating these risks.

[0003] In the early days of crowd detection, the primary focus was on images captured by surveillance or fixed cameras. These cameras provide images from a variety of viewing angles and lighting conditions, but they only capture the number of people in a local area, limiting the scope of analysis to the camera's coverage area. In recent years, with the advancements in aviation and drone platforms, aerial imagery has been used for crowd detection, significantly enhancing the ability to monitor and analyze crowd dynamics over large areas. However, aerial imagery is typically captured for specific missions, resulting in low temporal resolution and limited ability to analyze historical patterns and trace back events.

[0004] With the development of remote sensing technology, high-temporal and spatial resolution images are increasingly used to detect small objects. Remote sensing images provide a wider monitoring range and fixed time intervals, improving the efficiency of large-scale monitoring and the ability to analyze historical patterns in the distribution of land objects.

[0005] Currently, numerous methods and techniques are used for crowd recognition in images, such as Point to Point Network (P2PNet), Point Query Transformer (PET), Generalized Loss Function for Crowd Counting and Localization (GLC), Focal Inverse Distance Transform Map (FIDTM), and Multiscale Crowd Counting and Localization (MCCL). However, these methods fall short in meeting the requirements for crowd recognition in remote sensing images. Crowd signal characteristics in remote sensing images are weak, and conventional crowd detection algorithms can lead to target information loss, which increases detection errors. Summary of the Invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a method and system for crowd recognition in remote sensing satellite images taking into account spatial attention.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] According to one aspect of the present invention, a method for identifying people in remote sensing satellite images taking into account spatial attention is provided, the method comprising the following steps:

[0009] S1. Select and fuse remote sensing satellite images to generate a remote sensing satellite image set, and then interpret and process the remote sensing satellite image set to obtain a positioning map;

[0010] S2, build the SACRSI network, input the remote sensing satellite image set into the network for processing, and output the first-order and second-order positioning heads;

[0011] S3. Calculate the error between the positioning map and the first-order and second-order positioning heads through the focal loss function to obtain the first-order and second-order loss information to optimize the SACRSI network and output the trained SACRSI network.

[0012] S4. Input the remote sensing satellite image to be processed into the trained SACRSI network, output the second-order positioning head, and adopt the local maximum detection strategy to convert the second-order positioning head into the position coordinates of each person, thereby completing the crowd recognition for remote sensing satellite images.

[0013] As a preferred technical solution, the specific steps of selecting and fusing remote sensing satellite images in S1 are: selecting an image with a spatial resolution of n meters in the panchromatic band, and using the surface-to-point regression kriging method to fuse the multispectral band of the image with the panchromatic band, where n≤0.3.

[0014] As a preferred technical solution, the specific steps for interpreting and processing the remote sensing satellite image set in S1 are:

[0015] First, the image mosaicking and cropping method is used to select areas with a large number of people. Secondly, the historical images of Google Earth corresponding to the area are used as auxiliary data, and the positions of individual people are annotated in the form of point annotations to obtain annotated image data. Finally, the annotated image data is downsampled and the focal inverse distance transform method is used to generate a positioning map.

[0016] As a preferred technical solution, the focal inverse distance transformation formula is:

[0017]

[0018] Where I represents the generated localization map; α and β are weight factors; C is a constant; and P(x,y) represents the distance between the pixel at (x,y) coordinates and its nearest annotated pixel.

[0019] As a preferred technical solution, the SACRSI network in S2 includes a pre-processing module and an Hourglass network, wherein the pre-processing module includes a residual layer and a spatial attention layer; the Hourglass network includes three branches, the first branch is a jump connection module, which outputs the first branch feature F1', the second branch is a feature extraction module that outputs the second branch feature F2', and the third branch is an intermediate supervision module that outputs the positioning head and the third branch feature F3'; the remote sensing satellite image obtains the first-order output feature and the first-order positioning head after passing through the SACRSI network, wherein the feature information is obtained by fusion of the outputs of the three branches, and its first-order output features are input into the SACRSI network to output the second-order positioning head.

[0020] As a preferred technical solution, the working process of the SACRSI network includes:

[0021] S21, input the remote sensing satellite image into the residual layer for processing to obtain a feature map F;

[0022] S22, input the feature map F into the spatial attention layer to obtain a feature map F' with spatial attention;

[0023] S23, input the feature map F' with spatial attention into the three branches of the Hourglass network respectively, and output three branch features F1', F2' and F3' and the positioning head respectively;

[0024] S24. Fuse the three branch features to obtain the output features and the positioning head.

[0025] Furthermore, the specific process of step S23 is:

[0026] In the first branch, the feature map F' with spatial attention is input into the skip connection module to obtain the first branch feature;

[0027] In the second and third branches, after downsampling the feature map F' with spatial attention, it is sequentially input into the upsampling layer and the BnReluConv layer for processing to obtain the enhanced feature map F". In the downsampling process, the convolution layer is used instead of the pooling layer.

[0028] In the second branch, the enhanced feature map F” is passed through a 1×1 convolution module to obtain the second branch feature F2’;

[0029] In the third branch, the enhanced feature map F” is input into the 1×1 convolution module to obtain the positioning Logits information, and then the positioning Logits information is input into the 1×1 convolution module to obtain the feature F3’; at the same time, the Logits information is input into the Softmax function to obtain the positioning head information.

[0030] As a preferred technical solution, the process of optimizing the SACRSI network training in S3 is as follows:

[0031] First, set the hyperparameters. Then, in each epoch, obtain the first-order and second-order loss information of the network. Backpropagate based on the loss information to update the SACRSI network parameters and complete the optimization of the SACRSI network.

[0032] The hyperparameters include training batch, initial learning rate, learning rate decay weight, epoch, and optimizer.

[0033] As a preferred technical solution, the specific steps of adopting the local maximum detection strategy in S4 to convert the second-order positioning head into the position coordinates of each person are:

[0034] Use the maximum pooling operation to obtain all local maxima in the second-order positioning head as candidate points, and use an adaptive threshold to filter out false positives in the candidate points to obtain the position coordinates of each person. Then, perform upsampling to complete the crowd detection task.

[0035] The adaptive threshold is defined as 100 / 255.0 times the maximum value among the candidate points.

[0036] According to another aspect of the present invention, a crowd recognition system for remote sensing satellite images is provided. The system operates using the above-described method for crowd recognition in remote sensing satellite images that takes into account spatial attention. The system includes an image preprocessing and fusion module, a deep learning processing module, a network optimization and training module, and a crowd recognition and positioning module.

[0037] The image preprocessing and fusion module is used to select and fuse remote sensing satellite images to generate a remote sensing satellite image set, and then interpret and process the remote sensing satellite image set to obtain a positioning map;

[0038] The deep learning processing module is used to build a SACRSI network, input the remote sensing satellite image set into the network for processing, and output the first-order and second-order positioning heads;

[0039] The network optimization and training module is used to calculate the error between the positioning map and the first-order and second-order positioning heads through the focal loss function, obtain the first-order and second-order loss information, optimize the SACRSI network and output the trained SACRSI network;

[0040] The crowd recognition and positioning module is used to input the remote sensing satellite image to be processed into the trained SACRSI network, output the second-order positioning head, and adopt the local maximum detection strategy to convert the second-order positioning head into the position coordinates of each person, thereby completing crowd recognition for remote sensing satellite images.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. The present invention constructs a SACRSI network, and inputs the positioning map obtained by interpreting the remote sensing satellite image and the remote sensing satellite image set into the network to obtain the first-order and second-order positioning heads after processing. The focal loss function is used to calculate the error between the positioning map and the first-order and second-order positioning heads to obtain the first-order and second-order loss information, which is used to optimize the SACRSI network. The second-order positioning head output by the trained SACRSI network is then used, and the local maximum detection strategy is adopted to convert the second-order positioning head into the position coordinates of each person, thereby completing the crowd recognition for remote sensing satellite images. This makes the method of the present invention have obvious advantages in positioning accuracy compared with the classic crowd detection algorithm, and is an effective crowd detection method for remote sensing images.

[0043] 2. In the SACRSI network of the present invention, the remote sensing satellite image is input into the residual layer for processing to obtain the feature map F; the introduction of the residual layer enhances the network's ability to learn complex features, making it more adaptable to complex and diversified feature extraction processes, and enhancing the scope of application of this method.

[0044] 3. The SACRSI network in the present invention includes a pre-processing module and an hourglass network, wherein the pre-processing module includes a residual layer and a spatial attention layer; the feature map F is processed by the spatial attention layer to obtain a feature map F' with spatial attention; that is, after the basic features are extracted, the spatial attention mechanism further highlights the expression of crowd signals, which helps to accurately extract the crowd position, making this method more suitable for crowd recognition in remote sensing images with weak crowd signal characteristics.

[0045] 4. The present invention combines the positioning map obtained by interpretation with the positioning head obtained by SACRSI network processing, calculates the error between the positioning map and the first-order and second-order positioning heads through the focal loss function, and obtains the first-order and second-order loss information to optimize the training of the SACRSI network. This enables the present invention to show good results in crowd recognition tasks in complex scenarios (such as snow and beaches) and demonstrates strong generalization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Schematic diagram of the steps of a method for identifying people in remote sensing satellite images taking into account spatial attention in the present invention;

[0047] Figure 2 A flowchart of a method for identifying people in remote sensing satellite images taking into account spatial attention in the present invention;

[0048] Figure 3 A flowchart for interpretation in the present invention;

[0049] Figure 4 Schematic diagram of the structure of the basic module of the Hourglass network in the embodiment. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0051] In recent years, the increased frequency of crowd activity in public spaces such as commercial centers, squares, and transportation hubs has fueled economic prosperity. However, this activity has also brought with it certain public safety risks, such as traffic congestion, stampedes, social security incidents, and public health crises, threatening people's lives and property. Therefore, developing methods that can accurately identify the location of people in specific locations—crowd detection—is crucial to mitigating these risks.

[0052] In the early days of crowd detection, the primary focus was on images captured by surveillance or fixed cameras. These cameras provide images from a variety of viewing angles and lighting conditions, but they only capture the number of people in a local area, limiting the scope of analysis to the camera's coverage area. In recent years, with the advancements in aviation and drone platforms, aerial imagery has been used for crowd detection, significantly enhancing the ability to monitor and analyze crowd dynamics over large areas. However, aerial imagery is typically captured for specific missions, resulting in low temporal resolution and limited ability to analyze historical patterns and trace back events.

[0053] With the advancement of remote sensing technology, high-temporal and spatial resolution imagery is increasingly being used to detect small objects. Remote sensing imagery offers a wider monitoring range and fixed time intervals, improving the efficiency of large-scale monitoring and the ability to analyze historical patterns in the distribution of objects. Currently, images from Worldview, Beijing-3, and Jilin-1, with spatial resolutions of 0.30 meters or better, can clearly distinguish individuals within a crowd, enabling applications such as precise crowd detection over large areas and exploring historical patterns in population distribution.

[0054] Currently, there are no algorithms specifically designed for crowd recognition in remote sensing images. Crowd signal features in remote sensing images are weak, and conventional crowd detection algorithms can lose target information, increasing the detection error rate. Therefore, it is necessary to develop technologies that can effectively detect these weak features.

[0055] Example 1

[0056] In this embodiment, a crowd recognition method for remote sensing satellite images taking into account spatial attention is applied to identify crowds in multi-view remote sensing images. The method flow is as follows: Figure 1 As shown, the following steps are included:

[0057] S1. Select and fuse remote sensing satellite images to generate a remote sensing satellite image set, and then interpret and process the remote sensing satellite image set to obtain a positioning map;

[0058] S2, build the SACRSI network, input the remote sensing satellite image set into the network for processing, and output the first-order and second-order positioning heads;

[0059] S3. Calculate the error between the positioning map and the first-order and second-order positioning heads through the focal loss function to obtain the first-order and second-order loss information to optimize the SACRSI network and output the trained SACRSI network.

[0060] S4. Input the remote sensing satellite image to be processed into the trained SACRSI network, output the second-order positioning head, and adopt the local maximum detection strategy to convert the second-order positioning head into the position coordinates of each person, thereby completing the crowd recognition for remote sensing satellite images.

[0061] In this embodiment, it is specifically implemented as follows:

[0062] Step a in this embodiment is: interpreting remote sensing images and generating deep learning samples.

[0063] In this example, we first selected cloud-free, color-free, ultra-high spatial resolution satellite images with a panchromatic resolution of 0.30 meters or better over a densely populated study area. We then interpreted these images to generate deep learning sample data.

[0064] In the embodiment, the interpretation is manual interpretation, and the process is as follows Figure 3 As shown in the figure, the specific steps for interpreting and generating deep learning sample data are:

[0065] First, the spectral band and panchromatic band of the remote sensing image are fused using the surface-to-point regression kriging method to generate a fused image with a spatial resolution of 0.30 meters better than 0.30 meters.

[0066] Using visual methods, combined with image mosaicking and cropping techniques in ENVI software, we extracted areas with high crowd density and generated a fused image of the subareas. During the interpretation phase, historical Google Earth imagery of the corresponding areas was used to assist with annotation to prevent background interference. Interpretation markers were also created, annotating each person's location information (i.e., the individual's center pixel coordinates (x, y)). Ultimately, the subarea population labels were generated. The labeled data was then reviewed and organized, completing the manual interpretation.

[0067] Then the remote sensing image is cropped to a pixel size of 256 × 256. For images whose size is not divisible by 256, appropriate padding methods are used to fill the boundaries;

[0068] Step b in this embodiment is: constructing a convolutional neural network that takes spatial attention into account.

[0069] In order to overcome the difficulty in expressing and extracting weak crowd signals in convolutional networks, this embodiment proposes to use the residual idea and spatial attention mechanism to highlight the expression of crowd signals.

[0070] First, the crowd annotation data is downsampled to half the original image size (i.e., 128×128 pixels), and then a positioning map is generated based on the focal inverse distance transform method. The calculation formula is as follows:

[0071]

[0072] In formula (1), I represents the localization map; α and β are weight factors, which are set to 0.02 and 0.75 respectively; C is a constant, which is set to 1; P(x,y) represents the distance between the pixel at the (x,y) coordinate and its nearest annotated pixel.

[0073] The fused sliced ​​image is then input into the pre-processing module. First, the l×256×256 image is input into the residual block to obtain a feature map F of size 256×128×128; secondly, F is input into the spatial attention layer, which first calculates the average and maximum features of each channel respectively to obtain F avg and F max , whose size is 256×1×1; then, F avg and F max The concatenation is done in the channel dimension, and then a spatial attention layer is generated through a shared convolutional layer (with a kernel size of 7×7, padding set to 3, and a sigmoid activation function). Finally, the spatial attention is assigned as a weight to the original 256×128×128 feature map F to obtain F'.

[0074] Then, the feature F' is input into the Hourglass network. The structure of the basic module of the Hourglass network is as follows Figure 4As shown in the figure, it is divided into three branches: the first branch is the skip connection module, which obtains the feature F1'; the second branch is the feature extraction module, which first reduces the spatial size of the feature through a downsampling process. In this downsampling process, convolution is used instead of pooling operation. Subsequently, the feature is input into the upsampling layer, and then passes through the BnReluConv module to obtain the feature F", and finally passes through the 1×1 convolution module to obtain the second branch feature F2'; the third branch is the intermediate supervision module, which inputs F" into the 1×1 convolution module to obtain the positioning Logits information. After that, the positioning Logits information is input into the 1×1 convolution module to obtain the feature F3'. Finally, the features of the three branches F1', F2' and F3' are fused to obtain the first-order hourglass network output feature.

[0075] In this embodiment, in order to improve the depth of feature extraction, the first-order Hourglass network output features are re-input into the Hourglass network to obtain the second-order Hourglass network output features, and the second-order positioning Logits information is also obtained.

[0076] Finally, the first-order and second-order positioning Logits information is input into the Sigmoid function to obtain the corresponding positioning head, whose size is 1×128×128. At the same time, the loss of the first-order and second-order positioning heads is calculated with the real positioning map.

[0077] In this embodiment, the loss function is a focal loss function.

[0078] Step c in this embodiment is: setting hyperparameters and training network parameters.

[0079] The remote sensing image tiles were set to 256×256 pixels, and image enhancement operations were performed, including random cropping, flipping, and normalization. The training batch size was set to 4, the initial learning rate was set to 0.0003, the learning rate decay weight was set to 0.001, the number of epochs was set to 200, and the optimizer was AdamW. First-order and second-order loss information was obtained after each training epoch, and backpropagation was performed based on this loss information to update the parameters.

[0080] Step d in this embodiment is: applying the model to actual tasks to extract crowd location information from images with a spatial resolution of 0.3 meters or better.

[0081] After the training phase is completed, the second-order localization head is selected as the final result. The local maximum detection strategy is used to convert the second-order localization head into a real localization point: First, the maximum pooling operation is used to obtain all local maxima (candidate points), and the adaptive threshold T is used. a To filter out false positives, where T aDefined as 100 / 255.0 times the maximum value. By detecting the local maximum, the position coordinates of each person are obtained, and then the crowd detection task is completed through upsampling.

[0082] This example uses 13 Beijing-3 and Jilin-1 satellite images from eight provinces and municipalities in China. These images cover scenes such as parks, snowfields, and beaches, exhibiting significant spatial heterogeneity. After mosaicking and pan-sharpening, images with a spatial resolution of 0.3 meters were generated. Next, the images were interpreted and cropped to 256×256 pixels, while images without people were removed. This dataset was then horizontally and mirror-flipped to produce 2,595 images for training. Finally, the trained model was applied to a real-world task to test its accuracy.

[0083] In this embodiment, the image crowd recognition should be evaluated from the perspective of positioning as follows. For the positioning index, the primary goal is to convert the predicted individual coordinates P p The corresponding real coordinate P g Match. When P p and P g If the Euclidean distance between them is less than the distance threshold σ, the match is considered successful. The matching matrix and the number of true positives, false positives, and false negatives are calculated. Based on these indicators, the precision (P), recall (R), and F-Measure (F) are used to evaluate the positioning ability of the model. The specific formula is as follows:

[0084]

[0085] Where R is the recall rate, TP is the number of true positives, FN is the number of false negatives, P is the precision rate, FP is the number of false positives, and F is the F-Measure, which is the weighted harmonic average of precision and recall.

[0086] In this embodiment, the evaluation indicators are combined and compared with five classic crowd recognition algorithms. The experimental data of the simulation experiment is shown in Table 1.

[0087] Table 1 Statistics of the experimental results of ultra-high resolution image crowd recognition

[0088]

[0089] The experimental results in Table 1 show that this solution, by introducing a residual module and an attention mechanism, improves the neural network's ability to represent crowd signals. It even performs well in crowd recognition tasks in remote sensing imagery with strong spatial heterogeneity, including complex scenes such as snow and beaches. Specifically, this solution achieved an F-value of 0.598 and an R-value of 0.554 in positioning, both exceeding the next best method by 0.05. This demonstrates that this solution can effectively identify and locate crowds, reducing missed and false detections, and demonstrating strong robustness and significant positioning advantages.

[0090] Example 2

[0091] In this embodiment, a crowd recognition system for remote sensing satellite images taking into account spatial attention is used to identify people in multiple remote sensing images. The system process is as follows: Figure 2 As shown, the system includes image preprocessing and fusion module, deep learning processing module, network optimization and training module, and crowd recognition and positioning module;

[0092] The image preprocessing and fusion module is used to select and fuse remote sensing satellite images to generate a remote sensing satellite image set, and then interpret and process the remote sensing satellite image set to obtain a positioning map;

[0093] The deep learning processing module is used to build the SACRSI network, input the remote sensing satellite image set into the network for processing, and output the first-order and second-order positioning heads;

[0094] The network optimization and training module is used to calculate the error between the positioning map and the first-order and second-order positioning heads through the focal loss function, obtain the first-order and second-order loss information, optimize the SACRSI network and output the trained SACRSI network;

[0095] The crowd recognition and positioning module is used to input the remote sensing satellite image to be processed into the trained SACRSI network, output the second-order positioning head, and adopt the local maximum detection strategy to convert the second-order positioning head into the position coordinates of each person, thereby completing crowd recognition for remote sensing satellite images.

[0096] In this embodiment, it is specifically implemented as follows:

[0097] Step a in this embodiment is: interpreting remote sensing images and generating deep learning samples.

[0098] In this example, we first selected cloud-free, color-free, ultra-high spatial resolution satellite images with a panchromatic resolution of 0.30 meters or better over a densely populated study area. We then interpreted these images to generate deep learning sample data.

[0099] In the embodiment, the process of manual interpretation is as follows Figure 3 As shown in the figure, the specific steps for interpreting and generating deep learning sample data are:

[0100] First, the spectral band and panchromatic band of the remote sensing image are fused using the surface-to-point regression kriging method to generate a fused image with a spatial resolution of 0.30 meters better than 0.30 meters.

[0101] Using visual methods, combined with image mosaicking and cropping techniques in ENVI software, we extracted areas with high crowd density and generated a fused image of the subareas. During the interpretation phase, historical Google Earth imagery of the corresponding areas was used to assist with annotation to prevent background interference. Interpretation markers were also created, annotating each person's location information (i.e., the individual's center pixel coordinates (x, y)). Ultimately, the subarea population labels were generated. The labeled data was then reviewed and organized, completing the manual interpretation.

[0102] Then the remote sensing image is cropped to a pixel size of 256 × 256. For images whose size is not divisible by 256, appropriate padding methods are used to fill the boundaries;

[0103] Step b in this embodiment is: constructing a convolutional neural network that takes spatial attention into account.

[0104] In order to overcome the difficulty in expressing and extracting weak crowd signals in convolutional networks, this embodiment proposes to use the residual idea and spatial attention mechanism to highlight the expression of crowd signals.

[0105] First, the crowd annotation data is downsampled to half the original image size (i.e., 128×128 pixels), and then a positioning map is generated based on the focal inverse distance transform method. The calculation formula is as follows:

[0106]

[0107] In formula (1), I represents the localization map; α and β are weight factors, which are set to 0.02 and 0.75 respectively; C is a constant, which is set to 1; P(x,y) represents the distance between the pixel at the (x,y) coordinate and its nearest annotated pixel.

[0108] The fused sliced ​​image is then input into the pre-processing module. First, the l×256×256 image is input into the residual block to obtain a feature map F of size 256×128×128; secondly, F is input into the spatial attention layer, which first calculates the average and maximum features of each channel respectively to obtain F avg and F max , whose size is 256×1×1; then, F avg and F maxThe concatenation is done in the channel dimension, and then a spatial attention layer is generated through a shared convolutional layer (with a kernel size of 7×7, padding set to 3, and a sigmoid activation function). Finally, the spatial attention is assigned as a weight to the original 256×128×128 feature map F to obtain F'.

[0109] Then, the feature F' is input into the Hourglass network. The structure of the basic module of the Hourglass network is as follows Figure 4 As shown in the figure, it is divided into three branches: the first branch is the skip connection module, which obtains the feature F1'; the second branch is the feature extraction module, which first reduces the spatial size of the feature through a downsampling process. In this downsampling process, convolution is used instead of pooling operation. Subsequently, the feature is input into the upsampling layer, and then passes through the BnReluConv module to obtain the feature F", and finally passes through the 1×1 convolution module to obtain the second branch feature F2'; the third branch is the intermediate supervision module, which inputs F" into the 1×1 convolution module to obtain the positioning Logits information. After that, the positioning Logits information is input into the 1×1 convolution module to obtain the feature F3'. Finally, the features of the three branches F1', F2' and F3' are fused to obtain the first-order hourglass network output feature.

[0110] In this embodiment, in order to improve the depth of feature extraction, the first-order Hourglass network output features are re-input into the Hourglass network to obtain the second-order Hourglass network output features, and the second-order positioning Logits information is also obtained.

[0111] Finally, the first-order and second-order positioning Logits information is input into the Sigmoid function to obtain the corresponding positioning head, whose size is 1×128×128. At the same time, the loss of the first-order and second-order positioning heads is calculated with the real positioning map.

[0112] In this embodiment, the loss function is a focal loss function.

[0113] Step c in this embodiment is: setting hyperparameters and training network parameters.

[0114] The remote sensing image tiles were set to 256×256 pixels, and image enhancement operations were performed, including random cropping, flipping, and normalization. The training batch size was set to 4, the initial learning rate was set to 0.0003, the learning rate decay weight was set to 0.001, the number of epochs was set to 200, and the optimizer was AdamW. First-order and second-order loss information was obtained after each training epoch, and backpropagation was performed based on this loss information to update the parameters.

[0115] Step d in this embodiment is: applying the model to actual tasks to extract crowd location information from images with a spatial resolution of 0.3 meters or better.

[0116] After the training phase is completed, the second-order localization head is selected as the final result. The local maximum detection strategy is used to convert the second-order localization head into a real localization point: First, the maximum pooling operation is used to obtain all local maxima (candidate points), and the adaptive threshold T is used. a To filter out false positives, where T a Defined as 100 / 255.0 times the maximum value. By detecting the local maximum, the position coordinates of each person are obtained, and then the crowd detection task is completed through upsampling.

[0117] This example uses 13 Beijing-3 and Jilin-1 satellite images from eight provinces and municipalities in China. These images cover scenes such as parks, snowfields, and beaches, exhibiting significant spatial heterogeneity. After mosaicking and pan-sharpening, images with a spatial resolution of 0.3 meters were generated. Next, the images were interpreted and cropped to 256×256 pixels, while images without people were removed. This dataset was then horizontally and mirror-flipped to produce 2,595 images for training. Finally, the trained model was applied to a real-world task to test its accuracy.

[0118] In this embodiment, the image crowd recognition should be evaluated from the perspective of positioning as follows. For the positioning index, the primary goal is to convert the predicted individual coordinates P p The corresponding real coordinate P g Match. When P p and P g If the Euclidean distance between them is less than the distance threshold σ, the match is considered successful. The matching matrix and the number of true positives, false positives, and false negatives are calculated. Based on these indicators, the precision (P), recall (R), and F-Measure (F) are used to evaluate the positioning ability of the model. The specific formula is as follows:

[0119]

[0120] Where R is the recall rate, TP is the number of true positives, FN is the number of false negatives, P is the precision rate, FP is the number of false positives, and F is the F-Measure, which is the weighted harmonic average of precision and recall.

[0121] In this embodiment, the evaluation indicators are combined and compared with five classic crowd recognition algorithms. The experimental data of the simulation experiment is shown in Table 1.

[0122] In summary, this solution, by introducing a residual module and attention mechanism, improves the neural network's ability to represent crowd signals. It even performs well in crowd recognition tasks in remote sensing imagery with strong spatial heterogeneity, including complex scenes such as snow and beaches. Specifically, this solution achieved an F-value of 0.598 and an R-value of 0.554 in positioning, both exceeding the next best method by 0.05. This demonstrates that this solution can effectively identify and locate crowds, reducing missed and false detections, and demonstrates strong robustness and significant positioning advantages.

[0123] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for crowd recognition in remote sensing satellite images taking into account spatial attention, characterized in that: The method comprises the following steps: S1. Select and fuse remote sensing satellite images to generate a remote sensing satellite image set, and then interpret and process the remote sensing satellite image set to obtain a positioning map; S2, build the SACRSI network, input the remote sensing satellite image set into the network for processing, and output the first-order and second-order positioning heads; S3. Calculate the error between the positioning map and the first-order and second-order positioning heads through the focal loss function to obtain the first-order and second-order loss information to optimize the SACRSI network and output the trained SACRSI network. S4. Input the remote sensing satellite image to be processed into the trained SACRSI network, output the second-order positioning head, and adopt the local maximum detection strategy to convert the second-order positioning head into the position coordinates of each person, thereby completing the crowd recognition for remote sensing satellite images; The specific steps for interpreting and processing the remote sensing satellite image set in S1 are: First, we use image mosaicking and cropping methods to select areas with large crowds. Second, we use historical Google Earth images corresponding to these areas as auxiliary data and annotate the locations of individual people in the crowd using point annotations to generate annotated image data. Finally, we downsample the annotated image data and apply the focal inverse distance transform method to generate a location map. The SACRSI network in S2 includes a pre-processing module and an hourglass network, wherein the pre-processing module includes a residual layer and a spatial attention layer; the hourglass network includes three branches, the first branch is a jump connection module, which outputs the first branch feature The second branch is the feature extraction module outputting the second branch feature The third branch is the intermediate supervision module, which outputs the positioning head information and the third branch features After the remote sensing satellite image passes through the SACRSI network, it obtains the first-order output features and the first-order positioning head. Among them, the feature information is obtained by fusion of the outputs of the three branches. The first-order output features are input into the SACRSI network to output the second-order positioning head. In the first branch, the feature map with spatial attention Input the skip connection module to obtain the first branch feature; In the second and third branches, the feature maps with spatial attention After downsampling, it is sequentially input into the upsampling layer and Layer processing to obtain enhanced feature maps ; In the downsampling process, the convolution layer is used instead of the pooling layer; In the second branch, the feature map is enhanced through The convolution module obtains the second branch features ; In the third branch, the feature map is enhanced enter After the convolution module obtains the positioning Logits information, it inputs the positioning Logits information into Convolution module obtains features ; At the same time, input the Logits information into Softmax The function gets the positioning header information.

2. The method for identifying people in remote sensing satellite images taking spatial attention into account according to claim 1, characterized in that: The specific steps of selecting and fusing remote sensing satellite images in S1 are: selecting an image with a spatial resolution of n meters in the panchromatic band, and fusing the multispectral band of the image with the panchromatic band using the surface-to-point regression kriging method, where n≤0.

3.

3. The method for crowd recognition in remote sensing satellite images taking into account spatial attention according to claim 1, characterized in that: The calculation formula for the focal inverse distance transformation is: Where, represents the generated positioning map; and is the weight factor; is a constant; Indicates that it is located The distance between the pixel of the coordinate and its nearest annotation pixel.

4. The method for crowd recognition in remote sensing satellite images taking into account spatial attention according to claim 1, characterized in that: The working process of the SACRSI network includes: S21, input the remote sensing satellite image into the residual layer for processing, and obtain the feature map ; S22, feature map Input the spatial attention layer to obtain a feature map with spatial attention ; S23, feature map with spatial attention Input the three branches of the Hourglass network respectively and output the three branch features respectively 、 and and positioning heads; S24. Fuse the three branch features to obtain the output features and the positioning head.

5. The method for crowd recognition in remote sensing satellite images taking into account spatial attention according to claim 1, characterized in that: The process of optimizing and training the SACRSI network in S3 is as follows: First, set the hyperparameters. Then, in each epoch, obtain the first-order and second-order loss information of the network. Backpropagate based on the loss information to update the SACRSI network parameters and complete the optimization of the SACRSI network. The hyperparameters include training batch, initial learning rate, learning rate decay weight, epoch, and optimizer.

6. The method for identifying people in remote sensing satellite images taking into account spatial attention according to claim 1, characterized in that: The specific steps of adopting the local maximum detection strategy in step S4 to convert the second-order positioning head into the position coordinates of each person are: Use the maximum pooling operation to obtain all local maxima in the second-order positioning head as candidate points, and use an adaptive threshold to filter out false positives in the candidate points to obtain the position coordinates of each person. Then, perform upsampling to complete the crowd detection task. The adaptive threshold is defined as 100 / 255.0 times the maximum value among the candidate points.

7. A crowd recognition system for remote sensing satellite images, characterized in that: The system is operated by a method for crowd recognition in remote sensing satellite images taking into account spatial attention as described in any one of claims 1 to 6, the system comprising an image preprocessing and fusion module, a deep learning processing module, a network optimization and training module, and a crowd recognition and positioning module; The image preprocessing and fusion module is used to select and fuse remote sensing satellite images to generate a remote sensing satellite image set, and then interpret and process the remote sensing satellite image set to obtain a positioning map; The deep learning processing module is used to build a SACRSI network, input the remote sensing satellite image set into the network for processing, and output the first-order and second-order positioning heads; The network optimization and training module is used to calculate the error between the positioning map and the first-order and second-order positioning heads through the focal loss function, obtain the first-order and second-order loss information, optimize the SACRSI network and output the trained SACRSI network; The crowd recognition and positioning module is used to input the remote sensing satellite image to be processed into the trained SACRSI network, output the second-order positioning head, and adopt the local maximum detection strategy to convert the second-order positioning head into the position coordinates of each person, thereby completing crowd recognition for remote sensing satellite images.

Images