Remote sensing satellite image crowd identification method and system considering space attention

Abstract

The invention relates to a remote sensing satellite image crowd identification method and system considering space attention. According to the method, an SACRSI network is constructed, a positioning map obtained by interpreting a remote sensing satellite image and a first-order positioning head and a second-order positioning head obtained after a remote sensing satellite image set is input into the network to be processed are input, errors between the positioning map and the first-order positioning head and errors between the positioning map and the second-order positioning head are calculated through a focus loss function, and first-order loss information and second-order loss information are obtained; the method comprises the following steps: performing optimization training on an SACRSI network, outputting a second-order positioning head by using the trained SACRSI network, and converting the second-order positioning head into a position coordinate of each person by adopting a local maximum detection strategy so as to complete crowd identification aiming at a remote sensing satellite image. Compared with the prior art, the method has the advantages of being high in positioning precision, capable of effectively completing crowd detection of remote sensing images, high in generalization ability, wide in application range and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image recognition, and in particular to a method and system for remote sensing satellite image crowd recognition taking into account spatial attention. Background Art

[0002] In recent years, the increase in the frequency of crowd activities in public places such as commercial centers, squares and transportation hubs has promoted economic prosperity. However, these activities also bring certain public safety risks, such as traffic congestion, crowd stampede accidents, social security incidents and public health crises, which threaten people's lives and property safety. To this end, developing methods that can accurately identify the location of people in specific places, namely crowd detection, is particularly important to mitigate these risks.

[0003] In the early stages of crowd detection, the focus was on images captured by surveillance or fixed cameras. These cameras provide images from a variety of viewing angles and lighting conditions, but can only reflect the number of people in a local area, and the scope of analysis is limited by the camera coverage area. In recent years, with the advantages of aviation and drone platforms, aerial images have been used for crowd detection, which has certainly enhanced the ability to monitor and analyze the dynamics of large-scale crowds. However, aerial images are usually taken through specific tasks, with low temporal resolution, and are relatively lacking in the ability to analyze historical patterns and trace back.

[0004] With the development of remote sensing technology, high temporal and spatial resolution images are increasingly used to detect small objects. Remote sensing images provide a wider monitoring range and fixed time intervals, improving the efficiency of large-scale monitoring and the ability to analyze the historical laws of ground object distribution.

[0005] At present, there are many methods and technologies for crowd recognition tasks in images, such as Point to Point Network (P2PNet), Point Query Transformer (PET), A Generalized Loss Function for Crowd Counting and Localization (GLC), Focal Inverse Distance Transform Map (FIDTM), Multiscale Crowd Counting and Localization (MCCL). However, these methods are not enough to meet the needs of crowd recognition in remote sensing images. In remote sensing images, the signal characteristics of crowds are weak, and conventional crowd detection algorithms will lead to the problem of target information loss, thereby increasing the detection error rate. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a method and system for crowd recognition in remote sensing satellite images taking into account spatial attention.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] According to one aspect of the present invention, a method for identifying people in remote sensing satellite images taking into account spatial attention is provided, the method comprising the following steps:

[0009] S1, selecting and fusing remote sensing satellite images to generate a remote sensing satellite image set, and then interpreting and processing the remote sensing satellite image set to obtain a positioning map;

[0010] S2, construct the SACRSI network, input the remote sensing satellite image set into the network for processing, and output the first-order and second-order positioning heads;

[0011] S3, calculating the error between the positioning map and the first-order and second-order positioning heads through the focal loss function, obtaining the first-order and second-order loss information to optimize the SACRSI network and output the trained SACRSI network;

[0012] S4. Input the remote sensing satellite image to be processed into the trained SACRSI network, output the second-order positioning head, and adopt the local maximum detection strategy to convert the second-order positioning head into the position coordinates of each person, thereby completing the crowd recognition for remote sensing satellite images.

[0013] As a preferred technical solution, the specific steps of selecting and fusing remote sensing satellite images in S1 are: selecting an image with a spatial resolution of n meters in the panchromatic band, and using the surface-to-point regression kriging method to fuse the multispectral band of the image with the panchromatic band, where n≤0.3.

[0014] As a preferred technical solution, the specific steps of interpreting and processing the remote sensing satellite image set in S1 are:

[0015] Firstly, the image mosaicking and cropping method is used to select the area with a large number of people. Secondly, the historical images of Google Earth corresponding to the area are used as auxiliary data, and the positions of the individuals in the crowd are marked in the form of point annotations to obtain the annotated image data. Finally, the annotated image data is downsampled and the focal inverse distance transform method is used to generate a positioning map.

[0016] As a preferred technical solution, the focal inverse distance transformation formula is:

[0017]

[0018] Where I represents the generated localization map; α and β are weight factors; C is a constant; P(x,y) represents the distance between the pixel at (x,y) coordinates and its nearest annotated pixel.

[0019] As a preferred technical solution, the SACRSI network in S2 includes a pre-processing module and an Hourglass network, wherein the pre-processing module includes a residual layer and a spatial attention layer; the Hourglass network includes three branches, the first branch is a jump connection module, which outputs the first branch feature F1', the second branch is a feature extraction module that outputs the second branch feature F2', and the third branch is an intermediate supervision module that outputs a positioning head and a third branch feature F3'; the remote sensing satellite image passes through the SACRSI network to obtain a first-order output feature and a first-order positioning head, wherein the feature information is obtained by fusion of the outputs of the three branches, and the first-order output features are input into the SACRSI network to output a second-order positioning head.

[0020] As a preferred technical solution, the working process of the SACRSI network includes:

[0021] S21, inputting the remote sensing satellite image into the residual layer for processing to obtain a feature map F;

[0022] S22, input the feature map F into the spatial attention layer to obtain a feature map F' with spatial attention;

[0023] S23, input the feature map F' with spatial attention into the three branches of the Hourglass network respectively, and output three branch features F1', F2' and F3' and the positioning head respectively;

[0024] S24: Fuse the three branch features to obtain output features and a positioning head.

[0025] Furthermore, the specific process of step S23 is as follows:

[0026] In the first branch, the feature map F' with spatial attention is input into the skip connection module to obtain the first branch feature;

[0027] In the second branch and the third branch, after downsampling the feature map F' with spatial attention, it is sequentially input into the upsampling layer and the BnReluConv layer for processing to obtain an enhanced feature map F". In the downsampling process, the convolution layer is used instead of the pooling layer.

[0028] In the second branch, the enhanced feature map F" is passed through a 1×1 convolution module to obtain the second branch feature F2';

[0029] In the third branch, the enhanced feature map F” is input into the 1×1 convolution module to obtain the positioning Logits information, and then the positioning Logits information is input into the 1×1 convolution module to obtain the feature F3’; at the same time, the Logits information is input into the Softmax function to obtain the positioning head information.

[0030] As a preferred technical solution, the process of optimizing the SACRSI network training in S3 is:

[0031] First, set the hyperparameters, then obtain the first-order and second-order loss information of the network in each epoch, and perform backpropagation based on the loss information to update the SACRSI network parameters to complete the optimization of the SACRSI network;

[0032] The hyperparameters include training batch, initial learning rate, learning rate decay weight, epoch, and optimizer.

[0033] As a preferred technical solution, the specific steps of adopting the local maximum detection strategy in S4 to convert the second-order positioning head into the position coordinates of each person are:

[0034] Use the maximum pooling operation to obtain all local maxima in the second-order positioning head as candidate points, and use an adaptive threshold to filter out false positives in the candidate points to obtain the position coordinates of each person, and then complete the crowd detection task through upsampling operation;

[0035] The adaptive threshold is defined as 100 / 255.0 times the maximum value among the candidate points.

[0036] According to another aspect of the present invention, a crowd recognition system for remote sensing satellite images is provided, the system is operated by applying the above-mentioned crowd recognition method for remote sensing satellite images taking into account spatial attention, the system includes an image preprocessing and fusion module, a deep learning processing module, a network optimization and training module, and a crowd recognition and positioning module;

[0037] The image preprocessing and fusion module is used to select and fuse remote sensing satellite images to generate a remote sensing satellite image set, and then interpret and process the remote sensing satellite image set to obtain a positioning map;

[0038] The deep learning processing module is used to construct a SACRSI network, and input the remote sensing satellite image set into the network for processing, and output the first-order and second-order positioning heads;

[0039] The network optimization and training module is used to calculate the error between the positioning map and the first-order and second-order positioning heads through the focal loss function, obtain the first-order and second-order loss information, optimize the SACRSI network and output the trained SACRSI network;

[0040] The crowd recognition and positioning module is used to input the remote sensing satellite image to be processed into the trained SACRSI network, output the second-order positioning head, and adopt the local maximum detection strategy to convert the second-order positioning head into the position coordinates of each person, thereby completing the crowd recognition for the remote sensing satellite image.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. The present invention constructs a SACRSI network, and the positioning map obtained by interpreting the remote sensing satellite image is input into the network to obtain the first-order and second-order positioning heads after the remote sensing satellite image set is processed. The focal loss function is used to calculate the error between the positioning map and the first-order and second-order positioning heads to obtain the first-order and second-order loss information, which is used to optimize the SACRSI network. The second-order positioning head output by the trained SACRSI network is then used, and the local maximum detection strategy is adopted to convert the second-order positioning head into the position coordinates of each person, thereby completing the crowd recognition for remote sensing satellite images. This makes the method of the present invention have obvious advantages in positioning accuracy compared with the classic crowd detection algorithm, and is an effective crowd detection method for remote sensing images.

[0043] 2. In the SACRSI network of the present invention, the remote sensing satellite image is input into the residual layer for processing to obtain the feature map F; the introduction of the residual layer enhances the network's ability to learn complex features, making it more adaptable to complex and diversified feature extraction processes, thereby enhancing the scope of application of the present method.

[0044] 3. The SACRSI network in the present invention includes a pre-processing module and an Hourglass network, wherein the pre-processing module includes a residual layer and a spatial attention layer; the feature map F is processed by the spatial attention layer to obtain a feature map F' with spatial attention; that is, after the basic features are extracted, the spatial attention mechanism further highlights the expression of crowd signals, which helps to accurately extract the crowd position, making this method more suitable for crowd recognition in remote sensing images with weak crowd signal characteristics.

[0045] 4. The present invention combines the positioning map obtained by interpretation with the positioning head obtained by SACRSI network processing, calculates the error between the positioning map and the first-order and second-order positioning heads through the focal loss function, and obtains the first-order and second-order loss information to optimize the SACRSI network training. This enables the present invention to show good results in crowd recognition tasks in complex scenarios (such as snow and beaches) and demonstrates strong generalization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of the steps of a method for identifying people in remote sensing satellite images taking into account spatial attention in the present invention;

[0047] Figure 2 A flowchart of a method for identifying people in remote sensing satellite images taking into account spatial attention in the present invention;

[0048] Figure 3 A flowchart for interpreting the present invention;

[0049] Figure 4 Schematic diagram of the structure of the basic module of the Hourglass network in the embodiment. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0051] In recent years, the increase in the frequency of crowd activities in public places such as commercial centers, squares and transportation hubs has promoted economic prosperity. However, these activities also bring certain public safety risks, such as traffic congestion, crowd stampede accidents, social security incidents and public health crises, which threaten people's lives and property safety. Therefore, developing methods that can accurately identify the location of people in specific places, namely crowd detection, is particularly important in mitigating these risks.

[0052] In the early stages of crowd detection, the focus was on images captured by surveillance or fixed cameras. These cameras provide images from a variety of viewing angles and lighting conditions, but can only reflect the number of people in a local area, and the scope of analysis is limited by the camera coverage area. In recent years, with the advantages of aviation and drone platforms, aerial images have been used for crowd detection, which has certainly enhanced the ability to monitor and analyze the dynamics of large-scale crowds. However, aerial images are usually taken through specific tasks, with low temporal resolution, and are relatively lacking in the ability to analyze historical patterns and trace back.

[0053] With the development of remote sensing technology, high spatiotemporal resolution images are increasingly used to detect small objects. Remote sensing images provide a wider monitoring range and fixed time intervals, improving the efficiency of large-scale monitoring and the ability to analyze the historical laws of the distribution of objects. Today, images with a spatial resolution of 0.30 meters or better from Worldview, Beijing-3, and Jilin-1 can clearly distinguish individuals in a crowd, making it possible to detect crowds accurately over a large area and explore the historical laws of crowd distribution.

[0054] At present, there is no algorithm specifically for crowd recognition in remote sensing images. In remote sensing images, the signal characteristics of crowds are weak, and conventional crowd detection algorithms may cause target information loss, thereby increasing the detection error rate. Therefore, it is necessary to develop corresponding technologies to effectively detect such weak features.

[0055] Example 1

[0056] In this embodiment, a crowd recognition method for remote sensing satellite images taking into account spatial attention is applied to recognize crowds in multi-view remote sensing images. The method flow is as follows: Figure 1 As shown, the following steps are included:

[0057] S1, selecting and fusing remote sensing satellite images to generate a remote sensing satellite image set, and then interpreting and processing the remote sensing satellite image set to obtain a positioning map;

[0058] S2, construct the SACRSI network, input the remote sensing satellite image set into the network for processing, and output the first-order and second-order positioning heads;

[0059] S3, calculating the error between the positioning map and the first-order and second-order positioning heads through the focal loss function, obtaining the first-order and second-order loss information to optimize the SACRSI network and output the trained SACRSI network;

[0060] S4. Input the remote sensing satellite image to be processed into the trained SACRSI network, output the second-order positioning head, and adopt the local maximum detection strategy to convert the second-order positioning head into the position coordinates of each person, thereby completing the crowd recognition for remote sensing satellite images.

[0061] In this embodiment, it is specifically implemented as follows:

[0062] Step a in this embodiment is: interpreting remote sensing images and generating deep learning samples.

[0063] In this embodiment, firstly, in the densely populated study area, cloudless and color-free ultra-high spatial resolution satellite images are selected, and the spatial resolution of the panchromatic band reaches or exceeds 0.30 meters. Then, such images are interpreted and deep learning sample data is generated.

[0064] In the embodiment, the interpretation is manual interpretation, and the process is as follows Figure 3 As shown in the figure, the specific steps for interpreting and generating deep learning sample data are:

[0065] Firstly, the spectral band and panchromatic band of the remote sensing image are fused using the surface-to-point regression kriging method to generate a fused image with a spatial resolution of 0.30 meters better than 0.30 meters.

[0066] Then, we use visual methods, combined with the image mosaicking and cropping methods of ENVI software, to extract areas with more people and generate partitioned fusion images. In the interpretation stage, in order to avoid the interference of background on manual annotation, we use Google Earth historical images of the corresponding area for auxiliary annotation; at the same time, we establish interpretation marks to mark the location information of each person, that is, the central pixel coordinates (x, y) of the individual, and finally obtain the partitioned population label data. The label data is checked and sorted, and the manual interpretation is completed.

[0067] Then the remote sensing image is cropped to a pixel size of 256 × 256. For images whose size is not divisible by 256, an appropriate padding method is used to fill the borders;

[0068] Step b in this embodiment is: constructing a convolutional neural network that takes into account spatial attention.

[0069] In order to overcome the difficulty in expressing and extracting weak crowd signals in convolutional networks, this embodiment proposes to use the residual idea and spatial attention mechanism to highlight the expression of crowd signals.

[0070] First, the crowd annotation data is downsampled to half the size of the original image (i.e., 128×128 pixels), and then a positioning map is generated based on the focal inverse distance transformation method. The calculation formula is as follows:

[0071]

[0072] In formula (1), I represents the localization map; α and β are weight factors, which are set to 0.02 and 0.75 respectively; C is a constant, which is set to 1; P(x, y) represents the distance between the pixel at the (x, y) coordinate and its nearest annotated pixel.

[0073] The fused sliced ​​image is then input into the pre-processing module. First, the l×256×256 image is input into the residual block to obtain a feature map F of size 256×128×128; secondly, F is input into the spatial attention layer, which first calculates the average and maximum features of each channel respectively to obtain F avg and F max , whose size is 256×1×1; then, F avg and F max Concatenate in the channel dimension, then generate a spatial attention layer through a shared convolutional layer (convolution kernel size is 7×7, padding is set to 3, and connected sigmoid activation function). Finally, the spatial attention is assigned as a weight to the original 256×128×128 feature map F to obtain F'.

[0074] Then, the feature F' is input into the Hourglass network. The structure of the basic module of the Hourglass network is as follows: Figure 4As shown in the figure, it is divided into three branches: the first branch is a skip connection module to obtain feature F1'; the second branch is a feature extraction module, which first reduces the spatial size of the feature through a downsampling process. In this downsampling process, convolution is used instead of pooling. Subsequently, the feature is input into the upsampling layer, and then passed through the BnReluConv module to obtain the feature F", and finally passed through the 1×1 convolution module to obtain the second branch feature F2'; the third branch is an intermediate supervision module, and F" is input into the 1×1 convolution module to obtain the positioning Logits information. Afterwards, the positioning Logits information is input into the 1×1 convolution module to obtain the feature F3'. Finally, the features of the three branches F1', F2' and F3' are fused to obtain the first-order Hourglass network output feature.

[0075] In this embodiment, in order to improve the depth of feature extraction, the first-order Hourglass network output features are re-input into the Hourglass network to obtain the second-order Hourglass network output features, and the second-order positioning Logits information is also obtained.

[0076] Finally, the first-order and second-order positioning Logits information is input into the Sigmoid function to obtain the corresponding positioning head, whose size is 1×128×128. At the same time, the loss of the first-order and second-order positioning heads and the true positioning map is calculated.

[0077] In this embodiment, the loss function is a focal loss function.

[0078] Step c in this embodiment is: setting hyperparameters and training network parameters.

[0079] The block size of the remote sensing image is set to 256×256 pixels, and the image is enhanced, including random cropping, flipping, and normalization operations; the training batch is set to 4, the initial learning rate is set to 0.0003, the learning rate decay weight is set to 0.001, the epoch is set to 200, and the optimizer is set to Adam W. After each training epoch, the first-order and second-order loss information is obtained, and back propagation is performed based on the loss information to update the parameters.

[0080] Step d in this embodiment is: applying the model to actual tasks to extract crowd location information in images with a spatial resolution of or better than 0.3 meters.

[0081] After the training phase is completed, the second-order localization head is selected as the final result. The local maximum detection strategy is used to convert the second-order localization head into a real localization point: First, the maximum pooling operation is used to obtain all local maxima (candidate points), and the adaptive threshold T is used. a To filter out false positives, where T aDefined as 100 / 255.0 times the maximum value. Through local maximum detection, the position coordinates of each person are obtained, and then the crowd detection task is completed through upsampling operation.

[0082] In this example, 13 Beijing-3 and Jilin-1 satellite images from 8 provinces or municipalities in China were used, covering parks, snow, beaches, etc., with large spatial heterogeneity. After mosaicking and pan-sharpening, images with a spatial resolution of 0.3 meters were generated; secondly, the images were interpreted and cropped to a size of 256×256 pixels, and images without people were removed; then, the data set was horizontally and mirror-flipped to obtain 2595 images and trained; finally, the trained model was applied to actual tasks to test the model accuracy.

[0083] In this embodiment, the image crowd recognition should be evaluated from the perspective of positioning as follows. For the positioning index, its primary goal is to convert the predicted individual coordinates P p The corresponding real coordinates P g Match. When P p and P g If the Euclidean distance between them is less than the distance threshold σ, the match is considered successful. The matching matrix and the number of true positives, false positives, and false negatives are calculated. Based on these indicators, the precision (P), recall (R), and F-Measure (F) are used to evaluate the positioning ability of the model. The specific formula is as follows:

[0084]

[0085] In the formula, R is the recall rate, TP is the number of true positives, FN is the number of false negatives, P is the precision rate, FP is the number of false positives, and F is the F-Measure, which is the weighted harmonic average of precision and recall.

[0086] In this embodiment, the evaluation index is combined and compared with 5 classic crowd recognition algorithms. The experimental data of the simulation experiment is shown in Table 1.

[0087] Table 1 Statistics of the experimental results of crowd recognition with ultra-high resolution images

[0088]

[0089] From the experimental results in Table 1, we can see that this scheme improves the ability of neural networks in expressing crowd signals by introducing residual modules and attention mechanisms. It even performs well in crowd recognition tasks in remote sensing images with strong spatial heterogeneity, including complex scenes such as snow and beaches. Specifically, the F value of this scheme in positioning indicators is 0.598 and the R value is 0.554, which are 0.05 higher than the second best method. This shows that this scheme can effectively identify and locate crowds, reduce missed detections and false detections, and show strong robustness and obvious positioning advantages.

[0090] Example 2

[0091] In this embodiment, a crowd recognition system for remote sensing satellite images taking into account spatial attention is used to recognize people in multiple remote sensing images. The system flow is as follows: Figure 2 As shown, the system includes an image preprocessing and fusion module, a deep learning processing module, a network optimization and training module, and a crowd recognition and positioning module;

[0092] The image preprocessing and fusion module is used to select and fuse remote sensing satellite images to generate a remote sensing satellite image set, and then interpret and process the remote sensing satellite image set to obtain a positioning map;

[0093] The deep learning processing module is used to build the SACRSI network, and the remote sensing satellite image set is input into the network for processing, and the first-order and second-order positioning heads are output;

[0094] The network optimization and training module is used to calculate the error between the positioning map and the first-order and second-order positioning heads through the focal loss function, obtain the first-order and second-order loss information, optimize the SACRSI network, and output the trained SACRSI network;

[0095] The crowd recognition and positioning module is used to input the remote sensing satellite images to be processed into the trained SACRSI network, output the second-order positioning head, and adopt the local maximum detection strategy to convert the second-order positioning head into the position coordinates of each person, thereby completing the crowd recognition for remote sensing satellite images.

[0096] In this embodiment, it is specifically implemented as follows:

[0097] Step a in this embodiment is: interpreting remote sensing images and generating deep learning samples.

[0098] In this embodiment, firstly, in the densely populated study area, cloudless and color-free ultra-high spatial resolution satellite images are selected, and the spatial resolution of the panchromatic band reaches or exceeds 0.30 meters. Then, such images are interpreted and deep learning sample data is generated.

[0099] In the embodiment, the process of manual interpretation is as follows Figure 3 As shown in the figure, the specific steps for interpreting and generating deep learning sample data are:

[0100] Firstly, the spectral band and panchromatic band of the remote sensing image are fused using the surface-to-point regression kriging method to generate a fused image with a spatial resolution of 0.30 meters better than 0.30 meters.

[0101] Then, we use visual methods, combined with the image mosaicking and cropping methods of ENVI software, to extract areas with more people and generate partitioned fusion images. In the interpretation stage, in order to avoid the interference of background on manual annotation, we use Google Earth historical images of the corresponding area for auxiliary annotation; at the same time, we establish interpretation marks to mark the location information of each person, that is, the central pixel coordinates (x, y) of the individual, and finally obtain the partitioned population label data. The label data is checked and sorted, and the manual interpretation is completed.

[0102] Then the remote sensing image is cropped to a pixel size of 256 × 256. For images whose size is not divisible by 256, an appropriate padding method is used to fill the borders;

[0103] Step b in this embodiment is: constructing a convolutional neural network that takes into account spatial attention.

[0104] In order to overcome the difficulty in expressing and extracting weak crowd signals in convolutional networks, this embodiment proposes to use the residual idea and spatial attention mechanism to highlight the expression of crowd signals.

[0105] First, the crowd annotation data is downsampled to half the size of the original image (i.e., 128×128 pixels), and then a positioning map is generated based on the focal inverse distance transformation method. The calculation formula is as follows:

[0106]

[0107] In formula (1), I represents the localization map; α and β are weight factors, which are set to 0.02 and 0.75 respectively; C is a constant, which is set to 1; P(x, y) represents the distance between the pixel at the (x, y) coordinate and its nearest annotated pixel.

[0108] The fused sliced ​​image is then input into the pre-processing module. First, the l×256×256 image is input into the residual block to obtain a feature map F of size 256×128×128; secondly, F is input into the spatial attention layer, which first calculates the average and maximum features of each channel respectively to obtain F avg and F max , whose size is 256×1×1; then, F avg and F maxConcatenate in the channel dimension, then generate a spatial attention layer through a shared convolutional layer (convolution kernel size is 7×7, padding is set to 3, and connected sigmoid activation function). Finally, the spatial attention is assigned as a weight to the original 256×128×128 feature map F to obtain F'.

[0109] Then, the feature F' is input into the Hourglass network. The structure of the basic module of the Hourglass network is as follows: Figure 4 As shown in the figure, it is divided into three branches: the first branch is a skip connection module to obtain feature F1'; the second branch is a feature extraction module, which first reduces the spatial size of the feature through a downsampling process. In this downsampling process, convolution is used instead of pooling. Subsequently, the feature is input into the upsampling layer, and then passed through the BnReluConv module to obtain the feature F", and finally passed through the 1×1 convolution module to obtain the second branch feature F2'; the third branch is an intermediate supervision module, and F" is input into the 1×1 convolution module to obtain the positioning Logits information. Afterwards, the positioning Logits information is input into the 1×1 convolution module to obtain the feature F3'. Finally, the features of the three branches F1', F2' and F3' are fused to obtain the first-order Hourglass network output feature.

[0110] In this embodiment, in order to improve the depth of feature extraction, the first-order Hourglass network output features are re-input into the Hourglass network to obtain the second-order Hourglass network output features, and the second-order positioning Logits information is also obtained.

[0111] Finally, the first-order and second-order positioning Logits information is input into the Sigmoid function to obtain the corresponding positioning head, whose size is 1×128×128. At the same time, the loss of the first-order and second-order positioning heads and the true positioning map is calculated.

[0112] In this embodiment, the loss function is a focal loss function.

[0113] Step c in this embodiment is: setting hyperparameters and training network parameters.

[0114] The block size of the remote sensing image is set to 256×256 pixels, and the image is enhanced, including random cropping, flipping, and normalization operations; the training batch is set to 4, the initial learning rate is set to 0.0003, the learning rate decay weight is set to 0.001, the epoch is set to 200, and the optimizer is set to Adam W. After each training epoch, the first-order and second-order loss information is obtained, and back propagation is performed based on the loss information to update the parameters.

[0115] Step d in this embodiment is: applying the model to actual tasks to extract crowd location information in images with a spatial resolution of or better than 0.3 meters.

[0116] After the training phase is completed, the second-order localization head is selected as the final result. The local maximum detection strategy is used to convert the second-order localization head into a real localization point: First, the maximum pooling operation is used to obtain all local maxima (candidate points), and the adaptive threshold T is used. a To filter out false positives, where T a Defined as 100 / 255.0 times the maximum value. Through local maximum detection, the position coordinates of each person are obtained, and then the crowd detection task is completed through upsampling operation.

[0117] In this example, 13 Beijing-3 and Jilin-1 satellite images from 8 provinces or municipalities in China were used, covering parks, snow, beaches, etc., with large spatial heterogeneity. After mosaicking and pan-sharpening, images with a spatial resolution of 0.3 meters were generated; secondly, the images were interpreted and cropped to a size of 256×256 pixels, and images without people were removed; then, the data set was horizontally and mirror-flipped to obtain 2595 images and trained; finally, the trained model was applied to actual tasks to test the model accuracy.

[0118] In this embodiment, the image crowd recognition should be evaluated from the perspective of positioning as follows. For the positioning index, its primary goal is to convert the predicted individual coordinates P p The corresponding real coordinates P g Match. When P p and P g If the Euclidean distance between them is less than the distance threshold σ, the match is considered successful. The matching matrix and the number of true positives, false positives, and false negatives are calculated. Based on these indicators, the precision (P), recall (R), and F-Measure (F) are used to evaluate the positioning ability of the model. The specific formula is as follows:

[0119]

[0120] In the formula, R is the recall rate, TP is the number of true positives, FN is the number of false negatives, P is the precision rate, FP is the number of false positives, and F is the F-Measure, which is the weighted harmonic average of precision and recall.

[0121] In this embodiment, the evaluation index is combined and compared with 5 classic crowd recognition algorithms. The experimental data of the simulation experiment is shown in Table 1.

[0122] In summary, this scheme improves the ability of neural networks in expressing crowd signals by introducing residual modules and attention mechanisms, and even performs well in crowd recognition tasks in remote sensing images with strong spatial heterogeneity, including complex scenes such as snow and beaches. Specifically, the F value of this scheme in positioning indicators is 0.598 and the R value is 0.554, which are 0.05 higher than the second best method. This shows that this scheme can effectively identify and locate crowds, reduce missed detections and false detections, and show strong robustness and obvious positioning advantages.

[0123] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for crowd recognition in remote sensing satellite images taking into account spatial attention, characterized in that: The method comprises the following steps: S1, selecting and fusing remote sensing satellite images to generate a remote sensing satellite image set, and then interpreting and processing the remote sensing satellite image set to obtain a positioning map; S2, construct the SACRSI network, input the remote sensing satellite image set into the network for processing, and output the first-order and second-order positioning heads; S3, calculating the error between the positioning map and the first-order and second-order positioning heads through the focal loss function, obtaining the first-order and second-order loss information to optimize the SACRSI network and output the trained SACRSI network; S4. Input the remote sensing satellite image to be processed into the trained SACRSI network, output the second-order positioning head, and adopt the local maximum detection strategy to convert the second-order positioning head into the position coordinates of each person, thereby completing the crowd recognition for remote sensing satellite images.

2. The method for crowd recognition in remote sensing satellite images taking into account spatial attention according to claim 1, characterized in that: The specific steps of selecting and fusing the remote sensing satellite image in S1 are: selecting an image with a spatial resolution of n meters in the panchromatic band, and fusing the multispectral band of the image with the panchromatic band using the surface-to-point regression kriging method, wherein n≤0.

3.

3. The method for crowd recognition in remote sensing satellite images taking into account spatial attention according to claim 1, characterized in that: The specific steps of interpreting and processing the remote sensing satellite image set in S1 are: Firstly, the image mosaicking and cropping method is used to select the area with a large number of people. Secondly, the historical images of Google Earth corresponding to the area are used as auxiliary data, and the positions of the individuals in the crowd are marked in the form of point annotations to obtain the annotated image data. Finally, the annotated image data is downsampled and the focal inverse distance transform method is used to generate a positioning map.

4. The method for identifying people in remote sensing satellite images taking into account spatial attention according to claim 3, characterized in that: The focal inverse distance transformation calculation formula is: Where I represents the generated localization map; α and β are weight factors; C is a constant; P(x,y) represents the distance between the pixel at (x,y) coordinates and its nearest annotated pixel.

5. The method for crowd recognition in remote sensing satellite images taking into account spatial attention according to claim 1, characterized in that: The SACRSI network in S2 includes a pre-processing module and an Hourglass network, wherein the pre-processing module includes a residual layer and a spatial attention layer; the Hourglass network includes three branches, the first branch is a jump connection module, which outputs the first branch feature F1', the second branch is a feature extraction module that outputs the second branch feature F2', and the third branch is an intermediate supervision module that outputs the positioning head information and the third branch feature F3'; the remote sensing satellite image obtains the first-order output feature and the first-order positioning head after passing through the SACRSI network, wherein the feature information is obtained by fusion of the outputs of the three branches, and the first-order output feature is input into the SACRSI network to output the second-order positioning head.

6. The method for crowd recognition in remote sensing satellite images taking into account spatial attention according to claim 5, characterized in that: The working process of the SACRSI network includes: S21, inputting the remote sensing satellite image into the residual layer for processing to obtain a feature map F; S22, input the feature map F into the spatial attention layer to obtain a feature map F' with spatial attention; S23, input the feature map F' with spatial attention into the three branches of the Hourglass network respectively, and output three branch features F1', F2' and F3' and the positioning head respectively; S24: Fuse the three branch features to obtain output features and a positioning head.

7. The method for identifying people in remote sensing satellite images taking into account spatial attention according to claim 6, characterized in that: The specific process of step S23 is as follows: In the first branch, the feature map F' with spatial attention is input into the skip connection module to obtain the first branch feature; In the second branch and the third branch, after downsampling the feature map F' with spatial attention, it is sequentially input into the upsampling layer and the BnReluConv layer for processing to obtain an enhanced feature map F". In the downsampling process, the convolution layer is used instead of the pooling layer. In the second branch, the enhanced feature map F" is passed through a 1×1 convolution module to obtain the second branch feature F2'; In the third branch, the enhanced feature map F” is input into the 1×1 convolution module to obtain the positioning Logits information, and then the positioning Logits information is input into the 1×1 convolution module to obtain the feature F3’; at the same time, the Logits information is input into the Softmax function to obtain the positioning head information.

8. The method for crowd recognition in remote sensing satellite images taking into account spatial attention according to claim 1, characterized in that: The process of optimizing and training the SACRSI network in S3 is as follows: First, set the hyperparameters, then obtain the first-order and second-order loss information of the network in each epoch, and perform backpropagation based on the loss information to update the SACRSI network parameters to complete the optimization of the SACRSI network; The hyperparameters include training batch, initial learning rate, learning rate decay weight, epoch, and optimizer.

9. The method for identifying people in remote sensing satellite images taking into account spatial attention according to claim 1, characterized in that: The specific steps of adopting the local maximum detection strategy in step S4 to convert the second-order positioning head into the position coordinates of each person are: Use the maximum pooling operation to obtain all local maxima in the second-order positioning head as candidate points, and use an adaptive threshold to filter out false positives in the candidate points to obtain the position coordinates of each person, and then complete the crowd detection task through upsampling operation; The adaptive threshold is defined as 100 / 255.0 times the maximum value among the candidate points.

10. A crowd recognition system for remote sensing satellite images, characterized in that: The system is applied to work by a method for crowd recognition in remote sensing satellite images taking into account spatial attention as described in any one of claims 1 to 9, the system comprising an image preprocessing and fusion module, a deep learning processing module, a network optimization and training module, and a crowd recognition and positioning module; The image preprocessing and fusion module is used to select and fuse remote sensing satellite images to generate a remote sensing satellite image set, and then interpret and process the remote sensing satellite image set to obtain a positioning map; The deep learning processing module is used to construct a SACRSI network, and input the remote sensing satellite image set into the network for processing, and output the first-order and second-order positioning heads; The network optimization and training module is used to calculate the error between the positioning map and the first-order and second-order positioning heads through the focal loss function, obtain the first-order and second-order loss information, optimize the SACRSI network and output the trained SACRSI network; The crowd recognition and positioning module is used to input the remote sensing satellite image to be processed into the trained SACRSI network, output the second-order positioning head, and adopt the local maximum detection strategy to convert the second-order positioning head into the position coordinates of each person, thereby completing the crowd recognition for the remote sensing satellite image.

Images