Personnel vitality distribution diagram calculation method based on unmanned aerial vehicle and satellite image fusion

Abstract

The invention relates to the technical field of urban personnel vitality distribution evaluation, solves the technical problem that a traditional personnel vitality distribution analysis method cannot realize quantitative calculation of personnel activity conditions, and particularly relates to a personnel vitality distribution diagram calculation method based on unmanned aerial vehicle and satellite image fusion. Pixel areas belonging to personnel are automatically extracted from an unmanned aerial vehicle image, then personnel pixels extracted by the unmanned aerial vehicle are mapped to a satellite image, counting accumulation of personnel detection information is carried out on a pixel position counter of the satellite image, and finally pseudo-color rendering is carried out on an accumulation result of the counter to obtain a personnel vitality distribution diagram. According to the method provided by the invention, equipment communication recording of access personnel or acquisition of track information of equipment is not needed, and data acquisition and recording are realized completely based on a passive remote sensing image observation method.

Description

Technical Field

[0001] This invention relates to the field of urban population vitality distribution assessment technology, and in particular to a method for calculating population vitality distribution maps based on the fusion of UAV and satellite images. Background Technology

[0002] Urban public space vitality assessment refers to a comprehensive evaluation of the usage of urban public spaces, such as parks and squares, using quantitative assessment methods to evaluate the vitality of public spaces. Calculating a population activity distribution map can assist urban planning or municipal monitoring departments in optimizing urban construction layout, or be used to assess population activity in specific areas, facilitating more rational allocation of security and duty personnel.

[0003] Traditional methods for analyzing population activity distribution primarily rely on staff observations or questionnaires, lacking quantitative analytical tools. Another existing approach utilizes mobile phone big data for a comprehensive evaluation of population activity in urban public areas; yet another method uses GPS activity survey data to analyze indicators such as public transportation and facility usage. However, both mobile phone big data and GPS activity data require actively accessing the communication information of individuals' mobile phones or mobile devices, making independent data collection impossible. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method for calculating personnel activity distribution maps based on the fusion of UAV and satellite images, which solves the technical problem that traditional personnel activity distribution analysis methods cannot quantitatively calculate personnel activity.

[0005] To address the aforementioned technical problems, this invention provides the following technical solution: a method for calculating personnel vitality distribution maps based on the fusion of UAV and satellite images, the method comprising the following steps:

[0006] S1. Select a satellite image covering the observation area as the base image, and then set a counter with an initial value of 0 for each pixel position q in the satellite image coordinate system;

[0007] S2. Acquire video image data of the observation area taken by the drone, and detect and extract the pixel regions belonging to people in each frame of the drone image to obtain the pixel location set {p i}, i = 1, ..., m, where m is the number of extracted pixel positions;

[0008] S3. Perform image feature matching on the UAV image and satellite image respectively, and calculate the affine transformation matrix F from the UAV image to the satellite image coordinate system based on the correspondence of the feature points obtained from the matching.

[0009] S4. Based on the affine transformation matrix F, determine the coordinates p of each pixel in the pixel region belonging to the person. i Mapping these pixel positions to the satellite image coordinate system yields the set of pixel coordinates {q} representing the pixel region belonging to the person in the satellite image coordinate system. i}, i = 1, ..., m;

[0010] S5. Set the pixel coordinates q of the counter in the satellite image coordinate system. i The pixel coordinates p belonging to the person are mapped onto the top. i At that time, pixel coordinates q i The corresponding counter is incremented once.

[0011] S6. Repeat steps S2-S5 until all video image data has been processed. Statistically analyze the counters in the satellite image coordinate system and construct a personnel vitality distribution map with the same pixel size as the satellite image.

[0012] Furthermore, in step S2, a YOLOv11 or MaskRCNN network model is used to detect and extract pixel regions belonging to people in each frame of the drone image.

[0013] Furthermore, in step S3, the specific process includes the following steps:

[0014] S31. A deep neural network feature extraction and matching model is used to perform image feature matching on UAV images and satellite images to obtain the set of correspondences of matching feature points {(x,y)}. j ,(x′,y′) j}, j=1,...,N, where (x,y) j and (x′,y′) j , which are the pixel coordinates of the j-th pair of matching feature points in the satellite image and the UAV image, respectively, and N is the number of matching feature points;

[0015] S32. Let the mathematical form of the affine transformation matrix F be:

[0016]

[0017] Wherein, the affine transformation matrix F satisfies:

[0018]

[0019] S33. Based on the correspondence set {(x,y)} j ,(x′,y′) j The correspondence (x, y) between each pair of feature points in the sequence}, j = 1, ..., N. j ,(x′,y′) j Establish a system of equations, namely:

[0020] x = a 11 x′+a 12 y′+b1

[0021] y = a 21 x′+a 22 y′+b2

[0022] S34. Given N≥3 pairs of matching feature points, we need to solve for the parameter vector P=[a] of the affine transformation matrix F. 11 ,a 12 ,b1,a 21 [a2,b2] T The system of equations established by all matching points can be written in matrix form, that is:

[0023] AP = B, where A has 2N rows, and each pair of rows corresponds to an equation established based on matching feature points, i.e.:

[0024]

[0025] B = [x1, y1, x2, y2, ..., x N ,y N ] T

[0026] S35, The parameter vector P of the affine transformation matrix F = [a 11 ,a 12 ,b1,a 21 [a2,b2] T The least squares solution can be obtained by solving the normal equation, that is:

[0027] P=(A T A) -1 A T B

[0028] S36. Based on the calculated parameter vector P, and combined with P = [a 11 ,a 12 ,b1,a 21 [a2,b2] T Obtain the element a of the affine transformation matrix F 11 ,a 12 ,b1,a 21 ,a2,b2, and then construct the affine transformation matrix F, that is:

[0029]

[0030] In the formula, a 11 ,a 12 ,b1,a 21 a2 and b2 are elements obtained from the solution.

[0031] Furthermore, in step S31, the specific process includes the following steps:

[0032] S311. Input the UAV image and satellite image into the SuperPoint network respectively to obtain the feature point detection probability map;

[0033] S312. Select points with probability values ​​higher than a preset threshold in the feature point detection probability map as feature points, and extract a descriptor for each feature point to represent the local feature information of the feature point.

[0034] S313. A graph structure is constructed using the SuperGlue network based on the descriptors and spatial location information of the feature points, and the feature representation of each feature point is updated through a message passing mechanism.

[0035] S314. Calculate the matching score between each pair of feature points to represent the probability that they belong to the same physical point in the two images;

[0036] S315. Select the feature point pair with the highest matching score as the correspondence of the matching feature points.

[0037] Further, in step S4, the pixel mapping calculation process is as follows: q i =Fp i .

[0038] Furthermore, in step S6, the specific process includes the following steps:

[0039] S61. Select a color list containing colors with a brightness range of [0, L-1].

[0040] S62. Based on the maximum value v of the counter max and minimum value v min Normalize all values ​​of the counters to the range [0, L-1], and calculate the new value v for each counter. new ;

[0041] S63. For each pixel location in the satellite image, based on the new value v of its corresponding counter... new Find the index value and the new value v in the color list. new The color corresponding to the nearest index is assigned to that pixel position;

[0042] S64. Traverse all pixels to assign color values ​​to the entire satellite image, obtaining a pseudo-color image reflecting the distribution of human activity.

[0043] Furthermore, in step S61, the color list uses a 256-color lookup table, L = 256;

[0044] The color lookup table uses the Jet rainbow color lookup table, with [0, 255] fixed colors from the Jet lookup table pre-calculated and stored for later use.

[0045] Further, in step S62, the new value v new The calculation formula is:

[0046]

[0047] Among them, v old This is the actual value of the counter.

[0048] Furthermore, in step S64, the brighter the pixel area in the pseudo-color image, the more frequent the human activity; conversely, the darker the pixel area, the sparser the human activity.

[0049] By employing the above technical solution, the present invention provides a method for calculating personnel activity distribution maps based on the fusion of UAV and satellite images, which has at least the following beneficial effects:

[0050] 1. This invention utilizes the rapid aerial photography capabilities of drones and the global perspective of satellite images to provide a method for quantitatively calculating human activity and generating a pseudo-color human activity distribution map that can be displayed intuitively.

[0051] 2. The method proposed in this invention does not require access to personnel's equipment communication records or the trajectory information of the collection equipment, but achieves data collection and recording entirely based on passive remote sensing image observation. Attached Figure Description

[0052] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0053] Figure 1 This is a flowchart of the method for calculating the personnel vitality distribution map in this invention;

[0054] Figure 2 This is an example diagram of drone images obtained by drone aerial photography in this invention;

[0055] Figure 3 This is an example diagram of the fusion of UAV images and satellite images in this invention;

[0056] Figure 4 This is an example diagram of the personnel vitality distribution map in this invention. Detailed Implementation

[0057] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This will allow for a full understanding of how the present application uses technical means to solve technical problems and achieve technical effects, and to facilitate its implementation.

[0058] This embodiment proposes a method for calculating a population vitality distribution map based on the fusion of UAV and satellite imagery. Aerial photography of the observation area is conducted using a UAV, and the aerial video images are fused with satellite images to extract a spatial distribution map of population vitality, which can be used for assessing the vitality of urban public spaces. Figure 1 As shown, the method includes the following steps:

[0059] S1. Select a satellite image covering the observation area as the base image, and then set a counter for each pixel position q in the satellite image coordinate system. The initial value of all counters is 0.

[0060] S2. Acquire video image data of the observation area taken by the drone, and detect and extract the pixel regions belonging to people in each frame of the drone image to obtain the pixel location set {p i}, i = 1, ..., m, where m is the number of extracted pixel positions.

[0061] Use drones to conduct aerial photography of the park (observation area) at an altitude of 40 to 120 meters, primarily from a downward perspective, to collect video and image data. The time period for aerial photography can be selected based on the specific time of interest, such as morning, noon, or evening. Figure 2 The image shown is an example of an aerial photograph taken by a drone.

[0062] In this embodiment, a deep neural network object detection model is used to automatically detect people in each image of the video image data and extract the pixel regions belonging to people. The deep neural network object detection model can be a network model such as YOLOv11 or Mask R-CNN. In order to improve the detection accuracy of the model, drone images are first used for sample labeling, and the network model is optimized and trained. The trained model is then used to detect people in drone images.

[0063] S3. Using a deep neural network feature extraction and matching model, perform image feature matching on both the UAV image and the satellite image. Based on the correspondence of the matched feature points, calculate the affine transformation matrix F mapping from the UAV image to the satellite image coordinate system. The specific process includes the following steps:

[0064] S31. A deep neural network feature extraction and matching model is used to perform image feature matching on UAV images and satellite images to obtain the set of correspondences of matching feature points {(x,y)}. j ,(x′,y′) j}, j=1,...,N, where (x,y) j and (x′,y′) j , , are the pixel coordinates of the j-th pair of matching feature points in the satellite image and the drone image, respectively, and N is the number of matching feature points.

[0065] In this embodiment, the SuperPoint network plus the SuperGlue network is selected as the deep neural network feature extraction and matching model. Satellite images and a subset of UAV images are used for pre-training to obtain the network model parameters. The model can then automatically extract feature point coordinates from both satellite and UAV images and calculate the correspondence between matching feature points.

[0066] Through forward propagation, the SuperPoint network outputs a feature point detection probability map. The probability value of each pixel represents the likelihood that the point is a feature point. Typically, points with probability values ​​higher than a certain preset threshold are selected as feature points. For each detected feature point, the SuperPoint network extracts a descriptor. The descriptor is a fixed-length vector used to represent the local feature information of the feature point. Descriptor extraction is accomplished through the network's feature extraction branch, which typically uses convolutional layers and pooling layers to extract local feature information.

[0067] The feature matching process is as follows:

[0068] At the core of the SuperGlue network is a graph neural network (GNN) used to learn the matching relationships between feature points. The GNN constructs a graph structure based on the descriptors and spatial location information of the feature points, and updates the feature representation of each node (feature point) through a message passing mechanism.

[0069] Through iterative updates of the GNN, a matching score is calculated between each pair of feature points. The matching score represents the probability that two feature points belong to the same physical point in two images. Based on the matching score, the feature point pair with the highest score is selected as the matching result. Methods such as non-maximum suppression are typically used to remove redundant matching points. Finally, the remaining matching point pairs represent the correspondence between the matching feature points, for example, (x, y). j ,(x′,y′) j .

[0070] This embodiment uses the least squares method to calculate the affine transformation matrix F that maps UAV image pixels to the satellite image coordinate system. The calculation steps are as follows:

[0071] S32. Let the mathematical form of the affine transformation matrix F be:

[0072]

[0073] Wherein, the affine transformation matrix F satisfies:

[0074]

[0075] S33. Based on the correspondence set {(x,y)} j ,(x′,y′) j The correspondence (x, y) between each pair of feature points in the sequence}, j = 1, ..., N. j ,(x′,y′) j Establish a system of equations, namely:

[0076] x = a 11 x′+a 12 y′+b1

[0077] y = a 21 x′+a 22 y′+b2

[0078] S34. Given N≥3 pairs of matching feature points, we need to solve for the parameter vector P=[a] of the affine transformation matrix F. 11 ,a 12 ,b1,a 21 [a2,b2] T The system of equations established by all matching points can be written in matrix form, that is:

[0079] AP = B, where A has 2N rows, and each pair of rows corresponds to an equation established based on matching feature points, i.e.:

[0080]

[0081] B = [x1, y1, x2, y2, ..., x N ,y N ] T

[0082] S35, The parameter vector P of the affine transformation matrix F = [a 11 ,a 12 ,b1,a 21 [a2,b2] T The least squares solution can be obtained by solving the normal equation, that is:

[0083] P=(A T A) -1 A T B

[0084] S36. Based on the calculated parameter vector P, and combined with P = [a 11 ,a 12 ,b1,a 21 [a2,b2] T Obtain the element a of the affine transformation matrix F 11 ,a 12 ,b1,a 21 ,a2,b2, and then construct the affine transformation matrix F, that is:

[0085]

[0086] In the formula, a 11 ,a 12 ,b1,a 21 a2 and b2 are elements obtained from the solution.

[0087] S4. Based on the affine transformation matrix F, determine the coordinates p of each pixel in the pixel region belonging to the person. i Mapped to pixel positions in the satellite image coordinate system, such as Figure 3 The example diagram shown illustrates the fusion of drone and satellite imagery. The final result is a set of pixel coordinates {q} representing the pixel region belonging to the person in the satellite image coordinate system. i}, i=1,...,m, the pixel mapping calculation process is as follows: q i =Fp i .

[0088] S5. Set the pixel coordinates q of the counter in the satellite image coordinate system. i The pixel coordinates p belonging to the person are mapped onto the top. i At that time, pixel coordinates q i The corresponding counter is incremented once, and then steps S2-S5 are repeated until every frame of the drone image in all valid video image data has been processed.

[0089] S6. After processing all video image data, the counters in the satellite image (baseline map) coordinate system are statistically analyzed. A lookup table corresponding to the maximum and minimum values ​​and pseudo-color is set up. Color values ​​are assigned to the pixel positions corresponding to the counters, and a personnel activity distribution map with the same pixel size as the satellite image is constructed. The specific process includes the following steps:

[0090] S61. Select a color list containing a color brightness range of [0, L-1]. For example, use a 256-color lookup table, where L = 256. The color lookup table can use the Jet Rainbow color lookup table. Calculate the [0, 255] fixed colors (R, G, B) of the Jet lookup table in advance and store them for later use.

[0091] S62. Based on the maximum value v of the counter max and minimum value v min Normalize all values ​​of the counters to the range [0, L-1], and calculate the new value v for each counter. new for:

[0092]

[0093] Among them, v old This is the actual value of the counter.

[0094] S63. For each pixel location in the satellite image (baseline image), based on the normalized new value v of its corresponding counter... new Find the index value and the new value v in the color list. new The color corresponding to the nearest index is assigned to the pixel position.

[0095] S64. Traverse all pixels to assign color values ​​to the entire satellite image, obtaining a pseudo-color image reflecting the distribution of human activity. For example... Figure 4 As shown, an example diagram of personnel activity distribution is given. The brighter the pixel area, the more frequent the personnel activity; conversely, the darker the pixel area, the sparser the personnel activity.

[0096] This embodiment automatically extracts pixel regions belonging to people from UAV images, then maps the extracted personnel pixels onto satellite images, accumulates personnel detection information on the pixel position counter of the satellite image, and finally performs pseudo-color rendering on the accumulated counter results to obtain a personnel activity distribution map.

[0097] This embodiment leverages the rapid aerial photography capabilities of drones and the global perspective of satellite imagery to fuse drone and satellite images, providing a method for quantitatively calculating human activity and generating a visually appealing pseudo-color human activity distribution map. The method proposed in this invention does not require access to personnel communication records or the trajectory information of data collection devices; instead, it achieves data acquisition and recording entirely based on passive remote sensing image observation.

[0098] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0099] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Since the above embodiments are substantially similar to the method embodiments, their descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0100] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for calculating personnel vitality distribution maps based on the fusion of UAV and satellite imagery, characterized in that, The method includes the following steps: S1. Select a satellite image covering the observation area as the base image, and then set a counter with an initial value of 0 for each pixel position q in the satellite image coordinate system; S2. Acquire video image data of the observation area taken by the drone, and detect and extract the pixel regions belonging to people in each frame of the drone image to obtain the pixel location set {p i }, i = 1, ..., m, where m is the number of extracted pixel positions; S3. Perform image feature matching on the UAV image and satellite image respectively, and calculate the affine transformation matrix F from the UAV image to the satellite image coordinate system based on the correspondence of the feature points obtained from the matching. S4. Based on the affine transformation matrix F, determine the coordinates p of each pixel in the pixel region belonging to the person. i Mapping these pixel positions to the satellite image coordinate system yields the set of pixel coordinates {q} representing the pixel region belonging to the person in the satellite image coordinate system. i }, i = 1, ..., m; S5. Set the pixel coordinates q of the counter in the satellite image coordinate system. i The pixel coordinates p belonging to the person are mapped onto the top. i At that time, pixel coordinates q i The corresponding counter is incremented once. S6. Repeat steps S2-S5 until all video image data has been processed. Statistically analyze the counters in the satellite image coordinate system and construct a personnel vitality distribution map with the same pixel size as the satellite image.

2. The method for calculating the personnel vitality distribution map according to claim 1, characterized in that, In step S2, the YOLOv11 or Mask RCNN network model is used to detect and extract the pixel regions belonging to people in each frame of the drone image.

3. The method for calculating the personnel vitality distribution map according to claim 1, characterized in that, In step S3, the specific process includes the following steps: S31. A deep neural network feature extraction and matching model is used to perform image feature matching on UAV images and satellite images to obtain the set of correspondences of matching feature points {(x,y)}. j ,(x′,y′) j }, j=1,...,N, where (x,y) j and (x′,y′) j , which are the pixel coordinates of the j-th pair of matching feature points in the satellite image and the UAV image, respectively, and N is the number of matching feature points; S32. Let the mathematical form of the affine transformation matrix F be: Wherein, the affine transformation matrix F satisfies: S33. Based on the correspondence set {(x,y)} j ,(x′,y′) j The correspondence (x, y) between each pair of feature points in the sequence}, j = 1, ..., N. j ,(x′,y′) j Establish a system of equations, namely: x=a 11 x′+a 12 y′+b1 and; 21 x′+a 22 y′+b2 S34. Given N≥3 pairs of matching feature points, we need to solve for the parameter vector P=[a] of the affine transformation matrix F. 11 ,a 12 ,b1,a 21 [a2,b2] T The system of equations established by all matching points can be written in matrix form, that is: AP = B, where A has 2N rows, and each pair of rows corresponds to an equation established based on matching feature points, i.e.: B=[x1,y1,x2,y2,...,x N ,y N ] T S35, The parameter vector P of the affine transformation matrix F = [a 11 ,a 12 ,b1,a 21 [a2,b2] T The least squares solution can be obtained by solving the normal equation, that is: P=(A T A) -1 A T B S36. Based on the calculated parameter vector P, and combined with P = [a 11 ,a 12 ,b1,a 21 [a2,b2] T Obtain the element a of the affine transformation matrix F 11 ,a 12 ,b1,a 21 ,a2,b2, and then construct the affine transformation matrix F, that is: In the formula, a 11 ,a 12 ,b1,a 21 a2 and b2 are elements obtained from the solution.

4. The method for calculating the personnel vitality distribution map according to claim 3, characterized in that, In step S31, the specific process includes the following steps: S311. Input the UAV image and satellite image into the SuperPoint network respectively to obtain the feature point detection probability map; S312. Select points with probability values ​​higher than a preset threshold in the feature point detection probability map as feature points, and extract a descriptor for each feature point to represent the local feature information of the feature point. S313. A graph structure is constructed using the SuperGlue network based on the descriptors and spatial location information of the feature points, and the feature representation of each feature point is updated through a message passing mechanism. S314. Calculate the matching score between each pair of feature points to represent the probability that they belong to the same physical point in the two images; S315. Select the feature point pair with the highest matching score as the correspondence of the matching feature points.

5. The method for calculating the personnel vitality distribution map according to claim 1, characterized in that, In step S4, the pixel mapping calculation process is as follows: q i =Fp i .

6. The method for calculating the personnel vitality distribution map according to claim 1, characterized in that, In step S6, the specific process includes the following steps: S61. Select a color list containing colors with a brightness range of [0, L-1]. S62. Based on the maximum value v of the counter max and minimum value v min Normalize all values ​​of the counters to the range [0, L-1], and calculate the new value v for each counter. new ; S63. For each pixel location in the satellite image, based on the new value v of its corresponding counter... new Find the index value and the new value v in the color list. new The color corresponding to the nearest index is assigned to that pixel position; S64. Traverse all pixels to assign color values ​​to the entire satellite image, obtaining a pseudo-color image reflecting the distribution of human activity.

7. The method for calculating the personnel vitality distribution map according to claim 6, characterized in that, In step S61, the color list uses a 256-color lookup table, L = 256; The color lookup table uses the Jet rainbow color lookup table, with [0, 255] fixed colors from the Jet lookup table pre-calculated and stored for later use.

8. The method for calculating the personnel vitality distribution map according to claim 6, characterized in that, In step S62, the new value v new The calculation formula is: Among them, v old This is the actual value of the counter.

9. The method for calculating the personnel vitality distribution map according to claim 6, characterized in that, In step S64, the brighter the pixel area in the pseudo-color image, the more frequent the human activity; conversely, the darker the pixel area, the sparser the human activity.

Images