Urban air-ground radio map redrawing method based on multi-modal information fusion

Abstract

This invention presents a multimodal information fusion-based urban air-to-ground radio map redrawing method. By integrating prior knowledge from partially incomplete maps with a small amount of RSS measurements, a more comprehensive and robust information representation can be obtained. Through modal fusion of multi-class occlusion wireless channel models and multi-class virtual obstacle map models, wireless signal features and geographic information features are used together to redraw the radio map, better capturing their correlation and complementarity. The method of this invention is experimentally validated using an open-source RSS measurement dataset and compared with single-modal methods to evaluate its beneficial effects. The evaluation results demonstrate the advantages of this invention in wireless signal reception strength estimation and virtual obstacle map redrawing tasks, and show its effectiveness in reducing the amount of data required for the task and improving redrawing accuracy.

Description

Technical Field

[0001] This invention relates to the field of wireless communication, and more specifically, to a method for redrawing urban air-ground radio maps based on multimodal information fusion. Background Technology

[0002] In future communication networks, understanding geographic information and integrating it with air and ground wireless communications will be a crucial task. Radio maps, as a data structure representing the quality of wireless channels between geographic locations, have been widely regarded in recent years as a powerful indicator of the wireless environment, playing a vital role in various fields such as network monitoring and planning, indoor and outdoor positioning, traffic assistance, IoT data collection, and air quality monitoring. In these scenarios, radio signal sampling is performed using Received Signal Strength (RSS) over a region. However, constructing radio maps is extremely difficult due to the following challenges: First, constructing detailed radio maps requires a large amount of RSS data, and data acquisition is very time-consuming. Second, finding a suitable model for radio maps is challenging. Air-ground channels depend on the geographic environment, and their formation mechanisms are complex, involving signal reflection, diffraction, and scattering. Because radio maps change rapidly, it is difficult to obtain real-time, fine-grained radio maps over large areas. In recent years, with advancements in drone manufacturing technology and cost reductions, drones have been widely used in the communications field. Due to their unique 3D flexibility and aerial advantages, using drones as aerial anchor points to collect RSS data and assist in radio map creation has become an effective solution.

[0003] Currently, there are two main methods for using drones to assist in radio map creation. One method combines 3D city maps with traditional wireless channel parameter estimation methods for radio map redrawing; the other utilizes deep learning to explore the implicit mapping between RSS and obstacle occlusion levels from RSS data for radio map redrawing. However, both methods have their own shortcomings. For the first method, the main drawback is that wireless channel parameter estimation and virtual obstacle map redrawing are often complex and highly coupled in real-world scenarios. Using 3D city maps as prior knowledge only provides geographic information for local areas and may be incomplete or inaccurate. The other method suffers from the drawback that its redrawing results may be affected by the quality of the RSS data and requires a large amount of data, leading to instability and inaccuracy in radio map redrawing. Furthermore, training a deep learning algorithm typically takes a long time. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method for redrawing urban air-ground radio maps based on multimodal information fusion, which aims to reduce the amount of data required for the task and improve the accuracy of redrawing, in order to address the above-mentioned deficiencies of the aforementioned single-modal method.

[0005] The technical solution adopted by this invention to solve its technical problem is: to construct a method for redrawing urban air-ground radio maps based on multimodal information fusion, including...

[0006] S1. The virtual obstacle map generation network is pre-trained based on 3D maps of some cities and the initial network parameters are fixed.

[0007] S2. Based on the communication channel obstruction information between the UAV and ground users, the weight generation network is pre-trained and the initial network parameters are fixed.

[0008] S3. Based on the positional relationship between the UAV and the ground user and the received wireless channel signal strength, jointly optimize the virtual obstacle map generation network and the wireless channel parameter prediction network;

[0009] S4. Generate the urban air-ground radio map based on the optimized virtual obstacle map generation network, and predict the wireless channel parameters based on the optimized wireless channel parameter prediction network.

[0010] In the urban air-ground radio map redrawing method based on multimodal information fusion described in this invention, step S1 includes:

[0011] S11. Use an open-source geographic information dataset as a mask matrix to mask the original 3D city map to obtain the partial 3D city map.

[0012] S12. Use the original 3D city map as a label, and pre-train the virtual obstacle map generation network based on the partial 3D city map and store the network parameters.

[0013] In the urban air-ground radio map redrawing method based on multimodal information fusion described in this invention, the virtual obstacle map generation network includes two cascaded Unets. The partial 3D urban map is used as the input of the first Unet to redraw the initial virtual obstacle map. The initial virtual obstacle map and the partial 3D urban map are simultaneously input into the second Unet to train the virtual obstacle map generation network and store the network parameters.

[0014] In the urban air-ground radio map redrawing method based on multimodal information fusion described in this invention, step S2 includes:

[0015] S21. The weight generation network is constructed using a first classification strategy network and a second classification strategy network. The output results of the first classification strategy network and the second classification strategy network are both representations of the occlusion intensity type of a certain wireless channel link in various occlusion states in the virtual obstacle map.

[0016] S22. Train the first classification strategy network using the first source data and the first classification label respectively until it converges;

[0017] S23. Train the second classification strategy network using the second source data and the second classification label until it converges.

[0018] The first classification policy network is a neural network composed of several linear layers and activation functions, while the second classification policy network is a neural network composed of activation functions and a softmax function.

[0019] In the urban air-ground radio map redrawing method based on multimodal information fusion described in this invention, in step S22, the first classification label...

[0020]

[0021] in,

[0022] Among them, SumHeightDiff k Let z represent the height difference, which is the z-value at position corresponding to the k-th occlusion type on each wireless channel link p. m with h m,k The difference between the heights and z, where z m h represents the altitude value at the line connecting the ground user and the UAV's 3D position corresponding to the wireless channel link P. m,k The virtual obstacle height value represents the position of the line connecting the 3D positions of the ground user and the UAV corresponding to the wireless channel link P; Upperbound represents the upper limit of the cumulative occlusion height; s represents the occlusion weight allocation strategy based on the inverse exponential function. The value of k represents the influence factor of the two types of occlusion intensity determination methods. k can be 0, 1 or 2, and λ is the coefficient of the inverse exponential distribution, which determines the steepness of the inverse exponential distribution curve. In our experimental setup, λ is set to 1.

[0023] In the urban air-ground radio map redrawing method based on multimodal information fusion described in this invention, in step S22, the second classification label...

[0024] Where pathloss represents the actual loss of the received signal strength in the wireless channel, and This represents the predicted loss value for the received signal strength of a wireless channel.

[0025] In the urban air-ground radio map redrawing method based on multimodal information fusion described in this invention, the occlusion intensity types include line-of-sight links, soft occlusion caused by multipath effects with occlusion type k=1, and severe occlusion caused by concrete structures with occlusion type k=2.

[0026] In the urban air-ground radio map redrawing method based on multimodal information fusion described in this invention, step S3 further includes:

[0027] S31. The first Adam optimizer is used to fix the parameters of the wireless channel parameter prediction network while the virtual obstacle map generation network is fixed, and the parameters of the virtual obstacle map generation network are corrected by calculating the least square difference between the first classification label and the second classification label.

[0028] S32. The second Adam optimizer is used to optimize the wireless channel parameter prediction network. The first classification label and the second classification label are fixed at the same time. The parameters of the wireless channel parameter prediction network are corrected by calculating the least square difference between the actual value of the loss of the wireless channel signal reception strength and the predicted value of the loss of the wireless channel signal reception strength under different obstruction conditions.

[0029] In the urban air-ground radio map redrawing method based on multimodal information fusion described in this invention, in step S4, the smaller the minimum squared difference between the first classification label and the second classification label, the smaller the difference between the urban air-ground radio map generated by the virtual obstacle map generation network and the actual 3D urban map.

[0030] The smaller the minimum squared difference between the actual value of the loss of the wireless channel signal reception strength and the predicted value of the loss of the wireless channel signal reception strength, the closer the wireless channel parameters predicted by the wireless channel parameter prediction network are to the true values.

[0031] The urban air-ground radio map redrawing method based on multimodal information fusion of this invention improves the accuracy and reliability of location estimation and environmental perception through multimodal information fusion. By combining some incomplete map prior knowledge and a small amount of RSS measurements, a more comprehensive and robust information representation can be obtained. Through modal fusion of multi-class occlusion wireless channel models and multi-class virtual obstacle map models, wireless signal features and geographic information features are used together to redraw the radio map, so as to better capture their correlation and complementarity. The urban air-ground radio map redrawing method based on multimodal information fusion of this invention can be experimentally verified using an open-source RSS measurement dataset, and compared with single-modal methods to evaluate its beneficial effects. The evaluation experimental results demonstrate the advantages of the urban air-ground radio map redrawing method based on multimodal information fusion of this invention in wireless signal received strength estimation and virtual obstacle map redrawing tasks, and show its effectiveness in reducing the amount of data required for the task and improving the redrawing accuracy. Attached Figure Description

[0032] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0033] Figure 1 This is a flowchart of a preferred embodiment of the urban air-ground radio map redrawing method based on multimodal information fusion of the present invention;

[0034] Figure 2 This is a schematic diagram of the training process of a preferred embodiment of the urban air-ground radio map redrawing method based on multimodal information fusion of the present invention;

[0035] Figure 3 The algorithm of a preferred embodiment of the urban air-ground radio map redrawing method based on multimodal information fusion of the present invention is shown;

[0036] Figure 4 The experimental results of the urban air-ground radio map redrawing method based on multimodal information fusion of the present invention are shown;

[0037] Figures 5A-5C The present invention provides a comparison of the urban air-ground radio map redrawing method based on multimodal information fusion of the present invention with other existing urban air-ground radio map redrawing methods. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0039] The urban air-ground radio map redrawing method based on multimodal information fusion of this invention improves the accuracy and reliability of location estimation and environmental perception through multimodal information fusion. By combining some incomplete map prior knowledge and a small amount of RSS measurements, a more comprehensive and robust information representation can be obtained. Through modal fusion of multi-class occlusion wireless channel models and multi-class virtual obstacle map models, wireless signal features and geographic information features are used together to redraw the radio map, so as to better capture their correlation and complementarity. The urban air-ground radio map redrawing method based on multimodal information fusion of this invention can be experimentally verified using an open-source RSS measurement dataset, and compared with single-modal methods to evaluate its beneficial effects. The evaluation experimental results demonstrate the advantages of the urban air-ground radio map redrawing method based on multimodal information fusion of this invention in wireless signal received strength estimation and virtual obstacle map redrawing tasks, and show its effectiveness in reducing the amount of data required for the task and improving the redrawing accuracy.

[0040] The urban air-to-ground radio map redrawing method based on multimodal information fusion of the present invention aims to use the wireless channel parameters of UAVs and ground users and the partial urban map as prior knowledge, combined with the multimodal information of radio channel strength measurement (RSS), to jointly generate a radio map that can reflect the signal reception strength between any two given points.

[0041] The main tasks involved in creating radio maps with the assistance of drones are twofold: creating virtual obstacle maps and predicting wireless channel parameters. Since the link between ground users and drones is often obstructed by obstacles in the geographical environment, we first construct a wireless channel model based on multiple types of occlusion for subsequent virtual obstacle map creation and wireless channel parameter prediction.

[0042] For the link between ground users and drones, we use p = (p u ,p d )∈R 6 Description, where p u ,p d ∈R 3 These represent the 3D positions of the ground user and the aerial drone, respectively. The wireless signal is located at the 3D position p of the ground user. u 3D position of aerial drones p dDuring transmission, wireless signals are affected by obstacles in the geographical environment, resulting in multipath effects. Specifically, obstacles in the geographical environment, such as undulating terrain and vegetation, tall buildings, and glass facades in cities, cause wireless signals to undergo multiple reflections, refractions, and diffractions during transmission. These multiple reflection, refraction, and diffraction paths result in multiple versions of the wireless signal arriving at the receiving end with different time delays, phases, and amplitudes. These wireless signals are superimposed at the receiving end, forming a multipath effect, which significantly impacts the quality and reliability of the communication signal. Starting from the traditional logarithmic path loss model and considering the various channel parameter cases caused by multipath effects, we adopt the following classic wireless channel model to describe this large-scale fading signal caused by multipath effects:

[0043]

[0044] This is a wireless channel parameter model based on 0-1 binary classification, where g(p) represents the path loss of signal strength in wireless channel link P due to multipath effects, and D... k This represents the k-th type of occlusion. This indicates that the wireless channel link P is at D. k The occlusion type, where k takes the value 0 or 1. When k = 0, it indicates that the wireless channel link P is in a line-of-sight (LOS) state; when k = 1, it indicates that the wireless channel link P is in a non-line-of-sight (NLOS) state. β k ,α k This represents a pair of channel parameters for a wireless channel link P. For different types of obstruction, there is a corresponding pair of channel parameters (β). k ,α k Correspondingly, under different occlusion conditions, (β) k ,α k The value of α varies to distinguish the differences in wireless channel quality under different obstruction conditions. For example, comparing line-of-sight links, when the wireless channel link is in a non-line-of-sight link situation, α... k The value of will be larger to reflect the faster attenuation of the wireless channel signal under more severe occlusion. Here, we extend the occlusion situation considered in this model to a maximum of k=2, that is, add an intermediate state to the binary classification wireless channel parameter model to represent the multipath effect caused by reflection, refraction, and diffraction. Because its wireless channel quality is between that of a line-of-sight link and a completely non-line-of-sight link, it is set to k=1, while the original non-line-of-sight link occlusion type is changed to k=2. Therefore, our channel model based on multi-class occlusion is updated as follows:

[0045]

[0046] To correspond to our wireless channel model based on multi-class occlusion, we introduce a virtual obstacle map H to represent the virtual obstacle situation in the target area, where H ∈ R M×N×K , where M×N represents the pixel points on a certain type of virtual obstacle map, and K represents that the virtual obstacle map belongs to the k-th type of occlusion. The value of each pixel point in the virtual obstacle map H represents the height of the virtual obstacle at the corresponding position, and the value of k is 0, 1, or 2.

[0047] We believe that when k = 0, H k＝0 reflects the virtual obstacle map of some vegetation types; when k = 1, the corresponding layer of H k＝1 simulates the virtual obstacle map of the soft occlusion type generated by reflection, diffraction, and multi-path effects, etc.; and when k = 2, the corresponding layer of H k＝2 reflects the virtual obstacle map of the concrete structure building. Therefore, the virtual obstacle map H can correspond to and match the multi-class occlusion channel model, reflecting the three-dimensional spatial geographical structure in the target scenario we set. Additionally, it is worth mentioning that the final virtual obstacle map we generate may not be exactly the same as the 3D geographical map of the actual target scenario, because our virtual obstacle map reflects an obstacle form related to the channel state.

[0048] For the classification strategy of the occlusion types on the virtual obstacle map, it corresponds to the classification strategy in our multi-class occlusion signal model. We abstract it as S k (p; H), and define it as:

[0049]

[0050] where z m represents the height value at the connection position of the line between the ground user and the 3D position of the UAV corresponding to the wireless channel link P, h m,k represents the value at the corresponding pixel point on the k-th type of virtual obstacle map H k , that is, the height of the virtual obstacle at the corresponding position on the k-th type of virtual obstacle map H k , m represents each independent grid, represents the set of ground grids of a straight line covered and affected by the connection line of the corresponding ground user and UAV. For all l < k, it represents the occlusion types with occlusion types lower than k, that is, the occlusion types with lower occlusion degrees. That is, l also represents the occlusion type, but its occlusion degree is lower than that of the k-th type of occlusion. h m,l represents the value at the corresponding pixel point on the l-th type of virtual obstacle map H l . For example: If for a wireless channel link p, its corresponding z m is higher than the l-th type of occlusion and lower than the k-th type of occlusion, then the occlusion category of this link p is the l-th type.

[0051] Therefore, we define the radio map model based on multi-class occlusion as:

[0052]

[0053] During actual training, we found that this classification strategy may be too extreme when the data sample size is small, leading to significant errors in occlusion classification and map correction. With limited available samples, each data point and its associated link will affect the pixel value of a line segment on the corresponding occlusion category's virtual obstacle map. Given the small amount of data, this occlusion classification method is too aggressive. Therefore, we subsequently proposed a smoother occlusion classification method to help address this issue and improve map redrawing accuracy during network training.

[0054] This invention proposes a method for redrawing urban air-ground radio maps based on multimodal information fusion, namely, the aforementioned method for drawing virtual obstacle maps H. The specific steps are as follows: Figure 1 As shown, the training process is as follows: Figure 2 As shown. Figure 1 As shown, in step S1, the virtual obstacle map generation network is pre-trained based on a partial 3D city map, and the initial network parameters are fixed. In a preferred embodiment of the invention, an open-source geographic information dataset is used as a mask matrix to mask the original 3D city map to obtain the partial 3D city map; the original 3D city map is used as a label, and the virtual obstacle map generation network is pre-trained based on the partial 3D city map and the network parameters are stored. For example, the virtual obstacle map generation network may include two cascaded Unets.

[0055] In a preferred embodiment of the present invention, the original 3D city map may be actual 3D map data or 3D map measurement data obtained by means of methods such as remote sensing and depth measurement.

[0056] The following is combined with Figure 2 The specific training process is explained below. For example... Figure 2 As shown, in the first stage, a Unet-based virtual obstacle map generation network is trained using some open-source geographic information datasets. We mask the original 3D city map and use the original 3D city map as a label to train the network parameters of the Unet-based virtual obstacle map generation network, aiming to redraw a coarsely initialized 3D map using some prior knowledge of the map.

[0057] For initializing the redrawing of the original 3D map based on partial maps, we employ a lossy image redrawing method based on generative adversarial networks (GANs). This method uses two cascaded UNets with skip connections. UNet is a deep learning network model for image semantic segmentation. It has an encoder-decoder structure, where the encoder extracts high-level feature representations of the image, while the decoder recovers the spatial details of the segmentation result through upsampling and skip connections. Skip connections allow information to be passed between different layers of the network, helping to preserve image details and contextual information—in conjunction with its proposed Coherent Semantic Attention (CSA) layer embedded in the second stage, the damaged image is redrawn through adversarial training of the overall image and network parameters by two pairs of generators and discriminators, respectively. First, each unknown feature pixel in the unknown region is initialized using feature pixels from known regions; then, iterative optimization is performed by considering the spatial consistency of adjacent pixels to maintain global semantic consistency and local feature consistency. This method effectively fills in missing regions in the map pixel matrix while preserving the feature map structure by modeling the semantic correlation between feature pixels in unknown regions.

[0058] In data processing, dimensional expansion is a common operation. Through dimensional expansion, we can insert a new dimension at a specific location in existing data, thereby increasing the data's representational power and flexibility. To complement the various virtual obstacle map models mentioned earlier, we abstract the original 3D city map into a map pixel matrix based on the obstacle heights at corresponding locations. We then define the original ground city map H1∈R... 2 Expand along the z-axis dimension, dividing the original pixel position into three equal parts and initializing them as H2∈R. 3 We used an irregular mask dataset (an open-source dataset provided by Liu et al.) as our original 3D city map. We then used this dataset as the mask matrix, which is categorized based on the percentage of image area occupied by different shaped masks (e.g., 10%-60% area masks). During training, we used randomized mask matrices, setting the positions of the corresponding mask holes on a copy of the original 3D city map H2 to 0, indicating that we do not know the building height and distribution in that area. This method yielded the desired results. It is used as input to the virtual obstacle map generation network, and after being recovered by the first UNet, it generates... Reuse and The second UNet redrawing network, which contains a CSA layer, is used as the common input to produce a virtual obstacle map as the output. Using the original 3D city map H2 as labels, this part of the network was pre-trained in several batches to improve the 3D virtual obstacle map. The network gradually approaches the original 3D city map H2, and then saves the network parameters, enabling the virtual obstacle map generation network to initially possess the ability to roughly redraw the original 3D virtual obstacle map based on partial city 3D maps.

[0059] In step S2, the weight generation network is pre-trained and its initial parameters are fixed based on the occlusion information of the communication channel between the UAV and the ground user. In a preferred embodiment of the present invention, our proposed two-class occlusion classification strategy uses the wireless channel received signal strength data generated from noise to train the classification weight network (ClassificationWeight Net, hereinafter referred to as WeightNet) and fix its parameters, which is then used as a classifier in subsequent training.

[0060] Since we cannot directly obtain the occlusion category of a data point and its corresponding link from the dataset, we pre-set a relaxed pre-label generation rule based on (1.3) to facilitate neural network training. We designed two nonlinear classification policy networks to form a classification weight network WeightNet, which will be referred to as PretrainNet1 and PretrainNet2, respectively. PretrainNet1 is a simple neural network composed of several linear layers and activation functions, while PretrainNet2 is a simple neural network composed of activation functions and a softmax function. The output of each classification policy network is a three-dimensional normalized vector η, which represents the occlusion intensity type of a wireless channel link p in various occlusion states in the virtual obstacle map H.

[0061] For the classification policy network PretrainNet1, we employ a smoothing strategy to pre-label the training data. First, we define a numerical value as height difference and SumHeightDiff. k

[0062] SumHeightDiff k =∑ m (h m,k -z m )I{h m,k <z m} (2.1)

[0063] SumHeightDiff=∑ k SumHeightDiff k (2.2)

[0064] Height difference and SumHeightDiff kThis represents the z-value at position k for each type of occlusion on a wireless channel link p. m with h m,k The difference between the heights and z, where z m h represents the altitude value at the line connecting the ground user and the UAV's 3D position corresponding to the wireless channel link P. m,k The virtual obstacle height value represents the position of the line connecting the ground user and the UAV's 3D positions corresponding to the wireless channel link P; the sum of height differences (SumHeightDiff) represents the z-axis height at the corresponding position for all occlusion types on each wireless channel link p. m with h m,k The difference between the height and the height; I{h m,k <z m} indicates that it only exists in h m,k <z m We take 1 as an important indicator for measuring the degree of occlusion.

[0065] For the classification strategy S in the multi-class occlusion signal model k (p:H) uses η to represent the occlusion intensity type among various occlusion categories. Ideally, the values ​​of η in the three dimensions represent different occlusion intensity types. For example, (1,0,0) represents the lightest line-of-sight (LOS) occlusion type, (0,1,0) represents soft occlusion caused by multipath effects (k=1), and (0,0,1) represents severe occlusion caused by concrete structures (k=2). However, in actual neural network training, we perform relaxation processing, and the values ​​in the corresponding dimensions reflect the weight ratio of the wireless channel link p in the corresponding occlusion level. We combine the positional relationship between the virtual obstacle map H and the wireless channel link p with the occlusion weight allocation strategy based on the inverse exponential function after relaxation processing as follows:

[0066]

[0067] Upperbound is the upper limit of the cumulative occlusion height we set based on the problem scenario. When the absolute value of the sum of height differences (SumHeightDiff) exceeds the upper limit of the cumulative occlusion height (Upperbound), we consider the occlusion to be quite severe and can classify it as complete severe occlusion, i.e., the occlusion intensity type η is (0,0,1). Additionally, if the value of the sum of height differences (SumHeightDiff) is 0, that is, when all points on all links in the link are higher than the virtual obstacle height of all occlusion types (for any k: z...), then... m h m,kThis link is considered a complete line-of-sight (LOS) link, belonging to the lightest level of occlusion. Therefore, the occlusion intensity type η is set to (1,0,0). The two methods for determining the intensity of occlusion are represented by the influence factor, and λ represents the coefficient of the inverse exponential distribution, which determines the steepness of the inverse exponential distribution curve.

[0068] In other cases, we compare the sum of height differences (SumHeightDiff) with our preset cumulative occlusion height upper bound (UpperBound). For cases below the cumulative occlusion height upper bound but with obvious occlusion, we use two decision-making methods to dynamically adjust the occlusion intensity type (η) in the final generated occlusion types, and use... As weights, they are used to balance the two decisions in actual training. The value of SumHeightDiff varies depending on the data distribution of the dataset, used to weigh the impact of the two strategies on the final label generation process. The two decision-making methods are as follows: First, we use an inverse exponential function to determine the value of its third dimension. For a specific wireless communication link p, the closer the sum of height differences (SumHeightDiff) is to the upper bound of the cumulative occlusion height, the larger the value of its third dimension, meaning the final weight classification standard is closer to the classification of more severe occlusion at k=2. To maintain the normalization of the weight vector, the values ​​in the second dimension are complementary to those in the third dimension, which are above 1. As for the first dimension, since there is a clear case where the sum of height differences (SumHeightDiff) > 0, it does not meet the condition of the lightest line-of-sight link, so it is set to 0. In the second strategy, we use SumHeightDiffkS output by the Softmax function. SumHeightDiffkS can more accurately reflect the original height differences and z values ​​in different dimensions of SumHeightDiffk. m and h m,k The height difference is mainly to address the issue that in extreme cases, when the SumHeightDiff is too large due to the excessively high value of moderate obstacle occlusion at k=1, the factors with a larger weight in the second dimension cannot be reflected in the final weight matrix.

[0069] We will then use the source data generated from noise with a given mean and variance, and its corresponding label η. PretrainNet1 Both were used as training data and fed into the classification policy network PretrainNet1 for training. After several iterations, the classification policy network PretrainNet1 converged and served as a discriminator in the actual training process, judging based on the link location and the height difference between the virtual obstacle maps, to help train our map generation network.

[0070] The classification policy network PretrainNet2 is a simple neural network composed of activation and softmax functions. Corresponding to the channel parameters for each type of occlusion in our multi-class occlusion wireless channel model, we obtain the predicted pathloss value for the received wireless channel signal strength corresponding to each type of occlusion after receiving the logarithmic path distance between the drone and the ground user. PretrainNet2 compares the difference between the predicted pathloss value and the actual pathloss value in each dimension, intuitively selecting the dimension with the smallest difference as the occlusion type to which the data belongs.

[0071]

[0072] During training, we also use noise with a given mean and variance, and the label η. PretrainNet2 The network parameters are fixed after several batches, and like the classification policy network PretrainNet1, these network parameters will not be modified or changed in subsequent network training. They are used as another discriminator for occlusion category discrimination using the angle of wireless signal measurement.

[0073] In step S3, the virtual obstacle map generation network and the wireless channel parameter prediction network are jointly optimized based on the positional relationship between the UAV and the ground user and the wireless channel signal reception strength. In a preferred embodiment of the present invention, after a stage of pre-training, the wireless channel parameters and the virtual obstacle map are jointly optimized and trained on the target map based on the actual positional relationship between the UAV and the ground user and the loss of wireless channel signal reception strength.

[0074] Because our radio map model has a strong coupling between the radio channel parameters and the occlusion weight type on each radio channel link p, we use a semi-self-supervised method to better address the tasks of fitting channel parameters and correcting the generated virtual obstacle map in a single training iteration. This method employs two Adam optimizers to iteratively update the radio channel parameter prediction network and the virtual obstacle map generation network, respectively, thereby updating both the radio channel parameters and the virtual obstacle map in a single training iteration. The specific algorithm is as follows: Figure 3 As shown.

[0075] When using the Adam optimizer to refine the virtual obstacle map generation network, we temporarily fix the parameters of the wireless channel parameter prediction network (ParaNet) representing channel parameters under various occlusion states in the model, and calculate the first classification label η. PretrainNet1Second category label η PretrainNet2 The least squared error (MSE) loss1 is used to indirectly adjust and correct the parameters of the virtual obstacle map generation network, so as to achieve consistency between the weight classification output based on the difference between the virtual obstacle map and the actual geographical height and the obstacle type weight classification based on the wireless channel signal loss. As the loss1 gradually decreases, we can assume that the difference between the virtual obstacle map generated by the virtual obstacle map generation network (i.e., the final generated urban air-ground radio map) and the actual 3D urban map in the real area is gradually decreasing.

[0076] When using the Adam optimizer to correct the wireless channel parameter prediction network, for the classification weight vector η of the occlusion weight, i.e. the first classification label and the second classification label are fixed, we calculate the least squared difference loss2 between the wireless channel signal loss and the actual value of the wireless signal strength under the three types of occlusion to update the parameters on each layer of the wireless channel parameter prediction network. When this loss2 gradually decreases, we can assume that the difference between the actual value of the wireless channel signal reception strength loss and the predicted value of the wireless channel signal reception strength loss is decreasing, that is, the predicted wireless channel parameters are gradually approaching the channel parameters in the real scene.

[0077] We validated the urban air-to-ground radio map redrawing method based on multimodal information fusion proposed in this invention using known open-source datasets (UAV and ground sensor channel strength datasets). The datasets include both field-measured data and simulated data. The field-measured data is taken from a 300*300m area in the central urban area of ​​Shanghai, containing numerous buildings ranging in height from 10m to 130m. The 3D geographic map data is taken from the OpenStreetmap geographic city map dataset. The simulated data was generated using the 3D simulation software Remcom. For the simulation data, a ray-tracing model was used within the corresponding geographic area, simulating up to 6 reflections and 1 diffraction, with a narrowband sine wave at a frequency of 2.5GHz. The dataset structure is {p(i), y(i)}, where p(i) is the location of the transmitter and receiver pair (i.e., the location of the UAV and the ground user), and y(i) is the corresponding RSS value of the wireless channel in dB. We used data from areas where drones and ground sensors were located in the central urban area, totaling approximately 520,000 data entries. The actual map size was approximately 100*100 meters. We used two Adam optimizers with learning rates of 3e-2 and 3e-5, respectively, and optimized and updated the wireless channel parameter prediction network and the virtual obstacle map generation network with 3 iterations, 6 iterations, and a batch size of 128.

[0078] We display the redrawn virtual obstacle map (i.e., the generated urban open-air radio map) in a 256*256 tensor and use the least squares difference between it and the actual 3D urban map to measure the redrawing difference. For the pathloss prediction part of the wireless channel signal reception strength, we use the least squares difference between the actual and predicted values ​​of the wireless channel signal reception strength loss to measure the accuracy of the path loss prediction. We simulated three masks (A, B, C) with different mask areas, and the differences in map redrawing degree and pathloss prediction accuracy when using 1%, 5%, and 10% of the total data volume. The results are as follows. Figure 4 As shown, for example, A1 represents a simulation experiment using mask A with a total data volume of 1%, and a set of simulated fake data is used to represent the pathloss distribution of wireless channel signal reception strength on a horizontal plane with a height of 20m (channel gain distribution in the figure). It can be seen from the picture that due to the presence and obstruction of tall buildings, the pathloss distribution of wireless channel signal reception strength is not uniform in all directions.

[0079] We then compared the redrawing effects of several similar methods on the radio map in the same scenario. Here, we compare the results of our experimental group C, as shown below. Figures 5A-5C As shown, benchmark 1a represents a radio map redrawing method based on convolutional networks, benchmark 1b represents a radio map redrawing method based on linear networks, benchmark 2 represents a traditional non-machine learning method for radio map redrawing, including multiple methods such as kriging (benchmark2(kriging)) and ann (benchmark2(ann)), and benchmark 3 represents the ideal state when our method knows the complete input map information. Ours represents the urban air-ground radio map redrawing method based on multimodal information fusion of the present invention. Figure 5A The paper illustrates the differences in path loss restoration of wireless channel signal reception strength compared to several benchmarks using the urban air-ground radio map redrawing method based on multimodal information fusion of the present invention with different amounts of data. Figure 5B The paper illustrates the trends in path loss prediction accuracy for wireless channel signal reception strength (WSS) across several methods as a function of training epochs. It can be seen that after several training epochs, the urban air-ground radio map redrawing method based on multimodal information fusion of this invention achieves prediction accuracy for WSS using only a small amount of data that is very close to that of similar methods. Furthermore, at the initial stage of training, the WSS path loss decreases faster using the method of this invention, because the partial map prior information we introduce plays a crucial role in occlusion classification. The best-performing method among similar approaches, benchmark1b, requires several times longer to train.

[0080] Figure 5C When comparing the accuracy of several methods for radio map redrawing, the urban air-ground radio map redrawing method based on multimodal information fusion of the present invention introduces part of the map as prior information, which contains more knowledge of the distribution and morphology of buildings in the actual scene. Therefore, our advantage is more significant in this respect, especially when the amount of data is small, we can still obtain a redrawn radio map that is closer to the actual situation.

[0081] The urban air-ground radio map redrawing method based on multimodal information fusion of the present invention uses partial map prior knowledge as an aid to first initialize a virtual obstacle map, enabling high-quality reconstruction of wireless channel parameters and virtual obstacle maps quickly when channel measurement data is limited; it uses two types of decision-making methods, relaxation and intuitiveness, to generate obstacle occlusion weights on path pairs, improve classification differences, and assist the network in rapid training.

[0082] Although the present invention has been described through specific embodiments, those skilled in the art will understand that various modifications and equivalent substitutions can be made to the invention without departing from its scope. Furthermore, various modifications can be made to the invention for specific situations or materials without departing from its scope. Therefore, the present invention is not limited to the specific embodiments disclosed, but should include all embodiments falling within the scope of the claims.

[0083] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for redrawing urban air-ground radio maps based on multimodal information fusion, characterized in that, include: S1. The virtual obstacle map generation network is pre-trained based on 3D maps of some cities and the initial network parameters are fixed. S2. Based on the communication channel obstruction information between the UAV and ground users, the weight generation network is pre-trained and the initial network parameters are fixed. S3. Based on the positional relationship between the UAV and the ground user and the received wireless channel signal strength, jointly optimize the virtual obstacle map generation network and the wireless channel parameter prediction network; S4. Generate the urban air-ground radio map based on the optimized virtual obstacle map generation network, and predict the wireless channel parameters based on the optimized wireless channel parameter prediction network. Step S1 includes: S11. Use an open-source geographic information dataset as a mask matrix to mask the original 3D city map to obtain the partial 3D city map. S12. Use the original 3D city map as a label, and pre-train the virtual obstacle map generation network based on the partial 3D city map and store the network parameters; Step S2 includes: S21. The weight generation network is constructed using a first classification strategy network and a second classification strategy network. The output results of the first classification strategy network and the second classification strategy network are both representations of the occlusion intensity type of a certain wireless channel link in various occlusion states in the virtual obstacle map. S22. Train the first classification strategy network using the first source data and the first classification label respectively until it converges; S23. Train the second classification strategy network using the second source data and the second classification label until it converges. The first classification strategy network is a neural network composed of several linear layers and activation functions, and the second classification strategy network is a neural network composed of activation functions and a softmax function. In step S22, the first classification label in, Among them, SumHeightDiff k Let z represent the height difference, which is the z-value at position corresponding to the k-th occlusion type on each wireless channel link p. m with h m,k The difference between the heights and z, where z m h represents the altitude value at the line connecting the ground user and the UAV's 3D position corresponding to the wireless channel link P. m,k The virtual obstacle height value represents the line connecting the ground user and the UAV's 3D position corresponding to the wireless channel link P; Upperbound represents the upper limit of the cumulative occlusion height; s represents the occlusion weight allocation strategy based on the inverse exponential function; w represents the influence factor of the two occlusion intensity type determination methods; k takes the value of 0, 1 or 2; and λ is the coefficient of the inverse exponential distribution. In step S23, the second classification label Where pathloss represents the actual loss of the received signal strength in the wireless channel, and This represents the predicted loss value for the received signal strength of a wireless channel.

2. The urban air-ground radio map redrawing method based on multimodal information fusion according to claim 1, characterized in that, The virtual obstacle map generation network includes two cascaded Unets. The partial 3D city map is used as input to the first Unet to redraw the initial virtual obstacle map. The initial virtual obstacle map and the partial 3D city map are simultaneously input to the second Unet to train the virtual obstacle map generation network and store the network parameters.

3. The urban air-ground radio map redrawing method based on multimodal information fusion according to claim 1, characterized in that, The occlusion intensity types include line-of-sight links, soft occlusion caused by multipath effects with occlusion type k=1, and severe occlusion caused by concrete structures with occlusion type k=2.

4. The urban air-ground radio map redrawing method based on multimodal information fusion according to claim 1, characterized in that, Step S3 further includes: S31. The first Adam optimizer is used to fix the parameters of the wireless channel parameter prediction network while the virtual obstacle map generation network is fixed, and the parameters of the virtual obstacle map generation network are corrected by calculating the least square difference between the first classification label and the second classification label. S32. The second Adam optimizer is used to optimize the wireless channel parameter prediction network. The first classification label and the second classification label are fixed at the same time. The parameters of the wireless channel parameter prediction network are corrected by calculating the least square difference between the actual value of the loss of the wireless channel signal reception strength and the predicted value of the loss of the wireless channel signal reception strength under different obstruction conditions.

5. The urban air-ground radio map redrawing method based on multimodal information fusion according to claim 4, characterized in that, In step S4, the smaller the minimum squared difference between the first classification label and the second classification label, the smaller the difference between the urban air-ground radio map generated by the virtual obstacle map generation network and the actual 3D urban map. The smaller the minimum squared difference between the actual value of the loss of the wireless channel signal reception strength and the predicted value of the loss of the wireless channel signal reception strength, the closer the wireless channel parameters predicted by the wireless channel parameter prediction network are to the true values.

Images