Low-altitude communication node deployment method and system based on deep learning and reinforcement learning

Abstract

The embodiment of the application provides a low-altitude communication node deployment method and system based on deep learning and reinforcement learning, the method comprising: acquiring a city three-dimensional environment map; wherein the city three-dimensional environment map contains city terrain, building height and building shielding information; predicting, by a pre-determined radio map prediction model, signal-to-interference-and-noise ratio distribution under different node deployments for the city three-dimensional environment map to obtain a signal-to-interference-and-noise ratio prediction distribution map; iteratively optimizing, by a pre-determined reinforcement learning optimization model, the signal-to-interference-and-noise ratio prediction distribution map to determine an optimal deployment strategy; and determining a target base station position based on the optimal deployment strategy; wherein the target base station position is the position of a target node in a plurality of different nodes determined based on the optimal deployment strategy. The above scheme realizes high-precision propagation prediction and adaptive optimization deployment in a complex low-altitude environment, and can improve deployment efficiency and communication performance.

Description

Technical Field

[0001] This application relates to the fields of communication network optimization and artificial intelligence technology, specifically to a method and system for deploying low-altitude communication nodes based on deep learning and reinforcement learning. Background Technology

[0002] With the development of UAV communication networks and air-to-ground converged communication, the demand for wireless communication in low-altitude (0–300 meters) scenarios is rapidly increasing. However, existing communication network optimization mainly focuses on the deployment of ground-based macro base stations, and its optimization methods (such as genetic algorithms, simulated annealing, and convex optimization) largely rely on two-dimensional ground propagation models, making it difficult to accurately describe the channel characteristics and interference distribution in the complex three-dimensional environment of low altitude. Signal propagation in low-altitude environments is affected by multiple factors such as terrain, buildings, and trees. Although traditional electromagnetic simulation methods (such as ray tracing) are accurate, they are computationally intensive and cannot support large-scale, real-time communication deployment optimization.

[0003] Therefore, how to quickly and accurately predict the distribution of communication channels in low-altitude scenarios, and on this basis, realize the intelligent deployment optimization of communication nodes, has become a key issue in current research and engineering applications. Summary of the Invention

[0004] This application aims to provide a method and system for deploying low-altitude communication nodes based on deep learning and reinforcement learning, which can achieve high-precision propagation prediction and adaptive optimization deployment in complex low-altitude environments, thereby improving deployment efficiency and communication performance.

[0005] The technical solution of this application is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a method for deploying low-altitude communication nodes based on deep learning and reinforcement learning, the method comprising:

[0007] Obtain a three-dimensional urban environment map; wherein the three-dimensional urban environment map includes the city's terrain, building height, and building occlusion information;

[0008] Using a pre-defined radio map prediction model, the signal-to-interference-plus-noise ratio (SIR) distribution of the three-dimensional urban environment map under different node deployments is predicted, resulting in a predicted SIR distribution map. The radio map prediction model is a model based on a two-stage cascaded RadioUNet model and optimized input channel parameters, used to generate the predicted SIR distribution map.

[0009] The signal-to-interference-plus-noise ratio (SINR) prediction distribution map is iteratively optimized using a pre-determined reinforcement learning optimization model to determine the optimal deployment strategy; and the target base station location is determined based on the optimal deployment strategy; wherein the target base station location is the location of the target node among multiple different nodes determined based on the optimal deployment strategy.

[0010] In the above scheme, before predicting the signal-to-interference-plus-noise ratio (SIR) distribution of the urban 3D environment map under different node deployments using a pre-determined radio map prediction model to obtain the SIR prediction distribution map, the method further includes:

[0011] Obtain a 3D environmental map of the historical city; and construct a training dataset based on the 3D environmental map of the historical city.

[0012] The initial radio map prediction model is trained using the training dataset to determine the radio map prediction model.

[0013] In the above scheme, constructing a training dataset based on the historical city 3D environment map includes:

[0014] Based on the aforementioned historical city 3D environment map, low-altitude communication propagation performance data is generated.

[0015] The low-altitude communication propagation performance data is processed using a dominant path model to determine the city map and base stations; wherein, the number of base stations is two.

[0016] Based on the city map and the base station, a dual-base station signal-to-interference-plus-noise ratio (SINR) image is generated to determine the training dataset.

[0017] In the above scheme, the step of generating a dual-base station signal-to-interference-plus-noise ratio image based on the city map and the base station, and determining the training dataset, includes:

[0018] Identify the first base station and the second base station among the base stations;

[0019] Based on the city map and the base station, the path loss images of the first base station and the second base station in the low-altitude plane are calculated using the dominant path model.

[0020] Based on the path loss image and the base station's transmit power, a signal-to-interference-plus-noise ratio (SINR) distribution map is calculated; and based on the SINR distribution map, the training dataset is determined.

[0021] In the above scheme, the path loss image includes a first path loss image and a second path loss image; the base station's transmit power includes a first base station transmit power and a second base station transmit power;

[0022] The step involves calculating a signal-to-interference-plus-noise ratio (SINR) distribution map based on the path loss image and the base station's transmit power; and determining the training dataset based on the SINR distribution map, including:

[0023] The first received power is calculated based on the first path loss image and the first base station transmit power;

[0024] The second received power is calculated based on the second path loss image and the second base station transmit power;

[0025] Using the first base station as the main transmitting node and the second base station as the interference node, a first signal-to-interference-plus-noise ratio (SINR) distribution map is calculated based on the first received power and the second received power.

[0026] Using the second base station as the main transmitting node and the first base station as the interference node, a second signal-to-interference-plus-noise ratio (SIR) distribution map is calculated based on the first received power and the second received power.

[0027] Based on the first signal-to-interference-plus-noise ratio (SIR) distribution map and the second SIR distribution map, for the same location, the maximum pixel value between the two is selected as the final pixel value to determine the training dataset.

[0028] In the above scheme, training the initial radio map prediction model using the training dataset to determine the radio map prediction model includes:

[0029] Based on the training dataset, environmental features, transmit power, base station height, and mask matrix are obtained; wherein, the mask matrix represents a binary environmental map.

[0030] The environmental features, the transmission power, the base station height, and the mask matrix are input into the initial radio map prediction model for training, resulting in a signal-to-interference-plus-noise ratio (SINR) prediction sample distribution map.

[0031] Based on the signal-to-interference-plus-noise ratio (SINR) prediction sample distribution map and the pre-acquired real image, calculate the minimum multivariate constraint loss value;

[0032] If the minimized multinomial constraint loss value is less than a preset threshold, then the radio map prediction model is determined; if the minimized multinomial constraint loss value is greater than or equal to the preset threshold, then the initial radio map prediction model is trained based on the training dataset until the minimized multinomial constraint loss value is less than the preset threshold, and then the radio map prediction model is determined.

[0033] In the above scheme, the step of iteratively optimizing the signal-to-interference-plus-noise ratio (SINR) prediction distribution map using a pre-determined reinforcement learning optimization model to determine the optimal deployment strategy includes:

[0034] Based on the predicted signal-to-interference-plus-noise ratio (SINR) distribution map, an updated SINR prediction distribution map is obtained by iterative optimization using a pre-determined reinforcement learning optimization model.

[0035] Based on the updated signal-to-interference-plus-noise ratio (SINR) prediction distribution map, calculate the maximum expected return;

[0036] The optimal deployment strategy is determined based on maximizing the expected return to its maximum value.

[0037] Secondly, embodiments of this application provide a low-altitude communication node deployment system based on deep learning and reinforcement learning. The low-altitude communication node deployment system based on deep learning and reinforcement learning includes: an acquisition module, a prediction module, an optimization module, and a determination module, wherein...

[0038] The acquisition module is used to acquire a three-dimensional urban environment map; wherein, the three-dimensional urban environment map includes the city's terrain, building height, and building occlusion information;

[0039] The prediction module is used to predict the signal-to-interference-plus-noise ratio (SIR) distribution of the three-dimensional urban environment map under different node deployments using a pre-determined radio map prediction model, thereby obtaining a predicted SIR distribution map. The radio map prediction model is characterized by a model based on a two-stage cascade and input channel parameter optimization of the RadioUNet model, used to generate the predicted SIR distribution map.

[0040] The optimization module is used to iteratively optimize the signal-to-interference-plus-noise ratio prediction distribution map using a pre-determined reinforcement learning optimization model to determine the optimal deployment strategy.

[0041] The determining module is used to determine the location of the target base station based on the optimal deployment strategy; wherein, the location of the target base station is the location of the target node among multiple different nodes determined based on the optimal deployment strategy.

[0042] Thirdly, embodiments of this application provide a low-altitude communication node deployment device based on deep learning and reinforcement learning, comprising: a processor and a memory; wherein,

[0043] The memory is used to store computer programs;

[0044] The processor is configured to call and run the computer program from the memory to perform the method as described in the first aspect.

[0045] Fourthly, embodiments of this application provide a computer-readable storage medium storing executable instructions for causing a processor to perform the method described in the first aspect.

[0046] This application provides a method and system for deploying low-altitude communication nodes based on deep learning and reinforcement learning. The method includes: acquiring a three-dimensional urban environment map; wherein the three-dimensional urban environment map includes urban terrain, building height, and building occlusion information; predicting the signal-to-interference-plus-noise ratio (SIR) distribution under different node deployments on the three-dimensional urban environment map using a pre-determined radio map prediction model to obtain an SIR prediction distribution map; wherein the radio map prediction model is a model based on a two-stage cascaded RadioUNet model and input channel parameter optimization used to generate the SIR prediction distribution map; iteratively optimizing the SIR prediction distribution map using a pre-determined reinforcement learning optimization model to determine the optimal deployment strategy; and determining the target base station location based on the optimal deployment strategy; wherein the target base station location is the location of a target node among multiple different nodes determined based on the optimal deployment strategy. In the above scheme, the signal-to-interference-plus-noise ratio (SIR) distribution under different node deployments on the urban 3D environment map is predicted by a radio map prediction model, which can quickly estimate the low-altitude signal propagation performance. A reinforcement learning optimization model is used to search for the optimal deployment strategy based on the obtained SIR prediction distribution map. Based on the optimal deployment strategy, the location of the target base station is determined, that is, the location of the target node among multiple different nodes. Thus, the intelligent layout of the low-altitude communication network can be realized, which can achieve high-precision propagation prediction and adaptive optimization deployment in complex low-altitude environments, thereby improving deployment efficiency and communication performance. Attached Figure Description

[0047] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application. Obviously, the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0048] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0049] Figure 1 An optional flowchart illustrating a low-altitude communication node deployment method based on deep learning and reinforcement learning provided in this application embodiment. Figure 1 ;

[0050] Figure 2 An optional flowchart illustrating a low-altitude communication node deployment method based on deep learning and reinforcement learning provided in this application embodiment. Figure 2 ;

[0051] Figure 3 An optional flowchart illustrating a low-altitude communication node deployment method based on deep learning and reinforcement learning provided in this application embodiment. Figure 3 ;

[0052] Figure 4 A schematic diagram of the structure of a low-altitude communication node deployment system based on deep learning and reinforcement learning provided in an embodiment of this application;

[0053] Figure 5 This is a schematic diagram of a low-altitude communication node deployment device based on deep learning and reinforcement learning, provided as an embodiment of this application. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0055] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used in this application is for the purpose of describing embodiments of this application only and is not intended to be limiting of this application.

[0056] In the following description, references to "some embodiments," "this embodiment," "this application embodiment," and examples, etc., describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subset of all possible embodiments and may be combined with each other without conflict.

[0057] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0058] This application provides a method for deploying low-altitude communication nodes based on deep learning and reinforcement learning. Figure 1 An optional flowchart illustrating a low-altitude communication node deployment method based on deep learning and reinforcement learning provided in this application embodiment. Figure 1 , will combine Figure 1 The steps shown are explained.

[0059] S101. Obtain a three-dimensional urban environment map; wherein, the three-dimensional urban environment map includes the city's terrain, building height, and building occlusion information.

[0060] In some embodiments of this application, a low-altitude communication node deployment method based on deep learning and reinforcement learning is adapted to low-altitude communication node deployment scenarios.

[0061] In some embodiments of this application, a method for deploying low-altitude communication nodes based on deep learning and reinforcement learning is adapted to a low-altitude communication node deployment system based on deep learning and reinforcement learning. In some embodiments of this application, urban 3D environmental maps are obtained using various mapping software.

[0062] S102. Using a pre-determined radio map prediction model, the signal-to-interference-plus-noise ratio (SIR) distribution under different node deployments is predicted on the urban 3D environment map to obtain the SIR prediction distribution map. The radio map prediction model is a model based on the RadioUNet model that performs two-stage cascading and input channel parameter optimization to generate the SIR prediction distribution map.

[0063] In some embodiments of this application, the Radio Wave Neural Network (RadioWNet) is a model used to predict the propagation performance of low-altitude communications. The Radio Wave Neural Network (RadioWNet) is characterized by a two-stage cascade and input channel parameter optimization based on the RadioUNet model, used to generate a signal-to-interference-plus-noise ratio (SINR) prediction distribution map.

[0064] In some embodiments of this application, environmental features are extracted from the urban 3D environment map to obtain the terrain, building height, building occlusion information, transmission power, and base station height of urban buildings. Using a radio map prediction model, the signal-to-interference-plus-noise ratio (SIR) distribution under different node deployments is predicted on the urban 3D environment map to obtain a SIR prediction distribution map. The SIR prediction distribution map includes the initial base station location.

[0065] S103. Using a pre-determined reinforcement learning optimization model, iteratively optimize the signal-to-interference-plus-noise ratio (SINR) prediction distribution map to determine the optimal deployment strategy; and based on the optimal deployment strategy, determine the location of the target base station; wherein, the location of the target base station is the location of the target node among multiple different nodes determined based on the optimal deployment strategy.

[0066] In some embodiments of this application, based on the signal-to-interference-plus-noise ratio (SINR) prediction distribution map, an updated SINR prediction distribution map is obtained through iterative optimization using a pre-determined reinforcement learning optimization model; based on the updated SINR prediction distribution map, the expected return is calculated to maximize the expected return; when the expected return reaches its maximum value, the optimal deployment strategy is determined. Based on the optimal deployment strategy, the location of the target base station is determined.

[0067] For example, based on a deep reinforcement learning framework, using the propagation performance predicted by RadioWNet as environmental feedback, intelligent deployment of low-altitude communication nodes is achieved through a policy optimization algorithm. The specific implementation is as follows... Figure 2 As shown, a historical 3D urban environment map is acquired. Using the DPM data generation module and the RadioWNet training module, base station locations and SINR prediction images are obtained. The PPO reinforcement learning module is then used for optimization to obtain the optimal node target; the optimal node target is the target base station under the optimal deployment strategy.

[0068] The deployment process of low-altitude communication nodes is modeled as a Markov decision process (MDP), defined as follows:

[0069] The state space at each time t:

[0070] in, The SINR distribution predicted by RadioWNet, As an environmental feature, These represent the current transmission power and altitude information, respectively. x represents the horizontal coordinate of the building or predicted point, and y represents the vertical coordinate of the building or predicted point.

[0071] Action space at each time t: This indicates the horizontal and vertical position adjustments of the communication node at the current moment, where, and The amount of movement of the communication location coordinates and This indicates the change in launch altitude.

[0072] State transition function from time t to the next time t+1: ,in, This indicates that the new SINR distribution is calculated using the RadioWnet model.

[0073] Reward function: ,in, For average signal quality, For interference intensity, For coverage metrics, , and These are the weighting coefficients.

[0074] The PPO algorithm employs a two-part structure, including a policy network that outputs the probability distribution of node movement. And the value network expected to be reported in assessing the current state. Both networks are multilayer perceptrons or lightweight CNN structures, sharing some parameters to enhance feature consistency.

[0075] The PPO strategy aims to optimize a surrogate function that limits the update magnitude.

[0076]

[0077] in:

[0078]

[0079] It is a shearing function used to limit the magnitude of policy changes; E t For the expectation of time step t, For the dominant function, The strategy at the previous moment, R t This refers to the reporting, specifically the cumulative reward starting from time step t. Refers to the value state function; It is a proxy function; PPO refers to the PPO strategy mentioned in this application; min means taking the minimum.

[0080] The cutoff coefficients are updated for the policy to prevent excessive policy fluctuations. The value function is then trained by minimizing the squared error.

[0081]

[0082] The total loss is: β is the weighting coefficient; It is a value function;

[0083] In each iteration, the agent samples several deployment action sequences. RadioWNet quickly predicts the SINR, calculates the reward, and updates the policy network parameters. The optimization objective is to maximize the expected return.

[0084]

[0085] in, Using the discount factor, the optimal deployment strategy is obtained after the reinforcement learning optimization model training converges. argmax refers to finding a value that maximizes the objective function, which is the expected return mentioned earlier; the final output is the optimal communication node deployment scheme. ;in, Indicates the optimal base station location; This represents the optimal coverage.

[0086] Understandably, this application uses a radio map prediction model to predict the signal-to-interference-plus-noise ratio (SIR) distribution under different node deployments on a three-dimensional urban environment map, enabling rapid estimation of low-altitude signal propagation performance. It employs a reinforcement learning optimization model to search for the optimal deployment strategy based on the obtained SIR prediction distribution map. Based on the optimal deployment strategy, it determines the location of the target base station, i.e., the location of the target node among multiple different nodes. This achieves intelligent layout of the low-altitude communication network, enabling high-precision propagation prediction and adaptive optimization deployment in complex low-altitude environments, thereby improving deployment efficiency and communication performance.

[0087] In some embodiments of this application, S104 and S105 are included before S102, as follows:

[0088] S104. Obtain a 3D environment map of the historical city; and construct a training dataset based on the 3D environment map of the historical city.

[0089] In some embodiments of this application, low-altitude communication propagation performance data is generated based on historical three-dimensional urban environment maps; the low-altitude communication propagation performance data is processed using a dominant path model to determine the city map and base stations; wherein, the number of base stations is two; based on the city map and base stations, dual-base station signal-to-interference-plus-noise ratio (SINR) images are generated to determine the training dataset.

[0090] In some embodiments of this application, a first base station and a second base station are identified; based on a city map and the base stations, path loss images of the first base station and the second base station on the low-altitude plane are calculated using a dominant path model; based on the path loss images and the transmit power of the base stations, a signal-to-interference-plus-noise ratio (SINR) distribution map is calculated; and based on the SINR distribution map, a training dataset is determined.

[0091] In some embodiments of this application, the path loss image includes a first path loss image and a second path loss image; the base station's transmit power includes a first base station transmit power and a second base station transmit power;

[0092] In some embodiments of this application, a first received power is calculated based on a first path loss image and the transmit power of a first base station; a second received power is calculated based on a second path loss image and the transmit power of a second base station; a first signal-to-interference-plus-noise ratio (SIR) distribution map is calculated based on the first received power and the second received power, with the first base station as the main transmitting node and the second base station as the interfering node; a second SIR distribution map is calculated based on the second base station as the main transmitting node and the first base station as the interfering node, with the first received power and the second received power; based on the first SIR distribution map and the second SIR distribution map, for the same location, the maximum pixel value between the two is selected as the final pixel value to determine the training dataset.

[0093] In some embodiments of this application, the first base station can be used as the main transmitting node and the second base station as the interference node. Based on the first path loss image, the second path loss image, the transmission power of the first base station, and the transmission power of the second base station, the third received power of the main transmitting node and the fourth received power of the interference node are calculated. Based on the third received power and the fourth received power, a first signal-to-interference-plus-noise ratio (SIR) distribution map is calculated. Using the second base station as the main transmitting node and the first base station as the interference node, based on the first path loss image, the second path loss image, the transmission power of the first base station, and the transmission power of the second base station, a fifth received power of the main transmitting node and a sixth received power of the interference node are calculated. Based on the fifth received power and the sixth received power, a second SIR distribution map is calculated. Based on the first SIR distribution map and the second SIR distribution map, for the same location, the maximum pixel value between the two is selected as the final pixel value to determine the training dataset.

[0094] For example, low-altitude communication propagation performance data is generated in a real urban environment to provide high-fidelity training samples for subsequent deep learning models. Based on the Dominant Path Model (DPM), the dominant propagation path loss (PL) between the transmitting base station and the receiving point is calculated in a 3D urban environment, and a signal-to-interference-plus-noise ratio (SINR) image is further generated. Figure 3 As shown, input the urban environment model, DPM calculates the path loss, generates a loss map of signal source and interference source, and calculates... Image fusion output low altitude sample.

[0095] Dominant Path Model (DPM) Modeling Principle: The dominant path model is a physical propagation model that considers the characteristic that the main energy of a signal propagates along the shortest or diffracted path in a complex urban environment. For a transmitter T and a receiver R, the dominant path length... It can be represented as:

[0096]

[0097] in, N represents the set of all possible paths; k This represents the number of turnaround points contained in the k-th path; This represents the coordinates of the i-th point on the path; This represents the coordinates of the (i-1)th point on the path.

[0098] Based on the dominant path length, DPM calculates the path loss as follows:

[0099]

[0100] Where f represents the carrier frequency; Indicates the path loss index; Indicates the additional loss along the main reflection path; Indicates the additional loss along the diffraction path; Represents system constants; Let T represent the logarithmic function; T represent the transmitting point; and R represent the receiving point.

[0101] Dual-base station signal-to-interference-plus-noise ratio image generation:

[0102] In the urban environment E(x,y,z), two sets of base station locations are selected: Indicates the signal source (main transmitting node); This indicates the source of interference (interference node).

[0103] The path loss distribution of the two base stations in the low-altitude plane was calculated using the DPM model:

[0104]

[0105] in, This indicates the generation of a two-dimensional path loss image using a dominant path model. A two-dimensional path loss image representing the signal source; A two-dimensional path loss image representing the interference source; for each two-dimensional path loss image, its pixel value corresponds to the loss level at the receiver point (x, y), and then based on the transmit power... and It can calculate the received power of the signal source and interference source at each point:

[0106]

[0107]

[0108] Given system noise Below, the signal-to-interference-plus-noise ratio (SIR) for each pixel is:

[0109]

[0110] Then convert to dB form: .

[0111] To improve the richness and robustness of the low-altitude dataset, this application introduces dual-scenario fusion logic to calculate the SINR distribution maps under two scenarios (switching between the signal sources and interference sources of two sets of base stations), namely... and Then, the maximum pixel value at the same position is taken to obtain the final SINR distribution map. :

[0112]

[0113] That is, at each spatial point, a value with a high signal-to-interference-plus-noise ratio is selected to simulate the dynamic optimal channel selection behavior of a low-altitude communication system under multi-link conditions, and the final output is used to construct a low-altitude SINR sample set.

[0114]

[0115] Where k represents the k-th pixel, The low-altitude SINR sample set is used. S105. The initial radio map prediction model is trained using the training dataset to determine the radio map prediction model.

[0116] In some embodiments of this application, environmental features, transmit power, base station height, and a mask matrix are obtained based on a training dataset; wherein the mask matrix represents a binary environmental map; the environmental features, transmit power, base station height, and mask matrix are input into an initial radio map prediction model for training to obtain a signal-to-interference-plus-noise ratio (SINR) prediction sample distribution map; based on the SINR prediction sample distribution map and pre-acquired real images, a minimum multinomial constraint loss value is calculated; if the minimum multinomial constraint loss value is less than a preset threshold, the radio map prediction model is determined; if the minimum multinomial constraint loss value is greater than or equal to the preset threshold, the initial radio map prediction model is trained again based on the training dataset until the minimum multinomial constraint loss value is less than the preset threshold, and the radio map prediction model is determined.

[0117] It should be noted that each image in the training dataset is binarized to obtain a binary environment map, i.e., a mask matrix.

[0118] For example, based on the obtained high-fidelity SINR data, a deep neural network model, RadioWNet (Radio Wave Neural Network), is trained to quickly and accurately predict the propagation performance of low-altitude communication. This model achieves high-precision prediction of SINR distribution under different node deployments by learning the nonlinear mapping relationship between complex terrain, building obstruction, and multipath propagation, replacing the traditional high-complexity simulation process.

[0119] The process of training a model:

[0120] The input is a multi-channel feature tensor:

[0121]

[0122] in, Indicates environmental features (topography, building obstruction, height, etc.). For transmission power, For base station height, The mask matrix, i.e., the binary environmental map, outputs the corresponding predicted SINR distribution:

[0123]

[0124] in, This represents an output relationship; RadioWNet uses an encoder-decoder structure combined with residual links to maintain the continuity of spatial features and the stability of propagation gradients, and is represented as follows:

[0125]

[0126] Where E is the encoder and D is the decoder.

[0127] During model training, the objective loss function is minimized using a multinomial constraint loss function, as follows:

[0128]

[0129] Where H and W represent the height and width of the fused image; SSIM refers to the structural similarity loss of the image. Y is the model's predicted output image, ▽ represents the gradient; the first term is the mean squared error; the second term is the structural similarity constraint; the third term ensures the spatial smoothness of the predicted image. , These are the weighting coefficients.

[0130] Parameters are updated via the Adam optimizer:

[0131]

[0132] Here, argmin refers to finding a value that minimizes the objective function; This is the optimal parameter combination.

[0133] Based on the above embodiments of a low-altitude communication node deployment method based on deep learning and reinforcement learning, this application also provides a low-altitude communication node deployment system based on deep learning and reinforcement learning, such as... Figure 4 As shown, Figure 4 This application provides a schematic diagram of the structure of a low-altitude communication node deployment system based on deep learning and reinforcement learning. The system includes: an acquisition module 401, a prediction module 402, an optimization module 403, and a determination module 404.

[0134] The acquisition module 401 is used to acquire a three-dimensional urban environment map; wherein, the three-dimensional urban environment map includes the city's terrain, building height, and building occlusion information;

[0135] The prediction module 402 is used to predict the signal-to-interference-plus-noise ratio (SIR) distribution of the three-dimensional urban environment map under different node deployments using a pre-determined radio map prediction model, thereby obtaining a predicted SIR distribution map. The radio map prediction model is characterized by a model based on a two-stage cascade and input channel parameter optimization of the RadioUNet model, used to generate the predicted SIR distribution map.

[0136] The optimization module 403 is used to iteratively optimize the signal-to-interference-plus-noise ratio prediction distribution map through a pre-determined reinforcement learning optimization model to determine the optimal deployment strategy;

[0137] The determining module 404 is used to determine the location of the target base station based on the optimal deployment strategy; wherein, the location of the target base station is the location of the target node among multiple different nodes determined based on the optimal deployment strategy.

[0138] In some embodiments of this application, the acquisition module 401 is further configured to acquire a historical urban 3D environment map before predicting the signal-to-interference-plus-noise ratio (SIR) distribution of the urban 3D environment map under different node deployments using a pre-determined radio map prediction model, and constructing a training dataset based on the historical urban 3D environment map.

[0139] The determining module 404 is further configured to train the initial radio map prediction model using the training dataset, and determine the radio map prediction model.

[0140] In some embodiments of this application, the determining module 404 is further configured to generate low-altitude communication propagation performance data based on the historical city 3D environment map; process the low-altitude communication propagation performance data using a dominant path model to determine the city map and base stations; wherein the number of base stations is two; and generate a dual-base station signal-to-interference-plus-noise ratio image based on the city map and the base stations to determine the training dataset.

[0141] In some embodiments of this application, the determining module 404 is further configured to determine the first base station and the second base station among the base stations; calculate the path loss images of the first base station and the second base station in the low-altitude plane based on the city map and the base stations using the dominant path model; calculate the signal-to-interference-plus-noise ratio (SINR) distribution map based on the path loss images and the transmit power of the base stations; and determine the training dataset based on the SINR distribution map.

[0142] In some embodiments of this application, the path loss image includes a first path loss image and a second path loss image; the transmission power of the base station includes a first base station transmission power and a second base station transmission power;

[0143] The determining module 404 is further configured to: calculate a first received power based on the first path loss image and the first base station transmit power; calculate a second received power based on the second path loss image and the second base station transmit power; use the first base station as the main transmitting node and the second base station as the interfering node, and calculate a first signal-to-interference-plus-noise ratio (SIR) distribution map based on the first received power and the second received power; use the second base station as the main transmitting node and the first base station as the interfering node, and calculate a second SIR distribution map based on the first received power and the second received power; and, based on the first SIR distribution map and the second SIR distribution map, select the maximum pixel value of the two for the same location as the final pixel value to determine the training dataset.

[0144] In some embodiments of this application, the determining module 404 is further configured to acquire environmental features, transmit power, base station height, and a mask matrix based on the training dataset; wherein the mask matrix represents a binary environmental map; input the environmental features, transmit power, base station height, and mask matrix into the initial radio map prediction model for training to obtain a signal-to-interference-plus-noise ratio (SINR) prediction sample distribution map; calculate the minimum multinomial constraint loss value based on the SINR prediction sample distribution map and pre-acquired real images; if the minimum multinomial constraint loss value is less than a preset threshold, then the radio map prediction model is determined; if the minimum multinomial constraint loss value is greater than or equal to the preset threshold, then the initial radio map prediction model is trained again based on the training dataset until the minimum multinomial constraint loss value is less than the preset threshold, and then the radio map prediction model is determined.

[0145] In some embodiments of this application, the determining module 404 is further configured to perform iterative optimization based on the signal-to-interference-plus-noise ratio (SINR) prediction distribution map using a pre-determined reinforcement learning optimization model to obtain an updated SINR prediction distribution map; calculate the maximum expected return based on the updated SINR prediction distribution map; and determine the optimal deployment strategy when the maximum expected return reaches its maximum value.

[0146] Based on the above embodiments of a low-altitude communication node deployment method based on deep learning and reinforcement learning, this application also provides a low-altitude communication node deployment device based on deep learning and reinforcement learning, such as... Figure 5 As shown, Figure 5This is a schematic diagram of a low-altitude communication node deployment device based on deep learning and reinforcement learning, provided in an embodiment of this application. The low-altitude communication node deployment device 5 based on deep learning and reinforcement learning includes a processor 501 and a memory 502. The memory 502 is used to store computer programs; the processor 501 is used to call and run the computer programs from the memory to execute a low-altitude communication node deployment method based on deep learning and reinforcement learning as described in the above embodiment.

[0147] In the embodiments of this application, the processor 501 described above can be at least one of the following: Application-Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that for different devices, the electronic device used to implement the above processor function can also be other types, and the embodiments of this application do not specifically limit it.

[0148] This application provides a computer-readable storage medium storing a computer program for implementing, when executed by a processor, a low-altitude communication node deployment method based on deep learning and reinforcement learning as described in any of the above embodiments.

[0149] For example, the program instructions corresponding to a low-altitude communication node deployment method based on deep learning and reinforcement learning in this embodiment can be stored on storage media such as optical discs, hard disks, and USB flash drives. When the program instructions corresponding to the low-altitude communication node deployment method based on deep learning and reinforcement learning in the storage media are read or executed by an electronic device, a low-altitude communication node deployment method based on deep learning and reinforcement learning as described in any of the above embodiments can be implemented.

[0150] Furthermore, the functional modules in the embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional module.

[0151] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the embodiments in this application are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, these will not be repeated here.

[0153] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0154] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0155] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0156] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0157] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0158] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments. The above descriptions are merely implementation methods of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the protection scope of the embodiments of this application. Therefore, the protection scope of the embodiments of this application should be determined by the protection scope of the claims.

Claims

1. A method for low-altitude communication node deployment based on deep learning and reinforcement learning, characterized in that, The method includes: Obtain a three-dimensional urban environment map; wherein the three-dimensional urban environment map includes the city's terrain, building height, and building occlusion information; Using a pre-defined radio map prediction model, the signal-to-interference-plus-noise ratio (SIR) distribution of the three-dimensional urban environment map under different node deployments is predicted, resulting in a predicted SIR distribution map. The radio map prediction model is a model based on a two-stage cascaded RadioUNet model and optimized input channel parameters, used to generate the predicted SIR distribution map. The signal-to-interference-plus-noise ratio (SINR) prediction distribution map is iteratively optimized using a pre-determined reinforcement learning optimization model to determine the optimal deployment strategy; and the target base station location is determined based on the optimal deployment strategy; wherein, the target base station location is the location of the target node among multiple different nodes determined based on the optimal deployment strategy. Before predicting the signal-to-interference-plus-noise ratio (SIR) distribution of the urban 3D environment map under different node deployments using a pre-determined radio map prediction model to obtain the SIR prediction distribution map, the method further includes: Acquire a three-dimensional environmental map of a historical city; and generate low-altitude communication propagation performance data based on the three-dimensional environmental map of the historical city. The low-altitude communication propagation performance data is processed using a dominant path model to determine the city map and base stations; wherein, the number of base stations is two. Based on the city map and the base station, generate dual-base station signal-to-interference-plus-noise ratio (SINR) images to determine the training dataset; The initial radio map prediction model is trained using the training dataset to determine the radio map prediction model.

2. The method of claim 1, wherein, The step of generating a dual-base station signal-to-interference-plus-noise ratio (SINR) image based on the city map and the base stations, and determining the training dataset, includes: Identify the first base station and the second base station among the base stations; Based on the city map and the base station, the path loss images of the first base station and the second base station in the low-altitude plane are calculated using the dominant path model. Based on the path loss image and the base station's transmit power, a signal-to-interference-plus-noise ratio (SINR) distribution map is calculated; and based on the SINR distribution map, the training dataset is determined.

3. The method of claim 2, wherein, The path loss image includes a first path loss image and a second path loss image; the base station's transmit power includes a first base station transmit power and a second base station transmit power; The signal-to-interference-plus-noise ratio (SINR) distribution map is calculated based on the path loss image and the base station's transmit power. Based on the signal-to-interference-plus-noise ratio (SINR) distribution map, the training dataset is determined, including: The first received power is calculated based on the first path loss image and the first base station transmit power; The second received power is calculated based on the second path loss image and the second base station transmit power; Using the first base station as the main transmitting node and the second base station as the interference node, a first signal-to-interference-plus-noise ratio (SINR) distribution map is calculated based on the first received power and the second received power. Using the second base station as the main transmitting node and the first base station as the interference node, a second signal-to-interference-plus-noise ratio (SIR) distribution map is calculated based on the first received power and the second received power. Based on the first signal-to-interference-plus-noise ratio (SIR) distribution map and the second SIR distribution map, for the same location, the maximum pixel value between the two is selected as the final pixel value to determine the training dataset.

4. The method of claim 1, wherein, The step of training the initial radio map prediction model using the training dataset to determine the radio map prediction model includes: Based on the training dataset, environmental features, transmit power, base station height, and mask matrix are obtained; wherein, the mask matrix represents a binary environmental map. The environmental features, the transmission power, the base station height, and the mask matrix are input into the initial radio map prediction model for training, resulting in a signal-to-interference-plus-noise ratio (SINR) prediction sample distribution map. Based on the signal-to-interference-plus-noise ratio (SINR) prediction sample distribution map and the pre-acquired real image, calculate the minimum multivariate constraint loss value; If the minimized multivariate constraint loss value is less than a preset threshold, then the radio map prediction model is determined; If the minimized multinomial constraint loss value is greater than or equal to a preset threshold, then the initial radio map prediction model is trained based on the training dataset until the minimized multinomial constraint loss value is less than the preset threshold, and the radio map prediction model is determined.

5. The method of claim 1, wherein, The step of iteratively optimizing the signal-to-interference-plus-noise ratio (SINR) prediction distribution map using a pre-determined reinforcement learning optimization model to determine the optimal deployment strategy includes: Based on the predicted signal-to-interference-plus-noise ratio (SINR) distribution map, an updated SINR prediction distribution map is obtained by iterative optimization using a pre-determined reinforcement learning optimization model. Based on the updated signal-to-interference-plus-noise ratio (SINR) prediction distribution map, calculate the maximum expected return; The optimal deployment strategy is determined based on maximizing the expected return to its maximum value. 6.A low-altitude communication node deployment system based on deep learning and reinforcement learning, characterized in that, The low-altitude communication node deployment system based on deep learning and reinforcement learning includes: an acquisition module, a prediction module, an optimization module, and a determination module, wherein, The acquisition module is used to acquire a three-dimensional urban environment map; wherein, the three-dimensional urban environment map includes the city's terrain, building height, and building occlusion information; The prediction module is used to predict the signal-to-interference-plus-noise ratio (SIR) distribution of the three-dimensional urban environment map under different node deployments using a pre-determined radio map prediction model, thereby obtaining a predicted SIR distribution map. The radio map prediction model is characterized by a model based on a two-stage cascade and input channel parameter optimization of the RadioUNet model, used to generate the predicted SIR distribution map. The optimization module is used to iteratively optimize the signal-to-interference-plus-noise ratio prediction distribution map using a pre-determined reinforcement learning optimization model to determine the optimal deployment strategy. The determining module is used to determine the location of the target base station based on the optimal deployment strategy; wherein, the location of the target base station is the location of the target node among multiple different nodes determined based on the optimal deployment strategy; The acquisition module is further configured to acquire historical urban 3D environment maps before predicting the signal-to-interference-plus-noise ratio (SIR) distribution of the urban 3D environment map under different node deployments using a pre-determined radio map prediction model, and to generate low-altitude communication propagation performance data based on the historical urban 3D environment maps. The determining module is further configured to process the low-altitude communication propagation performance data using a dominant path model to determine the city map and base stations; wherein the number of base stations is two; based on the city map and the base stations, a dual-base station signal-to-interference-plus-noise ratio image is generated to determine a training dataset; and the initial radio map prediction model is trained using the training dataset to determine the radio map prediction model.

7. A low-altitude communication node deployment device based on deep learning and reinforcement learning, characterized in that, include: Processor and memory, of which, The memory is used to store computer programs; The processor is configured to call and run the computer program from the memory to perform the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It stores executable instructions for causing a processor to execute, thereby implementing the method of any one of claims 1 to 5.

Images