Perception-enhanced channel multi-modal predictive modeling method
By using the dual-base perceived channel data of perceived data as a timing feature mode, combining path loss and environmental information, multi-layer gated cyclic units and deep residual networks for feature fusion, the communication channel construction of the area where perceived data cannot be covered is solved The modular problem realizes high-precision channel prediction in complex environments.
Patent Information
- Application Number
- CN202411501926.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-25
- Publication Date
- 2025-01-03
AI Technical Summary
In the sixth generation mobile communication system, perceived data cannot achieve full coverage echo detection of the target area, resulting in hard demand for communication links in the blind spots of the perceived signal. The prior art is difficult to predictively model based on limited perceived data. The communication channel characteristics of the region, and the prediction accuracy and efficiency are low.
Adopting perceptually enhanced multi-modal precognitive modeling method, the dual-base perceptual channel data with existing perceptual data is used as the timing feature mode, combining path loss, root mean square delay expansion and environmental information, through multi-layer gate control The loop unit and the pretrained deep residual network perform feature fusion to predict timing communication data of the area not covered by the perceptual data.
Improve the prediction ability of non-stationary small-scale channel characteristics in complex environments, improve prediction accuracy and convergence speed, and learn small-scale timing relationships in longer-distance non-stationary channels to satisfy the communication channel that is not covered by the perceived data. Modeling requirements.
Smart Images
Figure CN119254361A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a perception-enhanced channel multimodal predictive modeling method, belonging to the technical field of wireless communications. Background Art
[0002] In the sixth generation mobile communication system, the communication and perception integration (ISAC) technology will use wireless signals to realize the detection, positioning, imaging, identification and other perception functions of the target, and obtain the surrounding physical environment information, so as to realize the hardware and software integration and symbiotic performance improvement of communication and perception. In the ISAC synaesthesia system, the measurement and modeling of communication and perception channels are the basis of ISAC system design, deployment and performance optimization. Perception data is generally used as prior knowledge to assist in the modeling of communication channels. However, in the actual ISAC system, the perception data cannot achieve full coverage echo detection of the target area, but there is still a rigid demand for communication links in the blind area of the perception signal. How to predictively model the communication channel characteristics of the unknown area based on the limited perception data of the known area, and to assist the accuracy and efficiency of the prediction based on the information of the actual synaesthesia environment, is a key technical problem in the ISAC system.
[0003] In the prior art, the use of perception data in the ISAC channel focuses on taking the perception cluster at the same time as known prior knowledge, recording the location information of the perception cluster through algorithms such as inversion positioning and tracking, parsing the delay and angle of the perception path, and then determining the location and shared cluster information of a part of the shared scatterers based on the set sharing degree parameters, and generating a part of the communication channel parameters accordingly, completing the assistance of perception data for communication channel modeling. However, such methods can only work at the same time and under the premise that the transceiver distance is relatively close, and the selection method of the sharing rate parameters and shared clusters is generally specified manually. Therefore, the communication channel modeling assisted by perception data has low accuracy and cannot meet the perception enhancement communication channel modeling in areas not covered by perception data. In addition, for the existing channel modeling technologies assisted by environmental information, they all directly input link features, image features, etc. into the deep learning network to predict a limited number of large-scale channel parameters, such as path loss, received power, and delay spread, which do not contain timing information and cannot be directly applied to the complete large-scale and small-scale channel modeling of unknown locations. Although existing channel prediction technologies contain timing information, the prediction range is limited to a quasi-coherent distance of several wavelengths. However, for areas not covered by perception data, the distance from the areas with existing perception data is much greater than the channel coherence distance. Within this range, the channel has experienced multipath delay power changes, and even multipath birth and death effects caused by changes in the propagation environment. Therefore, existing channel prediction technologies are difficult to apply to this problem. Summary of the invention
[0004] The problem to be solved by the technology of the present invention is: to overcome the shortcomings of the prior art, and to propose a perception-enhanced channel multimodal predictive modeling method, which uses the dual-base perception channel data of the existing perception data location as the time series feature mode to enhance the prediction performance of the communication channel data. Due to the non-stationarity of the time series delay power spectrum (PDP) sequence of multi-point long distance and complex electromagnetic propagation environment, the large-scale parameter features of the known perception data, including path loss, root mean square delay spread, and the link topology features of the perception receiving point location, including relative horizontal coordinates, relative vertical coordinates, relative three-dimensional distance, and pitch angle are input into the link feature extraction unit, the vector map image representing the environmental information is input into the visual feature extraction unit based on the pre-trained deep residual network after preprocessing, and the time series perception data is input into the multi-layer gated recurrent unit to extract the time series features. The above-mentioned multimodal channel and environmental features are feature fused and then connected to the multi-layer perceptron to predict the time series communication data of the area not covered by the perception data. For the values of the number of sampling points of perception data and communication data, the steady distance is used to measure the area where the channel does not change significantly. The number of known perception data points and the number of communication data points to be predicted are selected based on the steady distance.
[0005] The technical solution of the present invention is:
[0006] A method for perceptually enhanced channel multimodal predictive modeling, the method comprising the steps of:
[0007] Step 1: Calculate the stationary distance of the target area, and determine the number of time domain sampling points M of the perception timing sequence and the number of time domain sampling points N of the communication timing sequence according to the calculated stationary distance;
[0008] Step 2: discretize the delay power spectrum data according to the number of time domain sampling points M and the number of time domain sampling points N determined in step 1 to obtain discrete delay values, and then normalize the noise floor of the delay power spectrum data to obtain the time series modal characteristics of the perceived time series sequence;
[0009] Step 3: construct link modal features based on the statistical features and location topology features of the perceived time series;
[0010] Step 4: Based on the environment vector map, construct standardized and size-normalized visual modality features;
[0011] Step five, based on the temporal modal features obtained in step two, the link modal features constructed in step three, and the visual modal features constructed in step four, a multimodal network based on gated recurrent units, multilayer perceptrons, and pre-trained deep residual networks is constructed to obtain multimodal predictive modeling results.
[0012] In step 1, the stable distance of the target area at any time t1 is:
[0013]
[0014] Where Δd is the distance between adjacent sampling points, d qs (t1) is the stable distance at time t1; t2 is a time different from t1; c th is the threshold of correlation coefficient;
[0015]
[0016] Υ(t1;τ)=|R(t1;τ)| 2
[0017] γ(t2;τ)=|R(t2;τ)| 2
[0018] Among them, Υ(t1; τ) is the delay power spectrum data in units of modulus square at time t1, γ(t2; τ) is the delay power spectrum data in units of modulus square at time t2, τ is the delay dimension, R(t1; τ) is the channel impulse response at time t1, and R(t2; τ) is the channel impulse response at time t2.
[0019] In the step 1, the method for determining the number of time domain sampling points M of the perception timing sequence and the number of time domain sampling points N of the communication timing sequence according to the obtained stationary distance is:
[0020] The sensing mode is dual-base sensing. The ISAC system model is that the transmitter is fixed and the receiver moves along a certain route. The first M moments are used as dual-base sensing data, i.e., the number of time domain sampling points M of the sensing timing sequence, and the last N moments are used as communication data, i.e., the number of time domain sampling points N of the communication timing sequence, satisfying:
[0021]
[0022] in, is the average value of the stationary distance in the target area.
[0023] In step 2, the discrete delay value is:
[0024]
[0025] Where Δτ is the delay resolution, f s is the sampling rate, τ con is the continuous delay value, τ dis is the discrete delay value, is the rounding operator;
[0026] In the step 2, the method for normalizing the noise floor of the time-delay power spectrum data is:
[0027] Convert the delayed power spectrum data to logarithmic units:
[0028] Ω(t; τ) = 10·log 10 (Υ(t;τ))
[0029] Among them, γ(t; τ) is the delay power spectrum data in the unit of modulus square at time t, and Ω(t; τ) is the delay power spectrum data in the unit of dBm. All multipath powers are normalized according to the noise floor. If the noise floor is -p (dBm), all multipath powers in Ω(t; τ) are added with p, and then denormalized after the network outputs the result.
[0030] In step 2, the temporal modal features of the perceived temporal sequence are obtained as follows:
[0031] F t =Ω(t;τ)(t=1,2,...,M)+p
[0032] In the above formula, the first M moments in Ω(t; τ) are used as the input time steps of the network, the last N moments are used as the output time steps of the network, and the delay power spectrum data of the delay dimension is used as the feature dimension of each time step.
[0033] In step 3, the link modal features constructed are:
[0034]
[0035] The statistical features of the perception data include the mean value of the path loss and the mean value of the root mean square delay spread of the perception data in the first M time steps. The calculation formula is:
[0036]
[0037] in, is the transmission power of the mth sensing point, is the received power of the mth sensing point, and the calculation formula for the mean value of the RMS delay spread is:
[0038]
[0039] Among them, a m,l is the power of the lth multipath at the mth sensing point, L is the number of multipaths, τ m,l is the delay of the lth multipath at the mth sensing point;
[0040] The position topological features include the relative horizontal coordinate mean, relative vertical coordinate mean, relative three-dimensional distance mean, pitch angle mean, relative horizontal coordinate mean of the perception data of the first M time steps. The calculation method is:
[0041]
[0042] in, is the horizontal coordinate of the mth sensing point, x T is the horizontal coordinate of the transmitter, relative to the vertical coordinate
[0043] value The calculation method is:
[0044]
[0045] in, is the ordinate of the mth sensing point, y T is the vertical coordinate of the transmitter. Relative three-dimensional distance mean The calculation method is as follows, where z R is the height of the receiver, z T is the height of the transmitter;
[0046]
[0047] The calculation method of the mean pitch angle is:
[0048]
[0049] In step 4, the constructed standardized and size-normalized visual modality features are:
[0050] F map ={I1,I2,...,I N}
[0051] Among them, I n (n=1,2,…,N) is the nth picture, which is the nth channel of the multimodal network input, consisting of J*J pixels. constitute:
[0052]
[0053] Among them, R is the number of data, p is the original value of any pixel in the original image, and the construction method of the original image is:
[0054] The environment vector map of the target scene is a .shp structure file. A building is a polygon composed of multiple boundary lines or line segments when viewed from a bird's-eye view, and a prism when viewed in three dimensions. The .shp structure file stores the coordinate information of the height and boundary lines of each building.
[0055] When constructing a grayscale top view of a vector map representing a detailed propagation environment between a transmitter and a communication receiver, a grayscale top view of the entire scene is first drawn. The mapping relationship between grayscale values and building heights is:
[0056]
[0057] Where, d is the building height, d min is the minimum height of the building in the current scene, d max is the maximum height of the building in the current scene, g is the grayscale value converted from the current building height, ranging from 0 to 255;
[0058] After drawing the grayscale bird's-eye view map of the entire area, the rectangular area around the transmitter and receiver is intercepted from the vector map, which is the extension of the transceiver connection line to the surrounding area. e The area is the effective propagation area of the perception channel. Based on this, the image is cropped and rotated to the horizontal-vertical direction. The image rotation angle of the nth communication data point is as follows, where is the horizontal coordinate of the nth communication receiving position, is the ordinate of the nth communication receiving position, n = 1, 2, ..., N;
[0059]
[0060] On this basis, the size of all images is reshaped to J*J, and the grayscale values of the images are normalized from 0-255 to the range of 0-1.
[0061] In the step 5, the method for constructing a multimodal network based on a gated recurrent unit, a multilayer perceptron and a pre-trained deep residual network is:
[0062] Based on the extracted temporal modal features, visual modal features and link modal features, a perception-enhanced channel multimodal predictive modeling network is designed. The components of each module are as follows:
[0063] The temporal modal features are extracted by multi-layer gated recurrent units, and the delayed power spectrum data with M time steps is input, and the last layer of network state at the last moment is taken as the output of the gated recurrent network;
[0064] The N-channel visual modality features are first extracted through a 1*1 convolutional layer and transformed into three channels. Then, the image features are extracted by a deep residual network with ImageNet pre-trained weights.
[0065] The statistical features of the perception data and the location topology features are concatenated after passing through the fully connected layer to serve as link modal features.
[0066] After the time series mode, visual mode, and link mode are respectively passed through the feature extraction module based on the multi-layer perceptron, feature splicing is performed, and then the multi-layer perceptron is passed through and the embedding vector dimension is reshaped to N*delay points. The root mean square error is used as the loss function:
[0067]
[0068] Among them, Loss is the value of the loss function, B represents the batch size, b is the index of B, F is the length of the delay power spectrum, that is, the feature length, and f is the index of F. represents the predicted delay power spectrum of the bth data, the fth feature, and the nth time step in this batch, Represents the true delayed power spectrum of the b-th data, the f-th feature, and the n-th time step in this batch, b = 1, 2, ..., B, f = 1, 2, ..., F.
[0069] The structure of the deep residual network is as follows: first, the original image is projected onto C channels through a 7×7 convolutional layer with a step size of 2, and a pooling operation with a step size of 2 is performed on it. Then, the processed features are input into four residual modules. The four residual modules are represented by S1, S2, S3, and S4 respectively. Each residual module contains two paths, where the first path is a direct path of the input feature, and the second path performs two convolution operations on the feature to obtain the residual of the feature. Finally, the features on the two paths are added together. The stacking order of the network layers is: convolutional layer, batch normalization layer, rectified linear unit activation layer, convolutional layer, BN layer, residual addition, ReLU activation layer. In addition, all residual modules have no pooling layer, and downsampling is only achieved by setting the step size during convolution.
[0070] In step 5, the multimodal predictive modeling result obtained is:
[0071]
[0072] in, is the multi-layer perceptron function, is the deep residual network function, is the gated recurrent unit function, and ⊙ is the feature concatenation operator.
[0073] Beneficial Effects
[0074] The present invention discloses a method for predicting non-stationary small-scale channel characteristics in a complex environment by using a perception-enhanced channel multimodal predictive modeling method with the ability to predict non-stationary small-scale channel characteristics in a complex environment. The dual-base perception channel data of the existing perception data position is used as a time series feature mode to predict the communication small-scale data of adjacent positions with a time series relationship. A method for using link modal features including perception data statistical features and position topology features and visual modal features based on a grayscale vector map as auxiliary and perception time series modal to form a multimodal network, wherein the perception data statistical features include path loss and root mean square delay spread, and the position topology features include relative horizontal coordinates, relative vertical coordinates, relative three-dimensional distance, and pitch angle. A gated recurrent unit, a pre-trained deep residual network, and a fully connected network are designed to extract the features of each mode respectively and give the prediction results of the multi-time step delay power spectrum. A method for selecting perception and communication points based on a stationary distance metric calculates the stationary distance mean of the target area based on the delay power spectrum metric, ensuring that the channel does not experience non-stationary phenomena within the prediction period. Compared with the closest existing technology, the present application can use dual-base sensing channel data with mutual timing relationship to predict and perceive uncovered communication data, and combines environmental information and link information to improve prediction accuracy and convergence speed, and can learn small-scale timing relationships in long-distance non-stationary channels, which is missing in the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS
[0075] Figure 1 Building pipelines for visual modality features;
[0076] Figure 2 Schematic diagram of the network structure for channel multimodal predictive modeling;
[0077] Figure 3 Layout for ray tracing simulations;
[0078] Figure 4 is the delay power spectrum of the line-of-sight receiving route;
[0079] Figure 5 is the delay power spectrum of the non-line-of-sight receiving route;
[0080] Figure 6 is the probability density function of the stationary distance;
[0081] Figure 7 This is the prediction effect for the first time step of the first data under the line-of-sight condition;
[0082] Figure 8 This is the prediction effect for the last time step of the first data under the line-of-sight condition;
[0083] Fig. 9 The prediction effect of the first time step of the first data in the non-line-of-sight case;
[0084] Fig.10 The prediction effect of the last time step of the first data in the non-line-of-sight case;
[0085] Fig.11 It is the model convergence curve under the line-of-sight condition;
[0086] Fig.12 It is the model convergence curve under non-line-of-sight condition. DETAILED DESCRIPTION
[0087] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0088] A method for perceptually enhanced channel multimodal predictive modeling, the method comprising the steps of:
[0089] Step 1: Calculate the stationary distance of the target area according to the delay power spectrum measurement, and determine the number of time domain sampling points M of the perception timing sequence and the number of time domain sampling points N of the communication timing sequence according to the obtained stationary distance;
[0090] Step 2: discretize the delay power spectrum data according to the number of time domain sampling points M and the number of time domain sampling points N determined in step 1 to obtain discrete delay values, and then normalize the noise floor of the delay power spectrum data to obtain the time series modal characteristics of the perceived time series sequence;
[0091] Step 3: construct link modal features based on the statistical features and location topology features of the perceived time series;
[0092] Step 4: Based on the environment vector map, construct standardized and size-normalized visual modality features;
[0093] Step five, based on the temporal modal features obtained in step two, the link modal features constructed in step three, and the visual modal features constructed in step four, a multimodal network based on gated recurrent units, multilayer perceptrons, and pre-trained deep residual networks is constructed to obtain multimodal predictive modeling results.
[0094] In step 1, the stable distance of the target area at any time t1 is:
[0095]
[0096] Where Δd is the distance between adjacent sampling points, d qs (t1) is the stable distance at time t1; t2 is a time different from t1; c th is the threshold of correlation coefficient;
[0097]
[0098] γ(t1;τ)=|R(t1;τ)| 2
[0099] γ(t2;τ)=|R(t2;τ)| 2
[0100] Wherein, γ(t1; τ) is the delay power spectrum data in the unit of modulus square at time t1, γ(t2; τ) is the delay power spectrum data in the unit of modulus square at time t2, τ is the delay dimension, R(t1; τ) is the channel impulse response at time t1, and R(t2; τ) is the channel impulse response at time t2;
[0101] In the step 1, the method for determining the number of time domain sampling points M of the perception timing sequence and the number of time domain sampling points N of the communication timing sequence according to the obtained stationary distance is:
[0102] The perception mode is dual-base perception. The ISAC system model is that the transmitter is fixed and the receiver moves along a certain route. The first M moments are used as dual-base perception data, that is, the number of time domain sampling points M of the perception timing sequence, and the last N moments are used as communication data, that is, the number of time domain sampling points N of the communication timing sequence. To ensure that the time domain prediction system is meaningful, it is necessary to meet the multipath delay, power change and birth and death phenomena caused by the change of position and propagation environment in the channel within the M+N distance interval. Therefore, the number of time domain sampling points of the perception and communication timing sequence must meet the following requirements:
[0103]
[0104] in, is the average value of the stationary distance in the target area;
[0105] In step 2, the discrete delay value is:
[0106]
[0107] Where Δτ is the delay resolution, f s is the sampling rate, τ con is the continuous delay value, τ dis is the discrete delay value, is the rounding operator;
[0108] In the step 2, the method for normalizing the noise floor of the time-delay power spectrum data is:
[0109] Convert the delayed power spectrum data to logarithmic units:
[0110] Ω(t; τ) = 10·log 10 (Υ(t;τ))
[0111] Among them, γ(t; τ) is the delay power spectrum data in units of modulus square at time t, and Ω(t; τ) is the delay power spectrum data in units of dBm. All multipath powers are normalized according to the noise floor. If the noise floor is -p (dBm), all multipath powers in Ω(t; τ) are added with p, and then denormalized after the network outputs the result;
[0112] In step 2, the temporal modal features of the perceived temporal sequence are obtained as follows:
[0113] F t =Ω(t;τ)(t=1,2,...,M)+p
[0114] In the above formula, the first M moments in Ω(t; τ) are used as the input time steps of the network, the last N moments are used as the output time steps of the network, and the delay power spectrum data of the delay dimension is used as the feature dimension of each time step.
[0115] In step 3, the link modal features constructed are:
[0116]
[0117] The statistical features of the perception data include the mean value of the path loss and the mean value of the root mean square delay spread of the perception data in the first M time steps. The calculation formula is:
[0118]
[0119] in, is the transmission power of the mth sensing point, is the received power of the mth sensing point, and the calculation formula for the mean value of the RMS delay spread is:
[0120]
[0121] Among them, a m,l is the power of the lth multipath at the mth sensing point, L is the number of multipaths, τ m,l is the delay of the lth multipath at the mth sensing point;
[0122] The position topological features include the relative horizontal coordinate mean, relative vertical coordinate mean, relative three-dimensional distance mean, and pitch angle mean of the perception data of the first M time steps. The calculation method is:
[0123]
[0124] in, is the horizontal coordinate of the mth sensing point, x T is the horizontal coordinate of the transmitter. The calculation method is:
[0125]
[0126] in, is the ordinate of the mth sensing point, y T is the vertical coordinate of the transmitter. Relative three-dimensional distance mean The calculation method is as follows, where z R is the height of the receiver, z T is the height of the transmitter.
[0127]
[0128] The calculation method of the mean pitch angle is:
[0129]
[0130] In step 4, the constructed standardized and size-normalized visual modality features are:
[0131] F map ={I1,I2,...,I N}
[0132] Among them, I n (n=1,2,…,N) is the nth picture, which is the nth channel of the multimodal network input, consisting of J*J pixels. constitute:
[0133]
[0134] Among them, R is the number of data, and p is the original value of any pixel in the original image. The original image is constructed as follows:
[0135] The environmental vector map of the target scene is a .shp structure file. A building can be seen as a polygon composed of multiple boundary lines (line segments) when viewed from above, and a prism when viewed in three dimensions. Therefore, the shp structure file stores the height of each building and the coordinate information of the boundary lines.
[0136] When constructing a grayscale top view of a vector map representing a detailed propagation environment between a transmitter and a communication receiver, a grayscale top view of the entire scene is first drawn. The mapping relationship between grayscale values and building heights is:
[0137]
[0138] Where, d is the building height, d min is the minimum height of the building in the current scene, d maxis the maximum height of the buildings in the current scene, and g is the grayscale value converted from the current building height, ranging from 0 to 255.
[0139] like Figure 1 As shown in the figure, after drawing the grayscale bird's-eye view map of the entire area, the rectangular area around the transmitter and receiver is intercepted from the vector map, that is, the transceiver connection line extends to the surrounding area D e The area is the effective propagation area of the perception channel, based on which the image is cropped and rotated to the horizontal-vertical direction. The image rotation angle of the nth (n=1,2,…,N) communication data point is as follows, where is the horizontal coordinate of the nth communication receiving position, is the vertical coordinate of the nth communication receiving position.
[0140]
[0141] On this basis, the size of all images is reshaped to J*J, and the grayscale values of the images are normalized from 0-255 to the range of 0-1.
[0142] In the step 5, the method for constructing a multimodal network based on a gated recurrent unit, a multilayer perceptron and a pre-trained deep residual network is:
[0143] like Figure 2 As shown in the figure, based on the extracted temporal modal features, visual modal features and link modal features, a perception-enhanced channel multimodal predictive modeling network is designed. The composition of each module is described as follows:
[0144] The temporal modal features are extracted by multi-layer gated recurrent units. The delayed power spectrum data with M time steps is input, and the state of the last layer of the network at the last moment is taken as the output of the gated recurrent network.
[0145] The visual modality features of N channels are first extracted through a 1*1 convolution layer, transformed into three channels, and then the image features are extracted by a deep residual network with ImageNet pre-trained weights. The deep residual network structure adopted by the present invention is as follows: First, the original image is projected onto C channels through a 7×7 convolution layer with a step size of 2, and a pooling operation with a step size of 2 is performed on it, and then the processed features are input into four residual modules, which are represented by S1, S2, S3, and S4 respectively. Each residual module contains two paths, wherein the first path is a direct path of the input feature, and the second path performs two convolution operations on the feature to obtain the residual of the feature, and finally the features on the two paths are added. The stacking order of the network layer is: convolution layer, batch normalization (BN) layer, rectified linear unit (ReLU) activation layer, convolution layer, BN layer, residual addition, ReLU activation layer. In addition, all residual modules do not have a pooling layer, and downsampling is achieved only by setting the step size during convolution.
[0146] The statistical features of the perception data and the location topology features are concatenated after passing through the fully connected layer as link modal features.
[0147] After the time series mode, visual mode, and link mode are respectively passed through the feature extraction module based on the multi-layer perceptron, feature splicing is performed, and then the multi-layer perceptron is passed through and the embedding vector dimension is reshaped to (N*number of delay points). The root mean square error (RMSE) is used as the loss function:
[0148]
[0149] Among them, Loss is the value of the loss function, B represents the batch size, b (b = 1, 2, ..., B) is the index of B, F is the length of the delay power spectrum, that is, the feature length, and f (f = 1, 2, ..., F) is the index of F. represents the predicted delay power spectrum of the bth data, the fth feature, and the nth time step in this batch, Represents the real delayed power spectrum of the bth data, the fth feature, and the nth time step in this batch.
[0150] In step 5, the multimodal predictive modeling result obtained is:
[0151]
[0152] in, is the multi-layer perceptron function, is the deep residual network function, is the gated recurrent unit function, and ⊙ is the feature concatenation operator.
[0153] Example
[0154] To verify the correctness and feasibility of the present invention, we conducted simulation tests based on the above-mentioned perception-enhanced channel multimodal predictive modeling method. The test scenario is a complex urban scenario with a center frequency of 868MHz. Both the transmitting and receiving antennas are omnidirectional antennas. Ray tracing data is generated based on the commercial ray tracing software Wireless Insite. Figure 3 As shown in the figure, there are 10 transmitters with a spacing of 5m, the receivers are one line-of-sight and one non-line-of-sight path, and the receiver spacing is 0.17m. In the dataset, there are 10,500 data points in line-of-sight and 10,460 data points in non-line-of-sight. The signal noise floor is -150dBm.
[0155] The predictive network is based on Python's Pytorch architecture tool to build the network structure, configure network parameters and train. The image reshaping size L = 128, the batch size B = 64, the learning rate is 0.001, and the outward expansion distance of the cropped area image is 25m.
[0156] Figure 4 and Figure 5 The delay power spectra of the line-of-sight and non-line-of-sight receiving paths are plotted, and it can be seen that the channel has extensive multipath parameter changes and birth and death phenomena, and is non-stationary. The probability density function of the stationary distance of the target area is as follows: Figure 6 As shown in the figure, the mean of the stable distance of line-of-sight is 3.8746m, which is about 22.79 sampling points; the mean of the stable distance of non-line-of-sight is 3.5927m, which is about 21.13 sampling points. Based on this, the number of sensing points M = 25 and the number of communication points N = 10 are selected.
[0157] Figures 7 to 10 The prediction of the delay power spectrum of the first and last time steps of the first data after scrambling in line-of-sight and non-line-of-sight conditions is shown. It can be seen that good prediction results have been achieved, which confirms the feasibility of the present invention.
[0158] Fig.11 and Fig.12 The convergence curves of the model in line-of-sight and non-line-of-sight conditions are shown, and it can be seen that the error has reached less than 4dB, which is a good effect. At the same time, the prediction effect of any mode in the multi-mode only has decreased to varying degrees, which proves the effectiveness of the scheme designed by the present invention.
[0159] The obtained perception-enhanced channel multimodal prediction results can be used in long-distance synaesthesia systems in complex scenarios to provide accurate large- and small-scale communication channel data prediction results in locations where perception data is lacking, and can then be used for adaptive transmission design, communication system performance optimization, link-level and system-level simulation, etc.
[0160] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A perception-enhanced channel multimodal predictive modeling method, characterized in that The steps of the method include: Step 1: Calculate the stationary distance of the target area, and determine the number of time domain sampling points M of the perception timing sequence and the number of time domain sampling points N of the communication timing sequence according to the calculated stationary distance; Step 2: discretize the delay power spectrum data according to the number of time domain sampling points M and the number of time domain sampling points N determined in step 1 to obtain discrete delay values, and then normalize the noise floor of the delay power spectrum data to obtain the time series modal characteristics of the perceived time series sequence; Step 3: construct link modal features based on the statistical features and location topology features of the perceived time series; Step 4: Based on the environment vector map, construct standardized and size-normalized visual modality features; Step five, based on the temporal modal features obtained in step two, the link modal features constructed in step three, and the visual modal features constructed in step four, a multimodal network based on gated recurrent units, multilayer perceptrons, and pre-trained deep residual networks is constructed to obtain multimodal predictive modeling results.
2. The method for perceptually enhanced channel multimodal predictive modeling according to claim 1, characterized in that: In step 1, the stable distance of the target area at any time t1 is: Where Δd is the distance between adjacent sampling points, d qs (t1) is the stable distance at time t1; t2 is a time different from t1; c th is the threshold of the correlation coefficient; γ(t1;τ)=|R(t1;τ)| 2 γ(t2;τ)=|R(t2;τ)| 2 Among them, Υ(t1; τ) is the delay power spectrum data in units of modulus square at time t1, γ(t2; τ) is the delay power spectrum data in units of modulus square at time t2, τ is the delay dimension, R(t1; τ) is the channel impulse response at time t1, and R(t2; τ) is the channel impulse response at time t2.
3. The method for perceptually enhanced channel multimodal predictive modeling according to claim 2, characterized in that: In the step 1, the method for determining the number of time domain sampling points M of the perception timing sequence and the number of time domain sampling points N of the communication timing sequence according to the obtained stationary distance is: The sensing mode is dual-base sensing. The ISAC system model is that the transmitter is fixed and the receiver moves along a certain route. The first M moments are used as dual-base sensing data, i.e., the number of time domain sampling points M of the sensing timing sequence, and the last N moments are used as communication data, i.e., the number of time domain sampling points N of the communication timing sequence, satisfying: in, is the average value of the stationary distance in the target area.
4. The method for perceptually enhanced channel multimodal predictive modeling according to claim 3, characterized in that: In step 2, the discrete delay value is: Where Δτ is the delay resolution, f s is the sampling rate, τ con is the continuous delay value, τ dis is the discrete delay value, is the rounding operator; In the step 2, the method for normalizing the noise floor of the time-delay power spectrum data is: Convert the delayed power spectrum data to logarithmic units: Ω(t;τ)=10·log 10 (Y(t;τ)) Among them, γ(t; τ) is the delay power spectrum data in the unit of modulus square at time t, and Ω(t; τ) is the delay power spectrum data in the unit of dBm. All multipath powers are normalized according to the noise floor. If the noise floor is -p (dBm), all multipath powers in Ω(t; τ) are added with p, and then denormalized after the network outputs the result.
5. The method for perceptually enhanced channel multimodal predictive modeling according to claim 4, characterized in that: In step 2, the temporal modal features of the perceived temporal sequence are obtained as follows: F t =Ω(t;τ)(t=1,2,...,M)+p In the above formula, the first M moments in Ω(t; τ) are used as the input time steps of the network, the last N moments are used as the output time steps of the network, and the delay power spectrum data of the delay dimension is used as the feature dimension of each time step.
6. The method for perceptually enhanced channel multimodal predictive modeling according to claim 5, characterized in that: In step 3, the link modal features constructed are: The statistical features of the perception data include the mean value of the path loss and the mean value of the root mean square delay spread of the perception data in the first M time steps. The calculation formula is: in, is the transmission power of the mth sensing point, is the received power of the mth sensing point, and the calculation formula for the mean value of the RMS delay spread is: Among them, a m,l is the power of the lth multipath at the mth sensing point, L is the number of multipaths, τ m,l is the delay of the lth multipath at the mth sensing point; The position topological features include the relative horizontal coordinate mean, relative vertical coordinate mean, relative three-dimensional distance mean, pitch angle mean, relative horizontal coordinate mean of the perception data of the first M time steps. The calculation method is: in, is the horizontal coordinate of the mth sensing point, x T is the horizontal coordinate of the transmitter, relative to the vertical coordinate mean The calculation method is: in, is the ordinate of the mth sensing point, y T is the vertical coordinate of the transmitter. Relative three-dimensional distance mean The calculation method is as follows, where z R is the height of the receiver, z T is the height of the transmitter; The calculation method of the mean pitch angle is:
7. The method for perceptually enhanced channel multimodal predictive modeling according to claim 6, characterized in that: In step 4, the constructed standardized and size-normalized visual modality features are: F map ={I1,I2,...,I N } Among them, I n is the nth picture, n = 1, 2, ..., N, as the nth channel of the multimodal network input, By J*J pixels constitute: Among them, R is the number of data, p is the original value of any pixel in the original image, and the construction method of the original image is: The environment vector map of the target scene is a .shp structure file. A building is a polygon composed of multiple boundary lines or line segments when viewed from a bird's-eye view, and a prism when viewed in three dimensions. The .shp structure file stores the coordinate information of the height and boundary lines of each building. When constructing a grayscale top view of a vector map representing a detailed propagation environment between a transmitter and a communication receiver, a grayscale top view of the entire scene is first drawn. The mapping relationship between grayscale values and building heights is: Where, d is the building height, d min is the minimum height of the building in the current scene, d max is the maximum height of the building in the current scene, g is the grayscale value converted from the current building height, ranging from 0 to 255; After drawing the grayscale bird's-eye view map of the entire area, the rectangular area around the transmitter and receiver is intercepted from the vector map, which is the extension of the transceiver connection line to the surrounding area. e The area is the effective propagation area of the perception channel. Based on this, the image is cropped and rotated to the horizontal-vertical direction. The image rotation angle of the nth communication data point is as follows, where is the horizontal coordinate of the nth communication receiving position, is the ordinate of the nth communication receiving position, n = 1, 2, ..., N; On this basis, the size of all images is reshaped to J*J, and the grayscale values of the images are normalized from 0-255 to the range of 0-1.
8. The method for perceptually enhanced channel multimodal predictive modeling according to claim 7, characterized in that: In the step 5, the method for constructing a multimodal network based on a gated recurrent unit, a multilayer perceptron and a pre-trained deep residual network is: Based on the extracted temporal modal features, visual modal features and link modal features, a perception-enhanced channel multimodal predictive modeling network is designed. The components of each module are as follows: The temporal modal features are extracted by multi-layer gated recurrent units, and the delayed power spectrum data with M time steps is input, and the last layer of network state at the last moment is taken as the output of the gated recurrent network; The N-channel visual modality features are first extracted through a 1*1 convolutional layer and transformed into three channels. Then, the image features are extracted by a deep residual network with ImageNet pre-trained weights. The statistical features of the perception data and the location topology features are concatenated after passing through the fully connected layer to serve as link modal features. After the time series mode, visual mode, and link mode are respectively passed through the feature extraction module based on the multi-layer perceptron, feature splicing is performed, and then the multi-layer perceptron is passed through and the embedding vector dimension is reshaped to N*delay points. The root mean square error is used as the loss function: Among them, Loss is the value of the loss function, B represents the batch size, b is the index of B, F is the length of the delay power spectrum, that is, the feature length, and f is the index of F. represents the predicted delay power spectrum of the bth data, the fth feature, and the nth time step in this batch, Represents the true delayed power spectrum of the b-th data, the f-th feature, and the n-th time step in this batch, b = 1, 2, ..., B, f = 1, 2, ..., F.
9. The method for perceptually enhanced channel multimodal predictive modeling according to claim 8, characterized in that: The structure of the deep residual network is as follows: first, the original image is projected onto C channels through a 7×7 convolutional layer with a step size of 2, and a pooling operation with a step size of 2 is performed on it. Then, the processed features are input into four residual modules. The four residual modules are represented by S1, S2, S3, and S4 respectively. Each residual module contains two paths, where the first path is a direct path of the input feature, and the second path performs two convolution operations on the feature to obtain the residual of the feature. Finally, the features on the two paths are added together. The stacking order of the network layers is: convolutional layer, batch normalization layer, rectified linear unit activation layer, convolutional layer, BN layer, residual addition, ReLU activation layer. In addition, all residual modules have no pooling layer, and downsampling is only achieved by setting the step size during convolution.
10. A method for perceptually enhanced channel multimodal predictive modeling according to claim 8 or 9, characterized in that: In step 5, the multimodal predictive modeling result obtained is: in, is the multi-layer perceptron function, is the deep residual network function, is the gated recurrent unit function, and ⊙ is the feature concatenation operator.