Multi-mode intelligent channel modeling method for machine joint sense enabling

Abstract

The invention discloses a multi-modal intelligent channel modeling method for machine joint sense enabling, which belongs to the technical field of wireless communication, and comprises the following steps of: designing a scatterer recognition algorithm, intelligently analyzing and modeling a nonlinear mapping relationship between a perception data LiDAR point cloud and an electromagnetic propagation scatterer, and constructing a multi-modal intelligent channel model, so as to realize the modeling of a multi-modal intelligent channel. The method comprises the number, distance, angle, time delay and power parameters of clusters, effectively captures the non-stability and consistency of a wireless communication environment-channel, and can provide a high-precision channel modeling scheme for algorithm design and performance evaluation of a communication system.

Description

Technical Field

[0001] The present invention belongs to the field of wireless communication technology, and relates to a wireless communication system channel modeling technology, and specifically relates to a multimodal intelligent channel modeling method enabled by machine synesthesia, which is a multimodal intelligent channel modeling method assisted by a laser radar (Light Detection And Ranging, LiDAR) point cloud enabled by machine synesthesia (Synesthesia of Machines, SoM), and can provide a high-precision channel modeling solution for algorithm design and performance evaluation of communication systems. Background Art

[0002] Vehicle-to-Vehicle (V2V) communication plays an increasingly important role in Intelligent Transportation Systems (ITS). In order to improve the safety and efficiency of ITS, the Sixth-Generation (6G) V2V communication requires a comprehensive understanding of the environment around the vehicle and advanced V2V channel modeling research. Existing V2V channel models are mainly divided into two categories: statistical models and deterministic models. Statistical models randomly generate and determine channel parameters through statistical methods, with low complexity, but their accuracy is limited due to the random generation and determination of parameters. Unlike statistical models, deterministic models aim to reproduce the physical propagation process of radio electromagnetic waves in specific scenarios, with high accuracy, but huge computational overhead. Therefore, it is difficult for these two models to achieve a good balance between accuracy and complexity, and they cannot effectively support the design requirements of 6G V2V systems in ITS.

[0003] With the development of artificial intelligence (AI) and advanced technologies, a series of AI-based 6G V2V channel models have been proposed. AI's powerful ability to process massive data and identify complex patterns has significantly enhanced the ability to balance channel modeling accuracy and complexity. Existing work includes the proposed multipath component (MPC) clustering method, which implements MPC tracking based on Kalman filtering and Kuhn-Munkres algorithm, can identify the dynamic behavior of MPC and divide time-varying clusters. Other work proposes a fast AoA recognition model based on machine learning for the angle of arrival (AoA) in high-mobility V2V channels. In addition, artificial neural networks (ANNs) have also been used for channel modeling, demonstrating powerful information processing capabilities. Recent work has used ANNs to propose V2V channel models for path loss and shadowing effects, effectively leveraging the information processing potential of ANNs. Other work combines generative adversarial networks (GAN) and feedforward neural networks (FNN) to model the probability density function (PDF) and power distribution of received signals. Although AI-based V2V channel models have achieved significant improvements in accuracy, these models only rely on single radio frequency (RF) communication information and ignore the mapping relationship between the physical environment and the electromagnetic space. This limitation exacerbates the model's over-reliance on data sets and data collection scenarios, resulting in low scalability of the model. To overcome this limitation, a deeper understanding of the mapping relationship between the electromagnetic space and the physical environment is needed to achieve more accurate, general, and low-complexity V2V channel modeling.

[0004] Integrated Sensing And Communications (ISAC) technology provides a new opportunity to achieve more accurate, universal and low-complexity V2V channel modeling. ISAC can provide a deeper insight into the electromagnetic space and physical environment, and has attracted widespread attention in 6G V2V communication. The emergence of ISAC has opened up new possibilities for channel modeling. In addition, 6G autonomous driving vehicles in ITS will be equipped with multimodal devices, including millimeter-wave radar, LiDAR, RGB-D cameras and communication equipment. Compared with single RF radar perception, non-RF sensing devices (such as LiDAR) can capture more diverse environmental features, which can be used to improve communication performance and perception functions. In order to make full use of the multimodal information of communication equipment and various perception devices, existing work has proposed the new concept of "machine synesthesia" SoM based on the "human synesthesia" phenomenon, and clarified the goal of realizing intelligent multimodal perception and communication integration through artificial neural networks. Inspired by SoM, existing research has explored the mapping relationship between sensory data and large-scale channel fading (path loss), while the mapping relationship between sensory data and small-scale channel fading (multipath fading) is still relatively preliminary. Therefore, it is urgent to use artificial neural networks to analyze and model the nonlinear mapping relationship between sensory data and electromagnetic propagation scatterers to achieve more accurate, universal and low-complexity V2V channel modeling. Summary of the invention

[0005] In order to overcome the shortcomings of the above-mentioned prior art, the present invention proposes a multimodal intelligent channel modeling method enabled by machine synesthesia (SoM). By analyzing and modeling the nonlinear mapping relationship between the perception data LiDAR point cloud and the electromagnetic propagation scatterer, a multimodal intelligent channel model is constructed. The wireless communication environment-channel non-stationarity and consistency can be effectively captured. The model designed in this way provides high-precision channel modeling support for communication system algorithm design and performance evaluation.

[0006] The core of the present invention is a multimodal intelligent channel modeling method assisted by LiDAR point cloud enabled by "machine synesthesia" SoM. With the help of convolutional neural network SegNet and electromagnetic propagation mechanism, a scatterer recognition (ScaR) algorithm is proposed to intelligently parse and model the nonlinear mapping relationship between the perception data LiDAR point cloud and the electromagnetic propagation scatterer, thereby constructing a multimodal intelligent channel model to capture the environment-channel non-stationarity and consistency. In the multimodal intelligent channel model proposed in the present invention, two highly reliable simulation platforms, Wireless InSite and AirSim, are used to build a communication and perception intelligent fusion data set covering high, medium and low mobile agent density (MobileAgent Densities, MADs) scenes. The LiDAR point cloud is subjected to ground point removal, height threshold filtering and visible area filtering, and the physical environment grid features are extracted, including point cloud density, height and distance information. Furthermore, the ScaR algorithm is used and the electromagnetic propagation mechanism is considered to innovatively parse and model the complex mapping relationship between the LiDAR point cloud grid features and the electromagnetic propagation scatterer grid map. In order to model the environment-channel non-stationarity and consistency, the physical environment features obtained from the LiDAR point cloud are used in combination with the mapping relationship to identify the scatterer grid map, thereby modeling the spatial properties of the scatterer, namely the quantity and position. With the help of the complex mapping relationship between the LiDAR point cloud and the scatterer parsed and modeled by the ScaR algorithm, the multimodal intelligent channel modeling assisted by the LiDAR point cloud enabled by SoM is realized. The multimodal intelligent channel model proposed in the present invention has the ability to model and capture the non-stationarity and consistency of the wireless channel, thereby providing a high-precision channel model for the algorithm design, channel simulation and performance evaluation of the communication system, that is, by modeling the statistical characteristics of the channel, it helps designers evaluate the performance of the communication system and then optimize the communication system, including modulation schemes, signal coding, power control, etc.

[0007] To achieve the above objectives, the present invention designs a SoM-enabled LiDAR point cloud-assisted multimodal intelligent channel modeling method, including: 1. Building a communication and perception intelligent fusion data set based on Wireless InSite and AirSim simulation platforms; 2. Proposing a ScaR algorithm to parse and model the complex mapping relationship between the perception data LiDAR point cloud and the electromagnetic propagation scatterer; 3. With the help of the complex mapping relationship between the LiDAR point cloud and the scatterer parsed and modeled by the ScaR algorithm, the environment-channel non-stationarity and consistency are modeled, and a LiDAR point cloud-assisted multimodal intelligent channel model is constructed, which includes the number, distance, angle, delay and power parameters of clusters; 4. Obtaining the channel impulse response and channel transfer function, and introducing the frequency correlation factor of the cluster to characterize the frequency correlation of the path gain within the cluster.

[0008] The SoM-enabled LiDAR point cloud-assisted multimodal intelligent channel modeling method of the present invention comprises the following steps:

[0009] 1) Realize the deep integration and precise matching of the two simulation platforms, Wireless InSite and AirSim, and build a high-fidelity communication and perception intelligent fusion dataset, including LiDAR point cloud and electromagnetic propagation scatterer information.

[0010] 2) The LiDAR point clouds at the transmitting and receiving ends are fused to obtain complete physical environment feature information, and the ground points are further removed and height threshold filtering and visible area filtering are performed to reduce the redundant information of the LiDAR point cloud and extract part of the point cloud related to electromagnetic propagation.

[0011] 3) Considering that the main factors affecting the scatterer include the object position, material and angle, the effective LiDAR point cloud preprocessed in step 2) is subjected to raster feature extraction in terms of density, height and distance from the transceiver.

[0012] 4) Design the ScaR algorithm to parse and model the complex mapping relationship between the perception data LiDAR point cloud and the electromagnetic propagation scatterers, obtain the number of scatterers in each LiDAR point cloud grid, and achieve high-resolution and high-precision scatterer prediction.

[0013] 5) At the initial moment, based on the complex mapping relationship parsed and modeled in step 4), the LiDAR point cloud is used to identify the electromagnetic propagation scatterer grid map, and the grid map is converted into scatterer point coordinates. At the same time, dynamic scatterers and static scatterers are distinguished, and the K-Means clustering algorithm is further used to obtain dynamic clusters and static clusters.

[0014] 6) At the next time t n+1 =t n +Δt, as the physical environment changes, based on the complex mapping relationship analyzed and modeled in step 4), combined with the LiDAR point cloud data at the current moment, the scatterers at the current moment are identified, and dynamic scatterers and static scatterers are distinguished to obtain dynamic clusters and static clusters.

[0015] 7) Construct a SoM-enabled LiDAR point cloud-assisted multimodal intelligent channel model to model the number, distance, angle, delay, and delay parameters of clusters with high accuracy.

[0016] Furthermore, virtual links are introduced to characterize the complex processes between twin clusters (twin dynamic clusters and twin static clusters), and step 6) is continuously repeated. The ScaR algorithm is used to identify scatterers based on the perception data LiDAR point cloud, and the channel impulse response and channel transfer function at each moment are obtained. At the same time, frequency-related factors are introduced to model the frequency-related path gain, and a reliable simulation platform is provided for the performance evaluation and verification of system-level algorithms.

[0017] Through the above steps, SoM-enabled LiDAR point cloud-assisted multimodal intelligent channel modeling is realized, and the modeling of environment-channel non-stationarity and consistency is realized, thereby providing a reliable simulation platform for performance evaluation and verification of system-level algorithms.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] The present invention provides a SoM-enabled multimodal intelligent channel modeling method, and proposes a SoM-enabled LiDAR point cloud-assisted multimodal intelligent channel model with high accuracy. Using two highly reliable simulation platforms, WirelessInSite and AirSim, a communication and perception intelligent fusion dataset with high, medium and low MADs is constructed. The LiDAR point cloud is de-redundant, and the physical environment grid features are innovatively extracted. Furthermore, a ScaR algorithm is proposed, which uses a convolutional neural network SegNet and combines the electromagnetic propagation mechanism to deeply analyze and model the nonlinear mapping relationship between the perception data LiDAR point cloud and the electromagnetic propagation scatterer. Based on the parsed and modeled mapping relationship, scatterers are identified with the help of LiDAR point clouds, the spatial properties of the scatterers are obtained, the environment-channel non-stationarity and consistency are captured, and a SoM-enabled LiDAR point cloud-assisted multimodal intelligent channel model is constructed. The channel model proposed in the present invention can realize the successful design of the SoM system and provide a reliable simulation platform for the performance evaluation and verification of system-level algorithms.

[0020] The SoM-enabled LiDAR point cloud-assisted multimodal intelligent channel model proposed in the present invention has the following technical advantages:

[0021] (i) Based on the intelligent fusion dataset of communication and perception data under high, medium and low MADs, a SoM-enabled LiDAR point cloud-assisted multimodal intelligent channel model is constructed, which has high-precision modeling of channel statistical characteristics and is applicable to scenarios under different MADs conditions;

[0022] (ii) Using the electromagnetic propagation mechanism and ScaR algorithm, we innovatively process LiDAR point clouds and analyze and model the nonlinear mapping relationship between LiDAR point clouds and electromagnetic propagation scatterers;

[0023] (III) Based on the parsed and modeled mapping relationship, the LiDAR point cloud is used to identify electromagnetic propagation scatterers, and the number and position of scatterers are modeled with high resolution and high precision to capture the environment-channel non-stationarity and consistency, and to construct a SoM-enabled LiDAR point cloud-assisted multimodal intelligent channel model. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is the ScaR algorithm framework constructed by the present invention to parse and model the mapping relationship between the perception data LiDAR point cloud and the electromagnetic propagation scatterer.

[0025] Figure 2 is the geometrical framework of the multimodal channel model proposed in the present invention;

[0026] Among them, the distance between the transmitting and receiving ends is D tcv (t), the distance at the initial moment is D tcv (t0), the communication center frequency between the transmitter and receiver is f c , the kth static cluster close to the transmitter / receiver and the lth dynamic cluster close to the transmitter / receiver are defined as and The velocity vector of the dynamic cluster is The speeds of the transmitter and receiver are ν T (t) / ν R (t). DETAILED DESCRIPTION

[0027] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] The communication and perception intelligent fusion dataset constructed by the present invention is based on two high-fidelity simulation softwares, Wireless InSite and AirSim. The dataset constructed by the present invention method contains low, medium and high mobile agent density MADs, including high-precision LiDAR point cloud and scatterer information. Based on this dataset, the LiDAR point cloud is processed in combination with the electromagnetic propagation mechanism, and the ScaR algorithm is used. The network architecture is as follows: Figure 1 As shown in the figure, the density, height and distance information of the LiDAR point cloud are input, the network structure includes the SegNet backbone network and the designed task-specific modules, and the output is a scatterer grid map, which can parse and model the complex mapping relationship between the perception data LiDAR point cloud and the electromagnetic propagation scatterer. Figure 2 The geometric architecture of the constructed SoM-enabled LiDAR point cloud-assisted multimodal intelligent channel model is given.

[0029] The specific steps of the SoM-enabled LiDAR point cloud-assisted multimodal intelligent channel modeling method proposed in the present invention are as follows:

[0030] S10: Achieve deep fusion and precise matching between the two high-precision simulation platforms, Wireless InSite and AirSim, and build a high-fidelity communication and perception intelligent fusion dataset, including high-precision LiDAR point cloud and scatterer information.

[0031] S20: Fuse the LiDAR point clouds of the transmitter and receiver to obtain the physical environment information around the transmitter and receiver, and further remove the ground points and perform height threshold filtering and visible area filtering to reduce the redundant information of the LiDAR point cloud and extract part of the point cloud related to electromagnetic propagation.

[0032] S30: Considering the main factors affecting the scatterer, including the object position, material and angle, the pre-processed effective LiDAR point cloud is subjected to raster feature extraction in terms of density, height and distance from the transceiver.

[0033] S40: Design the ScaR algorithm to parse and model the complex mapping relationship between the perception data LiDAR point cloud and the electromagnetic propagation scatterers, obtain the number of scatterers in each LiDAR point cloud grid, and achieve high-resolution and high-precision scatterer prediction.

[0034] S50: At the initial moment, based on the complex mapping relationship parsed and modeled by the ScaR algorithm in step S40, the LiDAR point cloud is used to identify the electromagnetic propagation scatterer grid map, and the grid map is converted into scatterer point coordinates. At the same time, dynamic scatterers and static scatterers are distinguished, and the K-Means clustering algorithm is further used to obtain dynamic clusters and static clusters.

[0035] S60: At the next moment t n+1 =t n +Δt, as the physical environment changes, based on the complex mapping relationship parsed and modeled by the ScaR algorithm in step S40, the LiDAR point cloud at that moment is used to complete the scatterer identification at that moment, and distinguish between dynamic scatterers and static scatterers and obtain dynamic clusters and static clusters in the electromagnetic space.

[0036] S70: Build a multimodal intelligent channel model assisted by LiDAR point cloud enabled by SoM, and model the number, distance, angle, delay and delay parameters of clusters in electromagnetic space with high precision. Describe the complex transmission process between twin clusters in electromagnetic space by means of virtual links, and introduce frequency-related factors to model the frequency-related path gain, and repeat and iterate step S60 to obtain the channel impulse response and channel transfer function at each moment.

[0037] In step S10: using two high-precision simulation platforms, Wireless InSite and AirSim, to construct a high-fidelity communication and perception intelligent fusion data set. This step includes the following processes S11-S12:

[0038] S11: Realize the deep integration of Wireless InSite and AirSim simulation platforms.

[0039] For the construction of simulation scenes, it is necessary to deeply integrate the Wireless InSite and AirSim simulation platforms. The scene is rendered with high precision in AirSim, and the scene is reasonably simplified in Wireless InSite to meet its modeling requirements. When setting the trajectory of the mobile agent, the coordinates of the mobile agent in the Wireless InSite and AirSim platforms need to be modified synchronously to generate a dynamic scene with deep integration of communication and perception. Specifically, a large number of internal files need to be parsed in WirelessInSite to batch adjust the coordinates of the mobile agent, while in AirSim, the position of each mobile agent needs to be set frame by frame.

[0040] S12: Achieve precise matching between Wireless InSite and AirSim simulation platforms.

[0041] When building the simulation scene, it is necessary to ensure that the Wireless InSite and AirSim simulation platforms are accurately aligned for each object in the scene. Specifically, each object in the two platforms needs to be drawn and built one by one to ensure that they are exactly the same in size, position, and size. For the trajectory setting of the mobile agent, it is necessary to ensure that the trajectory of each mobile agent in the Wireless InSite and AirSim platforms is completely synchronized. The specific operation is to ensure that the position of each mobile agent in the two platforms is exactly the same at every moment, so as to achieve accurate matching of the trajectory of the mobile agent.

[0042] In step S20: the LiDAR point cloud of the transmitting and receiving ends is integrated to remove the redundant information of the LiDAR point cloud data. This step includes the following processes S21 to S23:

[0043] S21: Based on the three-dimensional coordinates of the LiDAR point cloud at the transmitting end and the three-dimensional coordinates of the LiDAR point cloud at the receiving end, as well as the coordinates and movement direction of the intelligent body, point cloud registration is performed to place the LiDAR point clouds of the transmitting and receiving ends in the same world coordinate system, thereby obtaining complete physical environment information around the transmitting and receiving ends.

[0044] S22: Based on the spliced ​​and fused LiDAR point clouds of the transmitter and the receiver, determine and delete ground points with a vertical coordinate of zero in the LiDAR point cloud to reduce data redundancy of the LiDAR point cloud.

[0045] S23: A height threshold filtering method is introduced, that is, the LiDAR point cloud with a height greater than a certain threshold is removed, and the visible area is further introduced to filter the point cloud. The present invention sets the visible area as an ellipsoid with the transceiver as the focus, and takes out the point cloud inside the ellipsoid as the pre-processed valid LiDAR point cloud, thereby screening out the LiDAR point cloud part that is most relevant to electromagnetic propagation.

[0046] In step S30: physical environment grid feature extraction is performed on the pre-processed valid LiDAR point cloud. This step includes the following processes S31 to S33.

[0047] S31: Extract the density grid features of the LiDAR point cloud. The value in each grid is the ratio of the total number of LiDAR point clouds in the grid to the area of ​​the grid, calculated as

[0048]

[0049] Among them, S grid (t) represents the LiDAR point cloud set in the grid, |·| represents the number of elements in the set, and x grid (t) and y grid (t) represent the length and width of the grid, respectively, x grid (t)×y grid (t) represents the area of ​​the grid.

[0050] S32: Extract the height grid feature of the LiDAR point cloud. The value in each grid is the maximum value of the height of the LiDAR point cloud in the grid, calculated as

[0051]

[0052] in, For S grid (t) Coordinates of the LiDAR point cloud along the z-axis.

[0053] S33: Extract the distance grid feature of the LiDAR point cloud. Consider a position encoding layer. For the grid with LiDAR point cloud, the distance is calculated as the average distance between the center of the grid and the communication transceiver, that is,

[0054] d grid (t)=(d Tg (t)+d Rg (t)) / 2

[0055] Among them, dTg (t) is the distance between the center of the grid and the communication transmitter, d Rg (t) is the distance between the center of the grid and the communication receiving end.

[0056] In step S40: design a ScaR algorithm (scatterer recognition algorithm, or scatterer prediction network model) to parse and model the complex mapping relationship between the perception data LiDAR point cloud and the electromagnetic propagation scatterer, that is, to predict the spatial information of the electromagnetic propagation scatterer from the perception data LiDAR point cloud. This step includes the following processes S41 to S43.

[0057] S41: The input of the ScaR algorithm is the density, height, and distance grid features of the valid LiDAR point cloud extracted in S30. The dimension of one sample is 3×80×80, that is, three channels, and the dimension of each channel is 80×80.

[0058] S42: The output of the ScaR algorithm is a grid map formed by scatterers in the visible area. Its dimension represents the precision of the prediction of the scatterer position and can be selected according to actual needs.

[0059] S43: The hidden layer structure of the ScaR algorithm includes two parts, the convolutional neural network SegNet and the task-specific module. SegNet adopts an encoder-decoder structure. The encoder extracts features through a series of convolutional layers, and the decoder restores the feature map into a segmentation map of the same size as the input image through deconvolution (transposed convolution). The decoder part of SegNet uses the pooling index information from the encoder pooling layer, which helps to more accurately restore the spatial resolution, especially in detail recovery. The task-specific module includes an image interpolation layer, a convolutional layer, a batch normalization layer and an activation function, wherein the image interpolation layer is implemented using the F.interpolate function in PyTorch, and the activation function uses the ReLu function. The task-specific module adjusts the output dimension of the task head of SegNet to the dimension required by the present invention, thereby realizing the output of the scatterer grid map.

[0060] In step S50: at the initial moment, the sensing data LiDAR point cloud is used to identify electromagnetic propagation scatterers, and further obtain and distinguish dynamic and static clusters. This step includes the following processes S51-S52.

[0061] S51: Use the perception data LiDAR point cloud to identify electromagnetic propagation scatterers. Based on the complex mapping relationship between the perception data LiDAR point cloud and the electromagnetic propagation scatterers analyzed and modeled in step S40, at the initial moment, the electromagnetic propagation scatterers are identified based on the LiDAR point cloud. The identification result scatterer raster map is converted into scatterer coordinates, and the information that can be obtained includes the number, delay and angle parameters of the scatterers. Therefore, the scatterers identified based on the perception data LiDAR point cloud promote the construction of a multimodal intelligent channel model with accurate channel parameters.

[0062] S52: Obtain and distinguish dynamic and static clusters. First, based on the height grid features of the LiDAR point cloud, since the height of the vehicle is relatively small, if the value in the height grid of the LiDAR point cloud is less than a set height threshold, it is detected as a mobile agent; otherwise, the LiDAR point cloud grid is detected as a static object, such as trees and buildings. When a scatterer is located in the grid of a detected mobile agent, the scatterer is a dynamic scatterer; when a scatterer is located in the grid of a detected static object, the scatterer is a static scatterer. Furthermore, the K-Means clustering algorithm is used for dynamic scatterers and static scatterers, respectively, to obtain dynamic clusters and static clusters in the electromagnetic space.

[0063] In step S60: at the next time t n+1 =t n +Δt, as the physical environment changes, based on the complex mapping relationship parsed and modeled by the ScaR algorithm in step S40, the LiDAR point cloud at that moment is used to complete the scatterer identification at that moment, and distinguish dynamic scatterers from static scatterers and obtain dynamic clusters and static clusters. This step includes the following processes S61-S62.

[0064] S61: Identify the next time t n+1 =t n +Δt scatterers. Since the mapping relationship captures the close interaction between the physical environment and the electromagnetic space, the predicted electromagnetic propagation scatterers will change continuously and consistently with the LiDAR point cloud, thus modeling the environment-channel non-stationarity in the time domain. In addition, the perception data LiDAR point clouds at adjacent moments are similar, and the identified scatterers are also similar at adjacent moments, thus capturing the environment-channel consistency in the time domain.

[0065] S62: Distinguish the next moment t n+1 =t n Similar to step S52, for time t n+1 =t nThe height grid features of the LiDAR point cloud under +Δt are used to identify mobile agents and static objects in the physical environment. When a scatterer is located in the grid of a mobile agent, it is a dynamic scatterer; when a scatterer is located in the grid of a static object, it is a static scatterer. Furthermore, the K-Means clustering algorithm is used for dynamic scatterers and static scatterers to obtain dynamic clusters and static clusters in the electromagnetic space.

[0066] In step S70: a multimodal intelligent channel model assisted by a SoM-enabled LiDAR point cloud is constructed to model the number, distance, angle, delay and delay parameters of clusters with high precision. A virtual link is used to characterize the complex transmission process between twin clusters in the electromagnetic space, and a frequency-related factor is introduced to model the frequency-related path gain.

[0067] S71: Build an architecture for wireless channel model embedded in the environment based on twin clusters, such as Figure 2 Specifically, the distance between the transmitting and receiving ends is D tcv (t), the distance at the initial moment is D tcv (t0), the communication center frequency between the transmitter and receiver is f c , the communication bandwidth is B. The number of static scatterers and dynamic scatterers is I s (t0) and I d (t0), the position and distance parameter D of the i / jth static / dynamic scatterer s,i (t) / D d,j (t) can be directly obtained from the predicted scatterer. The angular parameters of the scatterer are such as the horizontal departure angle (Azimuth Angle of Departure, AAoD) φ s,i (t) / φ d,j (t), horizontal angle of arrival (Azimuth Angle of Arrival, AAoA) ψ s,i (t) / ψ d,j (t), Elevation Angle of Departure (EAoD)ζ s,i (t) / ζ d,j (t), Elevation Angle of Arrival (EAoA) Generated by Gaussian distribution. Then, the static scatterers and dynamic scatterers are clustered using K-Means algorithm to obtain static clusters and dynamic clusters, respectively. The number of static clusters and dynamic clusters is J s (t0) and J d(t0) is generated by Logistic distribution. A part of the clusters close to the transmitter and the receiver are randomly matched to form twin clusters. There will be an abstract virtual link between the twin clusters for communication transmission, representing other clusters that may exist or reflection and scattering components above the second order. This model defines the kth static cluster close to the transmitter / receiver and the lth dynamic cluster close to the transmitter / receiver as and The velocity vector of the dynamic cluster is The speeds of the transmitter and receiver are ν T (t) / ν R (t). Furthermore, a virtual link approach is used to characterize the complex transmission process between twin clusters in electromagnetic space.

[0068] S72: Determine the channel impulse response. First, the complex channel gain of the Line-of-Sight (LoS) component of the present invention is given:

[0069]

[0070] Among them, R(t) is the rectangular window function, that is,

[0071]

[0072] The Doppler frequency, phase shift and time delay are calculated as

[0073]

[0074] Among them, <·> represents the inner product, represents the phase shift at the initial moment, λ is the carrier wavelength, and c is the speed of light. Secondly, for the ground reflection component, the complex channel gain can be expressed as

[0075]

[0076] Among them, P GR (t), f GR,T / R (t) and are the power of the ground reflection component, the Doppler frequency and phase offset of the transmitter / receiver respectively. Next, for the non-line-of-sight (NLoS) component, after the kth static cluster kth i The complex channel gain of a scatterer is expressed as

[0077]

[0078] in, is the normalized static scatterer power, and the Doppler frequency can be expressed as

[0079]

[0080] in, Indicates the transmitter / receiver and static cluster kth i The time delay can be calculated as

[0081]

[0082] Among them, τ i ′(t) represents the static twin cluster and The virtual delay of the virtual link between follows an exponential distribution. The phase offset can be calculated as

[0083]

[0084] in, represents the phase shift at the initial moment.

[0085] Finally, similar to static clusters Through dynamic cluster The first j The complex channel gain of a dynamic scatterer can be expressed as

[0086]

[0087] in, is the normalized dynamic scatterer power, and the Doppler frequency can be expressed as

[0088]

[0089] Where, represents the transmitter / receiver and the dynamic cluster The first j The distance between dynamic scatterers. The time delay can be calculated as

[0090]

[0091] Among them, τ′ j (t) represents the static twin cluster and The virtual delay of the virtual link between follows an exponential distribution. The phase offset can be calculated as

[0092]

[0093] in, represents the phase shift at the initial moment.

[0094] Finally, based on the above conclusions, the channel impulse response of the SoM-enabled LiDAR point cloud-assisted multimodal intelligent channel model can be expressed as:

[0095]

[0096] Where h(t,τ) is the channel impulse response, J s (t) and J d (t) are the number of static clusters and dynamic clusters, respectively, I s (t) and I d (t) are the number of static scatterers and dynamic scatterers, is the Rice factor, h LoS (t), h GR (t) and They are LoS component, ground reflection component, and the kth static cluster. kth i scatterers and passing dynamic clusters The first j The complex channel gain of a dynamic scatterer, τ LoS , τ GR , and are the delays corresponding to the above complex channel gains, Θ GR , Θ s and θ d They represent the power ratios of the ground reflection component, static cluster and dynamic cluster respectively, satisfying that the sum of the three is one.

[0097] S73: Calculate the channel transfer function and introduce frequency-related factors to model the frequency-related path gains of the automatic and static clusters. Based on the channel impulse response obtained in step S72, perform Fourier transform on the delay τ to obtain the channel transfer function, which can be written as:

[0098]

[0099] In order to obtain the frequency-dependent path gain, a frequency-dependent factor is introduced, which is

[0100]

[0101] in, is a frequency-dependent factor that depends on the specific communication scenario. This factor can be used to calculate the frequency-dependent path gain, thus capturing the frequency non-stationarity of the channel. H(t,f) is the channel transfer function, and J s (t) and J d (t) are the number of static clusters and dynamic clusters, respectively, I s (t) and Id (t) are the number of static scatterers and dynamic scatterers, is the Rice factor, h LoS (t), h GR (t) and They are LoS component, ground reflection component, and the kth static cluster. kth i scatterers and passing dynamic clusters The first j The complex channel gain of a dynamic scatterer, τ LoS , τ GR , and are the delays corresponding to the above complex channel gains, Θ GR , Θ s and θ d They represent the power ratios of the ground reflection component, static cluster and dynamic cluster respectively, satisfying that the sum of the three is one.

[0102] In summary, the present invention proposes a SoM-enabled LiDAR point cloud-assisted multimodal intelligent channel model. Based on the intelligent fusion data set of communication and perception data under high, medium and low MADs, a SoM-enabled LiDAR point cloud-assisted multimodal intelligent channel model is constructed, which has high-precision modeling of channel statistical characteristics and is suitable for scenarios under different MADs conditions; the electromagnetic propagation mechanism and ScaR algorithm are used to innovatively process the LiDAR point cloud and parse and model the nonlinear mapping relationship between the LiDAR point cloud and the electromagnetic propagation scatterer; based on the parsed and modeled mapping relationship, the LiDAR point cloud is used to identify the electromagnetic propagation scatterer, and the number and position of the scatterer are modeled with high resolution and high precision, so as to construct a SoM-enabled LiDAR point cloud-assisted multimodal intelligent channel model to capture the environment-channel non-stationarity and consistency.

[0103] It should be noted that the purpose of publishing the embodiments is to help further understand the present invention, but those skilled in the art can understand that various substitutions and modifications are possible without departing from the scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the contents disclosed in the embodiments, and the scope of protection claimed by the present invention shall be subject to the scope defined in the claims.

Claims

1. A multimodal intelligent channel modeling method enabled by machine synesthesia, characterized in that: Design scatterer recognition algorithms, intelligently analyze and model the nonlinear mapping relationship between the LiDAR point cloud of perception data and the electromagnetic propagation scatterers, and build a multimodal intelligent channel model to effectively capture the wireless communication environment - channel non-stationarity and consistency; including: 1) Build a high-fidelity communication and perception intelligent fusion dataset, including LiDAR point cloud and electromagnetic propagation scatterer information; 2) Perform data preprocessing to obtain effective LiDAR point cloud; 3) For the effective LiDAR point cloud after preprocessing, the density, height, and distance from the transmitting and receiving ends are extracted by rasterization, and the density raster features of the LiDAR point cloud, the height raster features of the LiDAR point cloud, and the distance raster features of the LiDAR point cloud are obtained respectively; 4) By designing a scatterer recognition algorithm (ScaR), the complex mapping relationship between the perception data LiDAR point cloud and the electromagnetic propagation scatterers is analyzed and modeled, and scatterer recognition prediction is performed to obtain the number of scatterers in each LiDAR point cloud grid; including: The input of the ScaR algorithm is the density, height and distance grid features of the effective LiDAR point cloud; the output is a grid map formed by scatterers in the visible area; The hidden layer structure includes a convolutional neural network SegNet and a task-specific module; SegNet adopts an encoder-decoder structure, where the encoder extracts features through a series of convolutional layers, and the decoder restores the feature map to a segmentation map of the same size as the input image through deconvolution; the decoder uses the pooling index information from the encoder pooling layer; the task-specific module includes an image interpolation layer, a convolutional layer, a batch normalization layer, and an activation function, which are used to adjust the output dimension of the SegNet task head, thereby realizing the output of the scatterer grid map; 5) At the initial moment, the electromagnetic propagation scatterer grid map is identified using the LiDAR point cloud, and the scatterer grid map is converted into scatterer point coordinates. The information obtained includes the number, delay and angle parameters of the scatterers; dynamic scatterers and static scatterers are distinguished at the same time, and dynamic clusters and static clusters are further obtained; 6) At the next moment, based on the LiDAR point cloud data at the current moment, identify the scatterers at the current moment, distinguish dynamic scatterers from static scatterers, and obtain dynamic clusters and static clusters; 7) Construct a multimodal intelligent channel model assisted by SoM-enabled LiDAR point cloud, and model the number, distance, angle, delay and delay parameters of clusters with high accuracy; including: S71: Build the architecture of wireless channel model based on twin cluster environment embedding, and use virtual links to represent the transmission process between twin clusters in electromagnetic space; S72: Determine a channel impulse response; the channel impulse response of the multimodal intelligent channel model enabled by machine synesthesia is expressed as: Where h(t,τ) is the channel impulse response, J s (t) and J d (t) are the number of static clusters and dynamic clusters, respectively, I s (t) and I d (t) are the number of static scatterers and dynamic scatterers, is the Rice factor, h LoS (t), h GR (t) and They are LoS component, ground reflection component, and the kth static cluster. k i scatterers and passing dynamic clusters The first j The complex channel gain of a dynamic scatterer, τ LoS , τ GR , and are the delays corresponding to the complex channel gains; Θ GR , Θ s and θ d They represent the power ratios of the ground reflection component, static cluster and dynamic cluster respectively, and their sum is 1; S73: Calculate the channel transfer function and introduce the frequency correlation factor to model the frequency correlation path gains of the automatic and static clusters respectively; Based on the channel impulse response, the Fourier transform is performed on the delay τ to obtain the channel transfer function, which is expressed as: After introducing the frequency-related factor, it can be expressed as: in, is the frequency correlation factor; H(t,f) is the channel transmission function; In this way, the frequency-dependent path gain is obtained, realizing multimodal intelligent channel modeling enabled by machine synesthesia.

2. The multimodal intelligent channel modeling method enabled by machine synesthesia as claimed in claim 1, characterized in that: Step 1) is to use Wireless InSite and AirSim simulation platforms to build a high-fidelity communication and perception intelligent fusion dataset for building simulation scenarios; This includes achieving deep integration and precise matching of Wireless InSite and AirSim simulation platforms.

3. The multimodal intelligent channel modeling method enabled by machine synesthesia as claimed in claim 2, characterized in that: include: S11: Achieve deep integration of Wireless InSite and AirSim simulation platforms; The scene was rendered with high precision in AirSim and simplified in Wireless InSite; When setting the trajectory of the mobile agent, the coordinates of the mobile agent in the Wireless InSite and AirSim platforms are modified synchronously to generate a dynamic scene with deep integration of communication and perception; S12: Achieve precise matching between Wireless InSite and AirSim simulation platforms; Draw and build each object on both platforms one by one, making the objects exactly the same in size, position and size; Set the trajectory of the mobile agent so that the trajectory of each mobile agent in the Wireless InSite and AirSim platforms is completely synchronized.

4. The multimodal intelligent channel modeling method enabled by machine synesthesia as claimed in claim 1, characterized in that: Step 2) preprocessing the data, fusing the LiDAR point clouds at the transmitting and receiving ends to obtain complete physical environment feature information; Remove ground points and perform height threshold filtering and visible area filtering to reduce redundant information of LiDAR point clouds and extract part of point clouds related to electromagnetic propagation; specifically include: S21: Based on the three-dimensional coordinates of the LiDAR point cloud of the transmitter and the three-dimensional coordinates of the LiDAR point cloud of the receiver, as well as the coordinates and movement direction of the intelligent body, point cloud registration is performed to place the LiDAR point clouds of the transmitter and receiver in the same world coordinate system, thereby obtaining complete physical environment information around the transmitter and receiver; S22: Based on the spliced ​​and fused LiDAR point clouds of the transmitter and the receiver, determine and delete ground points with a vertical coordinate of zero in the LiDAR point cloud to reduce data redundancy of the LiDAR point cloud; S23: Adopt the height threshold filtering method, that is, remove the LiDAR point cloud with a height greater than a certain threshold; further filter the point cloud according to the visible area, set the visible area as an ellipsoid with the transceiver as the focus, and take out the point cloud inside the ellipsoid as the preprocessed valid LiDAR point cloud, so as to screen out the LiDAR point cloud part that is most relevant to electromagnetic propagation.

5. The multimodal intelligent channel modeling method enabled by machine synesthesia as claimed in claim 1, characterized in that: Step 3) extracting rasterized features, specifically including: S31: extracting density grid features of LiDAR point cloud: the value in each grid is the ratio of the total number of LiDAR point clouds in the grid to the area of ​​the grid; S32: Extracting the height grid feature of the LiDAR point cloud: the value in each grid is the maximum value of the height of the LiDAR point cloud in the grid; S33: Extracting distance grid features of LiDAR point cloud: Based on the position encoding layer, for the grid where the LiDAR point cloud exists, the distance is calculated as the average distance between the center of the grid and the communication transceiver.

6. The multimodal intelligent channel modeling method enabled by machine synesthesia as claimed in claim 1, characterized in that: Step 4) In the hidden layer of the ScaR algorithm, the image interpolation layer of the task-specific module is specifically implemented using the F.interpolate function in PyTorch; the activation function uses the ReLu function.

7. The multimodal intelligent channel modeling method enabled by machine synesthesia as claimed in claim 1, characterized in that: In step 5), a clustering algorithm is specifically used to obtain dynamic clusters and static clusters in the electromagnetic space.

8. The multimodal intelligent channel modeling method enabled by machine synesthesia as claimed in claim 7, characterized in that: Differentiate between dynamic clusters and static clusters, specifically including: First, according to the height grid map of the LiDAR point cloud, if the value in the height grid of the LiDAR point cloud is less than the set height threshold, it is detected as a mobile agent; otherwise, the LiDAR point cloud grid is detected as a static object; when the scatterer is located in the grid of the detected mobile agent, the scatterer is a dynamic scatterer; when the scatterer is located in the grid of the detected static object, the scatterer is a static scatterer.

9. The multimodal intelligent channel modeling method enabled by machine synesthesia as claimed in claim 1, characterized in that: In step S71, the position and distance parameters of the static scatterer and the dynamic scatterer are obtained from the predicted scatterer; multiple angle parameters of the scatterer are generated by Gaussian distribution; the number of static clusters and dynamic clusters is generated by Logistic distribution; and the formed twin clusters communicate and transmit through virtual links.

Images