Ray Tracing Measurement Correction Method, System and Electronic Device Based on Multimodal Fusion

Abstract

The present invention discloses a ray tracing measurement correction method, system and electronic device based on multimodal fusion, which mainly solves the problem of insufficient prediction accuracy of traditional models in complex propagation environments. Its implementation scheme includes: respectively obtaining a convolutional layer training sample set, a test sample set and a self-attention layer training sample set including measured data; constructing a multimodal fusion network model including a convolutional neural network module, a self-attention mechanism module and a fusion module, and these modules are cascaded in sequence; using the training sample set to train the multimodal fusion prediction network model; using the test sample set as the input of the trained prediction network model for forward propagation to obtain the predicted value of the measured received power of the radio wave propagation. The present invention effectively reduces the error between the measured value and the theoretical value of the receiving point power in complex scenarios, can process different types of data in a targeted manner, improve the received power prediction accuracy, meet the application requirements of complex propagation environments, and can be used in electronic countermeasures, automated command and other systems and electromagnetic wave propagation.

Description

Technical Field

[0001] The present invention belongs to the field of big data and machine learning application technology, and specifically relates to a ray tracing measurement correction method, system and electronic equipment, which can be used in electronic countermeasures, automated command systems and electromagnetic wave propagation. Background Art

[0002] Ray tracing algorithms are a branch of radio wave propagation computational models. Based on geometric optics and consistent diffraction theory, these algorithms simulate and emulate radio wave propagation paths by constructing specific scenario information. Radio wave propagation models primarily include empirical and deterministic models. Empirical models are mathematical models fitted based on a large amount of measured data and can only be used in specific scenarios. However, in complex environments such as small urban areas, factors such as building obstruction can significantly reduce the prediction accuracy of empirical models. Ray tracing methods are widely used within deterministic models. Compared to empirical models, ray tracing algorithms can more accurately reflect the characteristics of the propagation environment, significantly improving computational accuracy. However, due to the complexity of the radio wave propagation environment, many environmental factors cannot be reflected through spatial modeling, resulting in a certain degree of error between the simulation results and the measured values at the receiving point.

[0003] Patent publication number CN116401940A discloses a radio wave propagation simulation method based on big data fitting. This method primarily predicts transmission loss in a scenario by training a radio wave propagation model. This method uses big data fitting to simulate radio wave propagation, replacing ray tracing methods to reduce computational complexity. However, because it does not consider the measured values at the receiving point, this can lead to significant discrepancies between the predicted and measured results in complex scenarios.

[0004] Patent document CN115828740A discloses a transmission loss prediction method for a radio wave propagation model based on a neural network. This method fine-tunes the simulation training model by introducing a small amount of measured data, making the model more realistic. However, since it does not take into account the propagation scenario information, it may lead to difficulties in predicting radio wave propagation in complex scenarios. Summary of the Invention

[0005] The purpose of the present invention is to address the deficiencies of the above-mentioned existing technologies and propose a ray tracing measurement correction method, system and electronic equipment based on multimodal fusion, so as to make full use of the measured data and improve the accuracy of radio wave propagation prediction by combining multimodal data and measured data.

[0006] To achieve the above-mentioned purpose, the technical solutions adopted by the present invention include:

[0007] 1. A ray tracing measurement correction method based on multimodal fusion, characterized by comprising:

[0008] Obtain the convolutional layer training sample set R1, the self-attention layer training sample set R2, and the test sample set E respectively, where R2 includes the measured data;

[0009] Construct a multimodal fusion network model H including a convolutional neural network module H1, a self-attention mechanism module H2, and a fusion module H3, wherein the H1 module is connected in parallel with the H2 module and then cascaded with the H3 module;

[0010] Use the training sample set to train the multimodal fusion prediction network model H;

[0011] The test sample set E is used as the trained prediction network model H * The input is forward propagated to obtain the predicted values of the received power actually measured by the radio wave propagation MK.

[0012] Furthermore, the convolutional layer training sample set R1 is obtained by concatenating the three feature vectors, namely, the building height feature vector f1, the emission point position feature vector f2, and the receiving point position feature vector f3, in the channel dimension to generate a three-dimensional tensor R1 containing multimodal information;

[0013] Furthermore, the self-attention layer training sample set R2 is obtained by first normalizing the electromagnetic characteristic influencing factors of the target point and the measured values of the auxiliary points and their influencing factors, and then splicing the normalized M target point vector sets B with the N auxiliary point vector sets A to obtain M splicing vectors C, and selecting K vectors in C as the training sample set R2.

[0014] Furthermore, the test sample set E is obtained by taking the remaining MK splicing vectors in the splicing vector C as the test sample set E.

[0015] Furthermore, the structure and function of each module in the multimodal fusion network model H include the following:

[0016] Convolutional neural network module H1, used to process map information, which includes two convolution-activation function-pooling layers, a flattening layer, a first fully connected layer-activation layer, and a first fully connected layer.

[0017] The self-attention mechanism module H2 is used to process the radio wave influence factor information of the receiving point. It includes two self-attention-normalization layers, a second fully connected-activation function layer, and a second fully connected layer connected in sequence. The input and output of each self-attention layer are connected by an identical path across layers and input into the normalization layer.

[0018] Fusion module H3, used to fuse spatial information and radio wave impact factor information, which includes a splicing layer and a third fully connected layer;

[0019] The output of the first fully connected layer and the output of the second fully connected layer are connected in parallel and then connected to the splicing layer to form a multimodal fusion network model H.

[0020] 2. A ray tracing measurement correction system based on multimodal fusion, comprising:

[0021] The data acquisition module is used to obtain ray tracing data and measured received power data, including radio wave propagation characteristic data such as operating frequency band, emission point and receiving point location, and normalize them;

[0022] The network module is used to build a multimodal fusion network model, including a convolutional neural network module, a self-attention mechanism module, and a fusion module, which are used to process map information, radio wave influencing factor information, and integrate spatial information with radio wave propagation characteristics respectively;

[0023] The training module is used to train the multimodal fusion network model, including inputting data into the network, performing forward propagation, calculating the loss value, and iteratively updating the weights and parameters through backpropagation until the maximum number of iterations is reached or the loss value converges, thereby obtaining the trained prediction network model;

[0024] The prediction module is used to predict the measured received power of unknown receiving points using the trained multimodal fusion network model, and to correct the ray tracing results to improve accuracy.

[0025] 3. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the steps of the multimodal fusion ray tracing measurement correction method is implemented.

[0026] Compared with the prior art, the present invention has the following advantages:

[0027] First, the present invention improves the accuracy of radio wave propagation predictions by constructing a multimodal fusion network. By inputting radio wave propagation influencing factor data and map data into separate network modules, the network can more specifically process different types of data. The self-attention module processes radio wave propagation influencing factors. By calculating feature correlations and weighting, it automatically identifies key features and suppresses secondary features, enabling in-depth exploration of the correlation between target points and auxiliary points. The convolutional neural network module processes map data and learns local patterns within the map, such as building occlusion and obstacle distribution, helping the network understand the path distribution of electromagnetic waves within the propagation environment.

[0028] Second, based on traditional ray tracing simulation data, the present invention adds measured data of the application area, effectively solving the problem of large errors between the measured and theoretical values of the receiving point power in complex scenarios, making up for the shortcomings of the traditional model, improving the prediction accuracy of the received power, and meeting the application needs of complex propagation environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flowchart of an embodiment of the ray tracing measurement correction method based on multimodal fusion of the present invention;

[0030] Figure 2 yes Figure 1 The multimodal fusion network structure diagram constructed in ;

[0031] Figure 3 This is a structural block diagram of an embodiment of a ray tracing measurement and correction system based on multimodal fusion according to the present invention;

[0032] Figure 4 This is a comparison chart of the prediction results of the present invention and the traditional algorithm for 19 line-of-sight points in the 5800MHz frequency band;

[0033] Figure 5 This is a comparison chart of the prediction results of the present invention and the traditional algorithm for 13 non-line-of-sight points in the 5800MHz frequency band. DETAILED DESCRIPTION

[0034] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0035] Example 1: A ray tracing measurement correction method based on multimodal fusion.

[0036] Reference Figure 1 The implementation steps of this example include the following:

[0037] Step 1: Obtain the convolutional layer training sample set R1, the self-attention layer training sample set R2, and the test sample set E.

[0038] 1.1) Obtain the convolutional layer training sample set R1:

[0039] The three feature vectors f1 of building height, f2 of transmitting point position and f3 of receiving point position are concatenated in the channel dimension to generate a three-dimensional tensor R1 containing multimodal information.

[0040] Each vector point of the building height feature vector f1 represents the building height of a spatial position. If the vector point does not contain a building, its value is 0;

[0041] The emission point position feature vector f2, the vector point corresponding to the emission point position and the surrounding vector points are set to 1, and the remaining vector points are set to 0;

[0042] The receiving point position feature vector f3, the vector point corresponding to the receiving point position and the surrounding vector points are set to 1, and the remaining vector points are set to 0.

[0043] In this example, the map of the predicted area is divided into 587×512 grids, and the size of each grid is 2m×2m.

[0044] 1.2) Get the attention layer training sample R2:

[0045] Select factors that affect radio wave propagation, including: transmission frequency, local coordinates of the receiving end, distance between the transmitting and receiving ends, ray tracing simulation power value, actual power value of the receiving point, and multipath information, where 0 represents line of sight and 1 represents non-line of sight;

[0046] Normalize the influence factors of the target point's radio wave characteristics and the measured values and influence factors of the auxiliary points. Then, concatenate the normalized M target point vector sets B with the N auxiliary point vector sets A to obtain M concatenated vectors C. Select K vectors from C as the attention layer training sample set R2. in:

[0047] The target point vector set B, the auxiliary point vector set A, and the splicing vector C are expressed as follows:

[0048] B={b m |1≤m≤M}, bm=[freqm,Losm,Xm,Ym,Zm,Distm,Pmsimulation,0],

[0049] A={a nm |1≤n≤N,1≤m≤M}, anm=[freqnm,Losnm,Xnm,Ynm,Znm,Distnm,Pnm simulation,Pnm measurement],

[0050] C={c m |1≤m≤M}, c m =[a 1m ,a 2m ,...,a nm ,b m ]

[0051] Where b m is the mth predicted target point vector; a nm is the nth auxiliary point vector corresponding to the mth target point; freq m 、Losm 、X m 、Y m , Z m 、Dist m 、P m仿真 Respectively represent the normalized frequency, multipath information, local coordinates, transmitter-receiver distance, and ray tracing simulation power value of the target point; freq nm 、Los nm 、X nm 、Y nm , Z nm 、Dist nm 、P nm仿真 、P nm实测 They respectively represent the normalized frequency, multipath information, local coordinates, transmitter-receiver distance, ray tracing simulation power value, and measured received power value of the nth auxiliary point corresponding to the mth target point.

[0052] 1.3) Get the test sample set E:

[0053] The remaining MK splicing vectors in the splicing vector C are used as the test sample set E.

[0054] In this example, it is assumed but not limited to M=636, N=3, and K=572.

[0055] Step 2: Construct a multimodal fusion network model H.

[0056] Reference Figure 2 , the implementation of this step includes the following:

[0057] 2.1) Establish a convolutional neural network module H1 consisting of two convolutional layers, three activation functions, two pooling layers, a flattening layer, and two fully connected layers, wherein the first convolutional layer, the first activation function, the first pooling layer, the second convolutional layer, the second activation function, the second pooling layer, the flattening layer, the first fully connected layer, the third activation function, and the second fully connected layer are connected in this order to process map information, wherein:

[0058] In the first convolutional layer, the convolution operation uses 32 3×3 convolution kernels;

[0059] In the second convolutional layer, the convolution operation uses 64 3×3 convolution kernels;

[0060] The first and second pooling layers both use maximum pooling, with the pooling window size set to 2×2 and the stride to 2. The number of neurons in the first fully connected layer is 256, and the number of neurons in the second fully connected layer is 1.

[0061] All three activation functions use ReLU.

[0062] 2.2) Establish a self-attention mechanism module H2 consisting of two self-attention layers, two normalization layers, two fully connected layers, and an activation function. The first self-attention layer, the first normalization layer, the second self-attention layer, the second normalization layer, the first fully connected layer, the activation function, and the second fully connected layer are connected in this order. The input and output of each self-attention layer are connected by an identical path across layers and input into the normalization layer for processing the radio wave impact factor information of the receiving point.

[0063] The number of neurons in the first fully connected layer is 32;

[0064] The number of neurons in the second fully connected layer is 1;

[0065] The activation function uses Sigmoid,

[0066] In the self-attention layer, the input feature matrix R2 is linearly transformed to generate the value matrix V=R2W V , key matrix K = R2W K and query matrix Q = R2W Q , the output of the self-attention layer is:

[0067] Output=softmax(QK T )·V

[0068] Where, They are weight matrices with three different parameters, K T is the transposed matrix of K; softmax is the layer normalization operation, d is the number of features of each target vector, in this example, d = 8;

[0069] 2.3) Establish a fusion module H3 consisting of a concatenation layer and a fully connected layer, which is used to fuse spatial information and radio wave impact factor information, where the number of neurons in the fully connected layer is 1;

[0070] 2.4) The convolutional neural network module H1 and the self-attention mechanism module H2 are connected in parallel, and then cascaded with the H3 module to form a multimodal fusion network model H.

[0071] Step 3: Use the training sample set to train the multimodal fusion prediction network model H.

[0072] 3.1) Set the maximum number of training times to T ≥ 500, and the prediction network model of the tth training is recorded as H t , whose weight parameter is set to w t , the bias parameter is set to b t , and let t = 1;

[0073] 3.2) R1 and R2 described in step 1 are used as the input of convolutional neural network module H1 and self-attention module H2 respectively. Their outputs are connected in parallel and passed through fusion module H3 to output K measured power prediction values. in is the predicted value of the e-th measured power;

[0074] 3.3) H calculated using the mean square error function MSE t Loss value L t and through L t Calculate H t The network parameter gradient is then used to optimize the weight w by the network parameter gradient. t and bias parameter b t Update to get the prediction network model H for this iteration t ,in:

[0075] Calculate H t The loss value L t The formula is: In the formula is the true value of the e-th measured power;

[0076] For the weight parameter w t and bias b t To update, the calculation formula is: P t ∈{ω t ,b t}where α is the learning rate and ε is a constant;

[0077] For v t Correction, v t =β1·v t-1 +(1-β1)·g t H t First-order moment estimates of network parameter gradients, is the gradient at iteration number t, β1 is the exponential decay rate of the first-order moment, is the t-th power of β1;

[0078] For s t Correction, H t The second-order moment estimate of the network parameter gradient, β2 is the exponential decay rate of the second-order moment, are the t-th power of β2 respectively;

[0079] 3.4) Repeat steps 3.2)-3.3) until the maximum number of iterations is reached or the loss value converges, and the trained prediction network model H is obtained. * ;

[0080] 3.5) Input the test sample set E into the trained prediction network model H * Perform forward propagation to obtain the predicted value of the received power of MK radio wave propagation measurements

[0081] Example 2: A ray tracing measurement and correction system based on multimodal fusion.

[0082] Reference Figure 3 , this example includes: data acquisition module 1, network module 2, training module 3 and prediction module 4.

[0083] Data acquisition module 1 is used to obtain ray tracing data and measured received power data, including radio wave propagation characteristic data such as operating frequency band, transmission point and reception point location, and normalize them to provide standardized input for training module 3;

[0084] Network module 2 is used to build a multimodal fusion network model, including a convolutional neural network module, a self-attention mechanism module, and a fusion module, which are used to process map information, radio wave propagation influencing factor information, and fuse spatial information with radio wave propagation characteristics respectively;

[0085] Training module 3 is used to train network module 2, including inputting the data acquired in data acquisition module 1 into network module 2, performing forward propagation, calculating the loss value, and iteratively updating the weights and parameters through backpropagation until the maximum number of iterations is reached or the loss value converges, thereby obtaining a trained prediction network model;

[0086] Prediction module 4 uses the prediction network model obtained by training module 3 to predict the measured received power of the unknown receiving point, thereby correcting the ray tracing results to improve the accuracy of radio wave propagation prediction.

[0087] Example 3: Electronic equipment.

[0088] The electronic device provided by the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor, communication interface, and memory communicate with each other via a communication bus. The processor can call logic instructions in the memory to execute a multimodal fusion ray tracing measurement correction method. The method includes: obtaining a training sample set and a test sample set; constructing a multimodal fusion network model comprising a convolutional neural network module, a self-attention mechanism module, and a fusion module; training the multimodal fusion prediction network model using the training sample set; and performing forward propagation using the test sample set as input to the trained prediction network model to obtain a predicted value of the measured received power of radio wave propagation.

[0089] The effects of the present invention can be further illustrated by the following simulation results.

[0090] 1. Simulation Experiment Conditions

[0091] The prediction model is run using a GPU of NVIDIA RTX 2080, and the prediction model adopts the prediction model H in Example 1. * , the loss function uses the mean square error function MSE, and the prediction area size is selected as 1174m×1024m.

[0092] 2. Simulation Experiment Content and Results

[0093] Simulation experiment 1, under the above conditions, the power of the line-of-sight type receiving point is predicted in the 5800MHz frequency band using the present invention and the traditional ray tracing method, and the prediction results are compared. Figure 4 shown.

[0094] from Figure 4 It can be seen that the average error between the results obtained by the prediction network model of the present invention and the measured values is only 3dB, while the average error between the traditional ray tracing method and the measured values is as high as 13dB. Compared with the traditional ray tracing method, the prediction accuracy of the present invention is improved by 10dB.

[0095] Simulation experiment 2, under the above conditions, the power of non-line-of-sight type receiving points in the 5800MHz frequency band is predicted using the present invention and the traditional ray tracing method, and the prediction results are compared. Figure 5 shown.

[0096] from Figure 5 It can be seen that the average error between the results obtained by the prediction network model of the present invention and the measured values is only 2.44dB, while the average error between the traditional ray tracing method and the measured values is as high as 17.44dB. Compared with the traditional ray tracing method, the prediction accuracy of the present invention is improved by 15dB.

[0097] The simulation results show that the prediction network model of the present invention effectively solves the problem of large errors between the measured and theoretical values of the receiving point power in complex scenarios, makes up for the shortcomings of the traditional model, improves the accuracy of the received power prediction, and meets the application requirements of complex propagation environments.

[0098] The above descriptions are merely a few specific examples of the present invention and do not constitute any limitation to the present invention. It is obvious that for professionals in this field, after understanding the content and principles of the present invention, they may make various modifications and changes in form and details without departing from the principles and structure of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

[0099] It should be noted that the step numbers in the specification and claims of the present invention are only for a clear description of the embodiments of the present invention and for ease of understanding, and the order of the step numbers is not limited.

Claims

1. A ray tracing measurement correction method based on multimodal fusion, characterized in that: include: Get the convolutional layer training sample sets respectively , self-attention layer training sample set , test sample set ,in Including measured data; The convolutional layer training sample set is obtained , is the building height feature vector , emission point position feature vector and the receiving point position feature vector These three eigenvectors are concatenated in the channel dimension to generate a three-dimensional tensor containing multimodal information. ; The self-attention layer training sample set is obtained , is to first normalize the influence factor of the target point's radio wave characteristics and the measured value of the auxiliary point and its influence factor, and then normalize the normalized A set of target point vectors Respectively Auxiliary point vector set Splice and get concatenation vectors , select in vectors as training sample sets , ; Acquiring a test sample set , is the splicing vector The remaining spliced vectors as the test sample set ; Construct a convolutional neural network module , self-attention mechanism module and fusion module Multimodal fusion network model , Module and Modules are connected in parallel with the Module cascade; Prediction network model for multimodal fusion using training sample sets Conduct training; The test sample set As a trained prediction network model The input is forward propagated to obtain The predicted value of the received power actually measured for radio wave propagation.

2. The method according to claim 1, characterized in that The convolutional layer training samples include dividing the map of the predicted area into grids, each grid is of size .

3. The method according to claim 1, characterized in that The parameter settings for each feature vector in the convolutional layer training sample include the following: The building height feature vector , where each vector point represents the height of a building at a spatial location. If the vector point does not contain a building, the value is 0; The emission point position feature vector , is to set the vector point corresponding to the launch point and its surrounding vector points to 1, and the remaining vector points to 0; The receiving point position feature vector , is to set the vector point corresponding to the receiving point and its surrounding vector points to 1, and the remaining vector points to 0.

4. The method according to claim 1, wherein: The target point vector set , auxiliary point vector set and concatenation vector , respectively expressed as follows: ， ， ， ； in, For the predicted target point vector, For the The target point corresponds to the Auxiliary point vectors.

5. The method according to claim 1, wherein The multimodal fusion network model The module structures and functions are as follows: The convolutional neural network module , used to process map information, which includes two sequentially connected convolution-activation function-pooling layers, a flattening layer, a first fully connected-activation layer, and a first fully connected layer; The self-attention mechanism module , used to process the radio wave influence factor information of the receiving point, which includes two self-attention-normalization layers, a second fully connected-activation function layer and a second fully connected layer connected in sequence, and the input and output of each self-attention layer are connected by an identical path across layers and input into the normalization layer; The fusion module , used to fuse spatial information and radio wave impact factor information, which includes the splicing layer and the third full connection layer; The output of the first fully connected layer is connected in parallel with the output of the second fully connected layer and then connected to the splicing layer to form a multimodal fusion network model. .

6. The method according to claim 5, characterized in that: The activation function in the first fully connected activation layer is ReLU, and the number of fully connected neurons is 256; The activation function in the second fully connected-activated layer is Sigmoid, and the number of fully connected neurons is 32; The number of neurons in the first fully connected layer is 1; The number of neurons in the second fully connected layer is 1.

7. The method according to claim 1, characterized in that The prediction network model of multimodal fusion using the training sample set Training is performed, which is implemented as follows: 8a) Set the maximum number of training times to ;No. The prediction network model trained this time is recorded as , whose weight parameter is , the bias parameter is , and order ; 8b) and As convolutional neural network modules and self-attention module The output is connected in parallel and sent to the fusion module ,get Measured power prediction value ; 8c) Calculated using the mean square error function MSE Loss value and through calculate The network parameter gradient is then used to optimize the weights by the network parameter gradient. and bias parameters Update to get the prediction network model of this iteration ; 8d) Repeat steps 8b)-8c) until the maximum number of iterations is reached or the loss value converges, obtaining a trained prediction network model.

8. A ray tracing measurement and correction system based on multimodal fusion, comprising: The data acquisition module is used to obtain ray tracing data and measured received power data, including radio wave propagation characteristic data such as operating frequency band, emission point and receiving point position, and normalize them; the data acquisition module obtains the convolution layer training sample set respectively , self-attention layer training sample set , test sample set ,in Including measured data; The convolutional layer training sample set is obtained , is the building height feature vector , emission point position feature vector and the receiving point position feature vector These three eigenvectors are concatenated in the channel dimension to generate a three-dimensional tensor containing multimodal information. ; Get the self-attention layer training sample set , is to first normalize the influence factor of the target point's radio wave characteristics and the measured value of the auxiliary point and its influence factor, and then normalize the normalized A set of target point vectors Respectively Auxiliary point vector set Splice and get concatenation vectors , select in vectors as training sample sets , ; Get the test sample set , is the splicing vector The remaining spliced vectors as the test sample set ; The network module is used to build a multimodal fusion network model, including a convolutional neural network module, a self-attention mechanism module, and a fusion module, which are used to process map information, radio wave influencing factor information, and integrate spatial information with radio wave propagation characteristics respectively; The training module is used to train the multimodal fusion network model, including inputting data into the network, performing forward propagation, calculating the loss value, and iteratively updating the weights and parameters through backpropagation until the maximum number of iterations is reached or the loss value converges, thereby obtaining the trained prediction network model; The prediction module is used to predict the measured received power of unknown receiving points using the trained multimodal fusion network model, and to correct the ray tracing results to improve accuracy.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the ray tracing measurement correction method for multimodal fusion according to any one of claims 1 to 7 is implemented.

Images