Deep Learning-Based Sensor-Integrated Network Beamforming Prediction Method and Related Equipment

Abstract

This application provides a deep learning-based beamforming prediction method and related equipment for integrated sensing and communication networks, belonging to the field of mobile communication technology. The method includes: a base station transmitting an ISAC signal in full-duplex mode and receiving echoes in each time slot; stacking historical echo signals into a three-dimensional tensor and inputting it into a deep neural network; the network uses CNN to extract spatial features, LSTM to capture time dependencies, and an attention mechanism for adaptive weighting, directly outputting the base station transmit beamforming matrix and RIS phase shift matrix for the next time slot. Network training employs an unconstrained optimization loss function based on a penalty function, integrating the system's total rate maximization objective with constraints such as power, unity modulus, and serial interference cancellation. This application bypasses the error propagation of channel estimation in traditional two-stage methods, reduces computational complexity, and significantly improves the total achievable communication rate of vehicles, making it suitable for intelligent transportation systems in high-speed mobile scenarios.

Description

Technical Field

[0001] This application relates to the field of mobile communication technology, and in particular to a beamforming prediction method and related equipment for a deep learning-based integrated sensing network. Background Technology

[0002] With the continuous evolution of mobile communication technology, Vehicle-to-Infrastructure (V2I), as a key component of intelligent transportation systems, plays a crucial role in providing vehicles with real-time traffic information, signal control status, and collaborative safety alerts. In high-speed mobility scenarios, V2I systems face increasingly stringent requirements for high throughput, low latency, and massive concurrent connections.

[0003] Millimeter wave technology, with its wide spectrum resources and high-speed data transmission capabilities, has been widely used in fields such as intelligent transportation. However, in practical deployments, millimeter waves face key challenges such as severe path loss, susceptibility to obstruction by buildings or vehicles, and limited effective coverage distance, making it difficult to provide a stable link continuously in complex and dynamic V2I environments.

[0004] To overcome performance bottlenecks caused by transmission link obstructions, Reconfigurable Smart Surfaces (RIS) have been introduced to assist communication. RIS consists of numerous low-cost, passive tunable units that can intelligently reshape the propagation environment by dynamically controlling the reflection phase of the incident signal, thereby improving the reliability of the communication link. On the other hand, to further improve spectrum utilization efficiency, Non-Orthogonal Multiple Access (NOMA) technology utilizes power domain multiplexing and serial interference cancellation (SIC) mechanisms to enable multiple vehicles to share the same time-frequency resources.

[0005] Communication-Sensing Integration (ISAC) technology leverages its inherent sensing capabilities to assist in beamforming design. By utilizing echo signals for sensing, motion parameters can be estimated to reconstruct channel state information (CSI), thereby reducing the overhead of pilot-based channel estimation. Deep learning is gradually emerging in the field of signal processing, demonstrating superior performance in solving complex optimization problems due to its powerful feature extraction and nonlinear mapping capabilities.

[0006] Currently, existing research has explored RIS-assisted ISAC systems or RIS-assisted V2I systems, but few studies have organically integrated RIS, ISAC, and NOMA for joint optimization in V2I scenarios. Most existing beamforming prediction technologies employ a two-stage algorithm: first, estimating the CSI using the sensed signal, and then optimizing the beamforming matrix based on the estimated CSI. This staged processing leads to estimation errors in the first stage affecting beamforming design in the second stage, thus reducing system performance. Furthermore, traditional analytical optimization algorithms suffer from high computational complexity and slow convergence due to the high-dimensional non-convexity of the problem, making it difficult to meet the millisecond-level decision-making latency requirements of V2I systems.

[0007] Therefore, how to design a beamforming prediction method that can overcome the above-mentioned defects has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0008] The main objective of this application is to propose a deep learning-based beamforming prediction method, system, electronic device, storage medium, and program product for integrated sensing networks, in order to solve the following technical problems: 1) In V2I systems, how to use RIS to overcome the link interruption problem caused by building obstruction and improve spectrum efficiency in conjunction with ISAC technology; 2) How to avoid performance loss caused by CSI estimation error in traditional two-stage beamforming prediction methods; 3) How to reduce the high computational complexity of jointly optimizing the base station transmit beamforming matrix and RIS phase shift matrix to meet the real-time requirements of V2I systems; 4) How to maximize the total achievable communication rate of all vehicles under the premise of satisfying transmit power constraints, RIS phase shift unity mode constraints, and SIC decoding constraints.

[0009] To achieve the above objectives, one aspect of this application proposes a deep learning-based beamforming prediction method for integrated sensing and communication networks, applied to a vehicle-to-infrastructure (V2I) system assisted by a smart reflector (RIS) and communication sensing network, comprising the following steps: In each time slot, the base station adopts a full-duplex mechanism and sends ISAC signals to multiple mobile vehicles in a non-orthogonal multiple access (NOMA) manner with the assistance of RIS, and receives echo signals reflected by the vehicles and transmitted back via RIS. The echo signals received in the current time slot and several historical time slots are stacked into a three-dimensional tensor, which is then used as the input to a deep neural network (DNN). The DNN is used to extract the spatiotemporal features of the input echo signal tensor, and the base station transmit beamforming matrix and RIS phase shift matrix of the next time slot are directly output. The base station transmit beamforming matrix and RIS phase shift matrix output by the DNN are applied to the communication and sensing transmission in the next time slot.

[0010] In some embodiments, the deep neural network (DNN) includes: The CNN module is used to extract the spatial features of the echo signal for each historical time slot; The LSTM module is used to capture the temporal dependencies of the feature sequences processed by the CNN module and output the hidden state sequence. The temporal attention module is used to adaptively weight the hidden state sequence output by the LSTM module to obtain the weighted global feature vector. The fully connected module is used to generate the RIS phase shift matrix and the base station transmit beamforming matrix based on the weighted global feature vector.

[0011] In some embodiments, the temporal attention module concatenates the hidden state of the LSTM at the final time step with the hidden state at each time step, calculates the attention weights using trainable parameters, and performs a weighted summation of the hidden state sequence to highlight the time steps containing key information.

[0012] In some embodiments, the output layer of the DNN includes a power constraint layer for power normalization of the real and imaginary parts of the base station transmit beamforming matrix output by the fully connected layer, so as to meet the total transmit power constraint of the base station.

[0013] In some embodiments, the RIS phase shift matrix is ​​directly output by the DNN as a phase shift vector and converted into a phase shift matrix that satisfies the unity modulus constraint through a complex exponential form.

[0014] In some embodiments, the training process of the DNN employs an unconstrained optimization loss function based on a penalty function, which integrates the goal of maximizing the total achievable communication rate of the system, as well as transmit power constraints, RIS phase shift unity mode constraints, and serial interference cancellation SIC decoding constraints.

[0015] To achieve the above objectives, another aspect of this application proposes a deep learning-based integrated sensing network beamforming prediction system, applied to a smart reflector RIS-assisted communication-sensing integrated ISAC vehicle-to-infrastructure (V2I) system, comprising: Signal transceiver module: configured at the base station, used to send ISAC signals to multiple mobile vehicles in each time slot using a full-duplex mechanism and with the assistance of RIS in a non-orthogonal multiple access (NOMA) manner, and to receive echo signals reflected by the vehicles and transmitted back via RIS. Data preprocessing module: used to stack the echo signals received in the current time slot and several historical time slots into a three-dimensional tensor; Deep Neural Network (DNN) module: used to extract spatiotemporal features from the input echo signal tensor and directly output the base station transmit beamforming matrix and RIS phase shift matrix for the next time slot; Beamforming application module: used to apply the base station transmit beamforming matrix and RIS phase shift matrix output by the DNN module to the communication and sensing transmission in the next time slot.

[0016] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0017] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0018] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0019] Compared with the prior art, this application has the following advantages: 1) System architecture innovation: This application proposes a V2I system framework based on ISAC with RIS assistance, which organically integrates RIS, ISAC and NOMA technologies, providing a new solution for the construction of wireless networks for future intelligent transportation systems, and effectively overcoming the problems of building obstruction and spectrum resource shortage.

[0020] 2) End-to-end prediction paradigm: This application proposes an end-to-end beamforming prediction method based on deep learning, which directly predicts the beamforming matrix of the next time slot from historical echo signals. This completely bypasses the two-stage process of first estimating CSI and then optimizing in the traditional method, avoids the propagation of CSI estimation error, and simplifies the prediction process.

[0021] 3) Dedicated deep network structure: This application designs a composite deep neural network of "CNN+LSTM+temporal attention", which can fully extract the spatiotemporal features of the echo signal. The attention mechanism can adaptively weight the information of different historical time slots, which significantly improves the accuracy of beamforming prediction.

[0022] 4) Efficient optimization solution: This application uses the penalty function method to transform the original non-convex constraint optimization problem into an unconstrained optimization problem, and designs a matching weighted loss function. It is trained by Monte Carlo approximation, which greatly reduces the computational complexity compared with traditional analytical algorithms and can meet the real-time requirements of V2I systems.

[0023] 5) Significantly improved performance: Simulation results show that the end-to-end beamforming prediction scheme proposed in this application has significant improvements in communication and speed for all vehicles compared with the convolutional LSTM scheme based on historical channels and the random phase shift scheme. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the transmission protocol in the embodiments of this application; Figure 2 This is a schematic diagram of a deep learning-based beamforming prediction framework in a V2I system based on ISAC with RIS assistance, as described in the embodiments of this application. Figure 3 This is a schematic diagram of the structure of an end-to-end deep neural network in an embodiment of this application; Figure 4 This is a schematic diagram illustrating the convergence performance of the algorithm under different learning rate settings in the embodiments of this application; Figure 5 This is a comparative diagram showing the communication capabilities and speeds of all vehicles under different schemes in the embodiments of this application; Figure 6 This is a schematic diagram illustrating the changes in communication capabilities and speeds of all vehicles under different historical time slots in this application embodiment; Figure 7 A flowchart of a deep learning-based beamforming prediction method for integrated sensing networks provided in this application embodiment. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0027] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0028] 1) Reconfigurable Intelligent Surface (RIS) is a cutting-edge wireless communication technology based on programmable artificial electromagnetic materials. It can dynamically control the propagation environment of electromagnetic waves, thereby significantly improving the coverage, capacity and energy efficiency of communication systems.

[0029] 2) Integrated Sensing and Communication (ISAC) is a technology that integrates communication and sensing functions into the same wireless system, using the same spectrum, hardware and signal resources to achieve synergistic effects between information transmission and environmental perception.

[0030] 3) Vehicle-to-Infrastructure (V2I) is an important component of vehicle-to-everything (V2X) technology. It refers to the two-way information exchange between vehicles and road infrastructure through wireless communication to improve traffic safety, efficiency and intelligence.

[0031] The significant potential of ISAC in improving spectral efficiency makes it a promising technology for B5G and 6G wireless networks. Extensive research has been conducted in industry and academia regarding RIS-assisted V2I systems based on ISAC technology.

[0032] The first existing technical solution proposes a hybrid RIS-enhanced ISAC vehicle-to-everything (V2X) system. In the considered system, the authors designed a deep reinforcement learning framework to jointly optimize the roadside unit (RSU) transmit beamforming matrix, V2I / V2V spectrum reuse scheme, and hybrid RIS reflection weights to address the V2I security and rate maximization issues, while ensuring V2V service quality and sensing performance. The second existing technical solution considers a RIS-assisted ISAC V2I system and proposes two deep learning-based system performance optimization algorithms. Both algorithms jointly optimize the active beamforming of the base station and the phase shift matrix of the RIS to maximize the achievable communication rate of the moving vehicle, while satisfying transmit power and phase shift matrix mode constraints. The third existing technical solution studies a millimeter-wave V2I system supported by NOMA technology assisted by a simultaneous transmission and reflection smart surface (STAR-RIS). It designs the STAR-RIS phase shift through deep unsupervised learning and combines analytical methods to optimize base station beamforming and power allocation, and studies the system's communication performance.

[0033] However, existing technical solutions have the following drawbacks: 1) Current research in this field mainly focuses on RIS-assisted ISAC systems or RIS-assisted V2I systems, but there is relatively little research exploring the performance optimization of V2I systems enabled by combining RIS and ISAC technologies. Considering the potential advantages of ISAC in solving spectrum resource limitations and the outstanding performance of RIS in dealing with building obstruction, research on V2I systems combining RIS and ISAC is of great significance for the future construction of wireless networks.

[0034] 2) Current research on joint beamforming prediction for RIS-assisted V2I or ISAC systems mostly employs a two-stage algorithm. In the first stage, channel state information is estimated using the reflected sensor signals received by the base station. In the second stage, the channel is reconstructed based on the estimated channel state information, and the beamforming matrix is ​​jointly optimized. This staged processing will cause the estimation error in the first stage to affect the beamforming design in the second stage, thereby leading to a decrease in the reachability and speed of the communication equipment.

[0035] 3) Currently, beamforming joint optimization in RIS-assisted V2I systems mostly employs traditional analytical optimization algorithms. However, due to the strong coupling between the RIS phase shift matrix and the base station transmit beam matrix, coupled with unity mode constraints and power limitations, the problem exhibits high-dimensional non-convex characteristics. Traditional analytical optimization algorithms have high computational complexity and slow convergence speed, making it difficult to meet the millimeter-level decision-making requirements of V2I systems.

[0036] In view of this, in order to overcome the problem of building occlusion in V2I systems combined with ISAC technology, this application designs a RIS-assisted ISAC-based V2I system.

[0037] To avoid the problems of inaccurate beamforming optimization due to channel estimation error in two-stage beamforming prediction algorithms and the high computational complexity and slow convergence of traditional analytical algorithms, this application proposes an end-to-end beamforming prediction method based on deep learning. This method establishes a direct relationship between the received echo and the beamforming matrix through a data-driven approach, thereby bypassing the need for the base station to obtain channel state information, simplifying the beamforming prediction process, and reducing computational overhead.

[0038] Based on the proposed system, in order to maximize the overall achievable communication rate of the vehicle, this application designs a method for joint optimization of the base station transmit beamforming matrix and RIS phase shift matrix, which satisfies the transmit power and phase shift matrix mode constraints.

[0039] like Figure 7 As shown in the figure, this application provides a beamforming prediction method for a deep learning-based integrated sensory network, including the following steps: Step S1: In each time slot, the base station adopts a full-duplex mechanism, sends ISAC signals to multiple mobile vehicles in a non-orthogonal multiple access (NOMA) manner with the assistance of RIS, and receives echo signals reflected by the vehicles and transmitted back via RIS. Step S2: Stack the echo signals received in the current time slot and several historical time slots into a three-dimensional tensor, which is used as the input to a deep neural network (DNN); Step S3: Use the DNN to extract spatiotemporal features from the input echo signal tensor and directly output the base station transmit beamforming matrix and RIS phase shift matrix for the next time slot; wherein, the training process of the DNN adopts an unconstrained optimization loss function based on a penalty function, which integrates the goal of maximizing the total achievable communication rate of the system as well as transmit power constraints, RIS phase shift unity mode constraints and serial interference cancellation (SIC) decoding constraints; Step S4: Apply the base station transmit beamforming matrix and RIS phase shift matrix output by the DNN to the communication and sensing transmission in the next time slot.

[0040] The solutions of the embodiments of this application will be described in detail below with reference to the accompanying drawings and specific application examples: (1) System Model In the RIS-assisted ISAC-based V2I system considered in this embodiment, K A single-antenna vehicle moves along a road area 20 meters long and 10 meters wide, centered on the RIS, parallel to the direction of the base station antenna. k The car in n The coordinates of each time slot are determined by the coordinates of the previous time slot and the average speed, which is between 8 and 10 meters per second, meaning the average vehicle speed is approximately 30 km / h, and the RIS is about 12 meters from the base station. N The RIS of a single reflective element can be modeled as a uniform planar array, while the base station is... M A uniform linear array is used. Due to building obstruction, direct communication between the base station and the vehicle is impossible. Therefore, a RIS (Radio Relay System) is deployed between the base station and the vehicle to establish a wireless information transmission link.

[0041] 1.1) Transmission Protocol To reduce the pilot overhead required for channel estimation and obtain accurate beamforming matrices based on sensing information in high-dynamic scenarios, this application proposes a parallel transmission protocol based on ISAC, supporting simultaneous communication, sensing, and computation functions. Its structure is shown below. Figure 1 As shown. This embodiment assumes that self-interference introduced by full-duplex technology can be suppressed to the thermal noise level through effective radio frequency and digital self-interference cancellation techniques. Therefore, the impact of self-interference on the reception of sensing signals can be ignored in this study. In the proposed protocol, the data stream is divided into multiple time slots. In each time slot, the base station transmits an ISAC signal, simultaneously completing downlink communication and sensing tasks. Specifically, the protocol consists of two phases: ISAC signal transmission and echo reception, and ISAC signal processing. (The text continues with a description of the second phase, which is not directly related to the previous paragraph.) nTaking a time slot as an example, in the first stage, the base station uses the optimal base station transmit beamforming matrix and RIS phase shift matrix predicted in the previous time slot to transmit the ISAC signal and receive the echo. Due to the time delay in the round-trip propagation of the ISAC signal, the reception of the echo will lag slightly relative to the start time of each time slot frame header. In the second stage, the base station predicts the beamforming matrix for the next time slot based on echo samples from different vehicles in the current time slot and historical time slots. Through this transmission method, the base station can directly perform beamforming design without explicitly obtaining the CSI.

[0042] 1.2) Channel Model Assume all wireless transmission channels are affected by large-scale path loss and small-scale fading, and experience independent Rician fading. The base station and RIS, and RIS and the... k The channels between each vehicle are modeled as follows: , (1) in, and These are respectively RIS and base station and RIS and the first k Path loss model between vehicles. and Represent the Ricean fading components, which respectively satisfy: , (2) in, This represents the Rice coefficient of the corresponding channel, with a value set to 10 dB. and This represents the line-of-sight (Los) component of the corresponding channel, where: (3) (4) Represents the array steering vector. and These represent the azimuth and elevation angles of the line-of-sight (Los) path associated with RIS, respectively. It is the departure angle associated with the base station. and These represent the non-line-of-sight (NLoS) components of the corresponding channels, and each element is an independent and identically distributed circularly symmetric complex Gaussian random variable with zero mean and 1 variance.

[0043] 1.3) Signal Model In the n In each time slot, the base station sends... K Vehicle transmits multi-beam ISAC signal vector ,in, To be assigned to the k The vehicle's repeater signal satisfies .set up This represents the transmit beamforming matrix of the base station, where, For the first k Vehicle-specific beamforming vectors, Let this be the transmission power allocated to this vehicle. Then, the signal transmitted by the base station can be represented as: (5) in satisfy , This represents the maximum transmit power of the base station. Let... Here is the RIS phase shift matrix in the downlink, where, and , respectively representing the first n The phase shift and amplitude coefficient of each reflective element. This embodiment focuses on the impact of the RIS phase shift on system performance, therefore, it sets... The value is 1. To simplify the symbolic representation, the slot index will only be retained when necessary. n Based on the above modeling, the first k The signal received by the vehicle can be written as: (6) in, Indicates the first k Additive white Gaussian noise (AWGN) received by each vehicle.

[0044] With the assistance of RIS, the base station employs a non-orthogonal multiple access strategy to transmit communication signals to the vehicle. The definition of the base station to the vehicle under RIS assistance is as follows: k The equivalent channel for a vehicle is Vehicles are sorted according to their equivalent channel gain, categorized into "strong users" and "weak users," and the channel gains are arranged in descending order. To eliminate interference between vehicles, a serial interference cancellation (SIC) decoding strategy is used for strong users, while weak users treat the strong user signals as noise when detecting their own data. At this point, the... k The achievable throughput of a vehicle can be expressed as: (7) in, This represents noise power. When... ,Right now At that time, the first k Before a vehicle can decode its own signal, it needs to decode the signal of the vehicle that is first decoded. d The vehicle signal, at this time the corresponding throughput is: (8) To ensure the successful implementation of serial interference cancellation technology, the decoding rate must meet the following requirements. .at this time K The total achievable throughput of all vehicles is expressed as: .

[0045] Since the base station can both communicate with and sense vehicles, this paper assumes that the uplink channel matrix is ​​the transpose of the downlink channel matrix. (RIS Uplink Phase Shift Matrix) Obtained through offline calibration, the echo signal received by the base station from the vehicle can therefore be expressed as: (9) in The ISAC probe signal consists of the first L downlink symbols. This is the additive white Gaussian noise received by the base station.

[0046] 1.4) Optimization problem Having completed the above definition, this application aims to improve the base station transmit beamforming matrix in a RIS-assisted, ISAC-based V2I system by jointly optimizing the base station transmit beamforming matrix. With RIS phase shift matrix To maximize the total achievable communication rate. The mathematical expression for the optimization problem is (P1): (10) Constraint C1 represents the unit modulus limit for the RIS phase shift. Constraint C2 represents the total transmit power limit for the base station. Constraint C3 represents the SINR received by each vehicle not being lower than the threshold. To ensure communication performance, constraint C4 is used to ensure the smooth progress of the SIC decoding process.

[0047] (2) Beamforming Joint Prediction Method In the optimization problem (P1) under consideration, the base station transmit beamforming matrix to be optimized is coupled with the RIS phase shift matrix, and C1, C3, and C4 are non-convex constraints. Furthermore, the high-speed movement of vehicles causes the channel state information to change rapidly over time, making it difficult for traditional real-time channel estimation methods to effectively obtain accurate CSI, and thus difficult to provide reliable spatial direction information and power allocation references for beamforming. Therefore, even with traditional optimization algorithms, the obtained optimization parameters may not meet the actual bounds of the system's achievable communication and speed. To effectively address these issues, this embodiment proposes an end-to-end beamforming prediction method based on deep learning. A dataset is constructed by modeling the base station received echo and related channel matrices. A carefully designed neural network architecture and weighted loss function are used, and the network is trained offline based on the dataset to fully learn the complex mapping relationship between input and output. This bypasses the complex process of traditional channel estimation and numerical optimization, achieving joint prediction of optimization parameters and providing a theoretical basis for beamforming design in intelligent transportation systems. In this section, we first introduce a deep learning-based beamforming prediction framework, then introduce a network based on historical echoes, consisting of a convolutional neural network (CNN), a long short-term attention mechanism (LSTM), and a temporal attention mechanism. Subsequently, we propose a beamforming prediction algorithm driven by this network.

[0048] 2.1) Deep Learning-Based Beamforming Prediction Framework Deep learning can effectively handle unconstrained optimization problems, thus addressing high-dimensional non-convex problems in beamforming joint design with lower computational complexity. Since the optimization problem (P1) contains non-convex constraints, a penalty function method can be used to transform the problem into an equivalent unconstrained problem (P2): (11) The penalty factor is used to control the weight of the penalty term. The RIS phase shift matrix is ​​modeled in complex exponential form to satisfy the non-convex constraint C1 in the optimization problem (P1). Since it is difficult to derive a closed-form optimal solution for the optimization problem (P2), this embodiment uses a data-driven approach to asymptotically approximate the optimal solution and leverages the strong feature extraction capabilities of deep neural networks (DNNs) to obtain an approximate optimal solution for the optimization problem (P2). Based on the above, a beamforming joint prediction framework is constructed as follows: Figure 2 As shown, it mainly consists of two stages: In the first stage, the original optimization problem is transformed into an unconstrained problem. That is, (P2), expected For the channel matrix The results were obtained. In the second stage, the Monte Carlo method was used to... Find an approximation: (12) in, This indicates that the DNN learns from the available input. To the expected output The mapping function, For the network parameters of the DNN, Indicates the first i The index of each training sample. When the number of training samples... When the value is sufficiently large, the approximation in formula (12) holds true. Based on this, the training loss of the DNN can be set as: (13) Ultimately through continuous updates By minimizing this loss function, the optimal base station transmit beamforming matrix and RIS phase shift matrix can be obtained from the DNN training process: (14) This represents the optimal network parameters obtained after training convergence.

[0049] 2.2) Network Structures for DNN Training Since echo sensing signals are essentially time-series data, accurately characterizing their temporal correlation is crucial for predicting the transmitted beamforming matrix and the RIS phase shift matrix. Therefore, this embodiment constructs a deep network structure composed of a convolutional neural network (CNN), a long short-term memory network (LSTM), and an attention module to simultaneously mine the temporal and spatial features of the echo signal, thereby achieving the prediction of the beamforming matrix. The overall process can be divided into three parts: data preprocessing, spatiotemporal feature extraction, and predicted beamforming matrix output, as follows... Figure 3 As shown.

[0050] Data preprocessing: in the first n -1 time slot, the ISAC base station receives the echo sensing signals reflected back from all vehicles based on formula (9). and will recently The signals from each historical time slot are stacked into a three-dimensional tensor. To extract features from the real and imaginary parts of the input signal separately, the complex-valued input is split into two parts: (15) in, It is a matrix rearrangement function: 2 corresponds to the two input channels of the CNN module.

[0051] Spatiotemporal feature extraction: The spatiotemporal feature extraction process mainly includes three modules: a CNN module for spatial feature extraction, an LSTM module for temporal dependency modeling, and a temporal attention module for adaptive feature weighting.

[0052] 1) CNN Module: This embodiment employs a CNN module to extract spatial features from each historical time slot echo signal. This module consists of two sub-CNNs to enhance feature extraction capabilities and improve the stability of feature representation. Each sub-CNN has the same network structure, including a convolutional layer, a batch normalization (BN) layer, and a ReLU activation layer. The convolutional layer uses a two-dimensional convolutional kernel to generate feature maps, and then a BN layer is added after each convolutional operation to accelerate training and alleviate problems such as vanishing or exploding gradients. Finally, nonlinearity is introduced through the ReLU activation function, enabling the network to learn more complex features and patterns.

[0053] 2) LSTM module: After completing the convolution operation, the resulting tensor... Rearranged as With an input format that conforms to the requirements of LSTM, namely a series of feature vectors, where To output the number of channels for the CNN, This represents the spatial dimension. After rearrangement, we can obtain... ,in The LSTM module can capture simultaneously The short-term and long-term dependencies in the data. Each LSTM unit uses the spliced ​​echo feature vector sequence. As input, output the corresponding hidden state vector. .

[0054] 3) Temporal Attention Module: The final hidden state output by the LSTM Contains from input Extracted time-related information. However, using only a single vector... Representing all temporal correlations may lead to information loss. Therefore, this invention utilizes LSTM hidden state sequences... An attention mechanism is introduced, which assigns greater weight to time steps containing more effective information. Specifically, the final hidden state... and Concatenation forms vector Then, attention weights are calculated through the attention module. , is represented as: (16) in and These are the trainable parameters of the attention mechanism. The weighted hidden state vector, i.e., the output of the attention mechanism module, can be written as: (17) Predicted beamforming matrix output: This module mainly consists of two layers: First, a fully-connected (FC) layer, which maps the extracted nonlinear spatiotemporal features to the RIS phase offset vector and the real and imaginary parts of the base station transmit beamforming matrix. Second, a power constraint layer, which performs power normalization to meet the total transmit power constraint.

[0055] 1) Fully Connected Layer: This embodiment designs three FC modules to generate the base station transmit beamforming matrix. and RIS phase shift matrix Optimize variables It depends only on the baseband equivalent channel, while Besides being affected by CSI, it is also affected by the phase shift matrix. Inspired by this characteristic, this embodiment uses the output of the attention module as the input of the FC-I module to directly generate the predicted RIS phase shift vector. Subsequently, the phase shift vector is transformed into the predicted RIS phase shift matrix using complex exponential form: (18) The above operations can bypass the constant-mode constraint in constraint C1. On the other hand, the output of the attention module is connected to... By concatenating these features, a high-dimensional feature vector containing both channel characteristics and RIS phase shift matrix information can be obtained, which can then be used to generate the required base station transmit beamforming matrix. .

[0056] 2) Power Constraint Layer: The vectors output by the FC-I and FC-II modules are respectively The real and imaginary parts of the matrix are not independent. Therefore, a dedicated power constraint layer is designed to normalize the transmit power, resulting in the final complex-valued matrix, which can be expressed as: (19) Through the above modules, the overall mapping function of the DNN can be expressed as: ,in This is the set of all trainable parameters of the network, including the parameters of CNN, LSTM, attention module, and each FC layer.

[0057] 2.3) DNN-based beamforming prediction algorithm In this section, based on the proposed deep neural network structure, a beamforming joint prediction algorithm is proposed, which consists of three stages: offline training, offline validation, and online prediction.

[0058] Offline training: Given a training set ,in The number of training samples, For the first i One training sample. Based on Figure 2 The loss function of the constructed framework can be defined as: (20) in For the first k The column vector corresponding to each vehicle. The maximum operation in the formula... This can be achieved using the ReLU function, i.e. Based on the loss function (20), the back propagation algorithm (BPA) is used to calculate the gradients of all trainable parameters, and the adaptive moment estimation algorithm (Adam) is used to update the parameters under an adaptive learning rate, thereby accelerating convergence and improving training stability.

[0059] Offline validation: A validation set is obtained by partitioning the entire dataset. ,in To verify the sample size, For the first i A validation set of samples is used. The trained DNN is then used to calculate the loss value on the validation set samples to evaluate the model's performance on unseen data and to assist in hyperparameter tuning and overfitting detection.

[0060] Online prediction: In the online phase, given historical spliced ​​echo signals... By inputting it into the trained DNN, the predicted base station transmit beamforming matrix can be directly output. With RIS phase shift matrix .

[0061] (3) Simulation results In this embodiment, simulation analysis verifies the effectiveness of the proposed deep learning-based end-to-end beamforming prediction method in improving the overall achievable throughput of the system, and compares it with the historical channel-based convolutional LSTM network scheme and the random phase shift scheme. In the historical channel-based convolutional LSTM network scheme, the real and imaginary parts of the historical channel matrix are used as input, spatial features are extracted through convolutional layers, and the LSTM module is used to capture temporal dependencies, thus directly predicting the beamforming matrix of the next time slot using the trained network without explicit channel tracking. To adapt to the V2I system based on ISAC with RIS assistance in this invention, the network output is extended to simultaneously generate the base station's transmit beamforming matrix and the RIS phase shift matrix. The dataset and training strategy are completely consistent with the proposed network to ensure fair comparison. In the random phase shift scheme, the RIS phase shift starts from... The phase shift of each reflector is not optimized, and the transmit beamforming matrix of the base station is optimized by the zero-forcing algorithm.

[0062] The simulation parameters are set as follows: a total of 4 vehicles, and each base station is equipped with 16 transmit and receive antennas. The RIS (Radio Resonance Array) is a uniform planar array consisting of 64 elements. The positions of the base station and the RIS are fixed, with three-dimensional Cartesian coordinates of (0m, 10m, 10m) and (0m, 0m, 5m), respectively. Considering that the channel between the RIS and the vehicles changes with the vehicle's position, multiple random samplings are performed on the vehicle positions to generate a large number of RIS-vehicle link channel samples, providing sufficient data for training the proposed network. The number of historical time slots is 4, and the Adam optimizer is used during offline training with a fixed learning rate of 0.0003. Gradients are calculated using 32 training samples for each parameter update. The dataset size is 2000 samples, divided into training, validation, and test sets in an 8:1:1 ratio. All simulation results are the average values ​​obtained from 200 independent runs on the test set.

[0063] Figure 4 This paper demonstrates the convergence performance of the proposed deep learning-based end-to-end beamforming prediction scheme in an ISAC-based V2I system with RIS assistance under different learning rate settings. It can be observed that when the learning rate decreases from 0.1 to 0.001, the average reachability rate increases by approximately 37.9%, but the curve exhibits strong oscillations after about 50 training epochs, indicating poor convergence stability, which is caused by excessively large update step sizes. When the learning rate is further reduced to 0.0001, the oscillations weaken, but the average reachability rate decreases significantly, resulting in slower convergence and limited optimization capabilities. In contrast, a learning rate of 0.0003 achieves a good trade-off between convergence speed, stability, and final performance: maintaining smooth convergence while avoiding the severe oscillations at high learning rate settings. Therefore, this application ultimately selects 0.0003 as the optimal learning rate during training to ensure stable convergence and reliable performance on different datasets.

[0064] Figure 5This paper demonstrates that when the proposed deep learning-based end-to-end beamforming prediction scheme is used in an ISAC-based V2I system with RIS assistance, all vehicles can achieve communication and speed. The results are compared with a convolutional LSTM network scheme based on historical channels and a random phase-shift scheme. Simulation results show that the proposed end-to-end beamforming prediction scheme significantly outperforms the other two schemes in terms of both communication and speed. This is because the phase-shift optimized RIS provides additional reflection links, enhancing downlink communication capabilities. Furthermore, the attention module in the proposed network adaptively weights the importance of different time steps or features, more effectively capturing temporal correlations and key information, thereby improving beamforming prediction accuracy. Therefore, accurate end-to-end beamforming prediction is crucial for improving the communication and speed capabilities of all vehicles.

[0065] Figure 6 This demonstrates that when the proposed deep learning-based end-to-end beamforming prediction scheme is employed in an ISAC-based V2I system with RIS assistance, all vehicles can achieve communication and rate increases with the number of historical time slots. The results of the changes were presented and compared with the convolutional LSTM network scheme based on historical channels and the random phase-shift scheme. Simulation results show that the proposed end-to-end beamforming prediction scheme significantly outperforms the other two schemes in terms of both communication capability and speed for all vehicles. This demonstrates that the proposed end-to-end beamforming prediction-based perception-assisted communication scheme is crucial for improving vehicle communication capability and speed.

[0066] (4) Advantages and beneficial effects Current research in this field mainly focuses on RIS-assisted ISAC systems or RIS-assisted V2I systems, but there is limited research exploring the performance optimization of V2I systems enabled by the combination of RIS and ISAC technologies. Furthermore, current research on joint beamforming prediction for RIS-assisted V2I or ISAC systems mostly employs a two-stage algorithm: first estimating channel state information, then reconstructing the channel based on the estimated channel state information to jointly optimize the beamforming matrix. This staged processing leads to estimation errors in the first stage affecting beamforming design in the second stage, resulting in reduced communication device feasibility and speed. Additionally, current beamforming joint optimization in these systems often uses traditional analytical optimization algorithms, which have high computational complexity and slow convergence speed, making it difficult to meet the millimeter-level decision-making requirements of V2I systems. Considering the current state of research, this application proposes a RIS-assisted ISAC-based V2I system framework and a corresponding beamforming prediction method. This application has the following main advantages: 1) This application proposes a V2I system based on ISAC with RIS assistance, which provides a new solution for the construction of wireless networks in future intelligent transportation systems, and conducts research on optimizing the system's performance.

[0067] 2) To avoid the problems of inaccurate beamforming optimization due to channel estimation error in the two-stage beamforming prediction algorithm and the high computational complexity and slow convergence of traditional analytical algorithms, this application proposes an end-to-end beamforming prediction method based on deep learning. That is, a data-driven approach is used to establish a direct relationship between the received echo and the beamforming matrix, thereby bypassing the need for the base station to obtain CSI and historical channel parameters, simplifying the beamforming prediction process, and reducing computational overhead.

[0068] 3) Based on the proposed system, in order to effectively improve the overall achievable communication rate of the vehicle, this application proposes a method for jointly optimizing the base station transmit beamforming matrix and RIS phase shift matrix, satisfying the transmit power and phase shift matrix mode constraints. The original problem is transformed into an unconstrained problem through a penalty method, and then a deep learning loss function is designed to solve this problem.

[0069] This application also provides a deep learning-based integrated sensory network beamforming prediction system that can implement the above-described method. The system includes: Signal transceiver module: configured at the base station, used to send ISAC signals to multiple mobile vehicles in each time slot using a full-duplex mechanism and with the assistance of RIS in a non-orthogonal multiple access (NOMA) manner, and to receive echo signals reflected by the vehicles and transmitted back via RIS. Data preprocessing module: used to stack the echo signals received in the current time slot and several historical time slots into a three-dimensional tensor; Deep Neural Network (DNN) module: used to extract spatiotemporal features from the input echo signal tensor and directly output the base station transmit beamforming matrix and RIS phase shift matrix for the next time slot; Beamforming application module: used to apply the base station transmit beamforming matrix and RIS phase shift matrix output by the DNN module to the communication and sensing transmission in the next time slot.

[0070] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0071] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0072] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0073] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0074] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0075] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0076] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0077] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented in the embodiments of this program product are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments. The executable computer program code or "code" used to perform the various embodiments can be written in high-level programming languages ​​such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0078] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0079] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0080] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0081] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0082] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0083] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0084] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0085] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0086] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0087] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0088] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A beamforming prediction method based on a deep learning-integrated sensory network, characterized in that, The application of the Intelligent Reflector RIS-assisted communication and sensing integrated ISAC vehicle-to-infrastructure (V2I) system includes the following steps: In each time slot, the base station adopts a full-duplex mechanism and sends ISAC signals to multiple mobile vehicles in a non-orthogonal multiple access (NOMA) manner with the assistance of RIS, and receives echo signals reflected by the vehicles and transmitted back via RIS. The echo signals received in the current time slot and several historical time slots are stacked into a three-dimensional tensor, which is then used as the input to a deep neural network (DNN). The DNN is used to extract the spatiotemporal features of the input echo signal tensor, and the base station transmit beamforming matrix and RIS phase shift matrix of the next time slot are directly output. The base station transmit beamforming matrix and RIS phase shift matrix output by the DNN are applied to the communication and sensing transmission in the next time slot.

2. The method according to claim 1, characterized in that, The deep neural network (DNN) includes: The CNN module is used to extract the spatial features of the echo signal for each historical time slot; The LSTM module is used to capture the temporal dependencies of the feature sequences processed by the CNN module and output the hidden state sequence. The temporal attention module is used to adaptively weight the hidden state sequence output by the LSTM module to obtain the weighted global feature vector. The fully connected module is used to generate the RIS phase shift matrix and the base station transmit beamforming matrix based on the weighted global feature vector.

3. The method according to claim 2, characterized in that, The time attention module concatenates the hidden state of the LSTM at the final time step with the hidden state at each time step, calculates the attention weights using trainable parameters, and performs a weighted summation of the hidden state sequence to highlight the time steps containing key information.

4. The method according to claim 1, characterized in that, The output layer of the DNN includes a power constraint layer, which is used to normalize the real and imaginary parts of the base station transmit beamforming matrix output by the fully connected layer to meet the total transmit power constraint of the base station.

5. The method according to claim 1, characterized in that, The RIS phase shift matrix is ​​obtained by directly outputting the phase shift vector from the DNN and converting it into a phase shift matrix that satisfies the unity modulus constraint through complex exponential form.

6. The method according to claim 1, characterized in that, The training process of the DNN adopts an unconstrained optimization loss function based on a penalty function. This loss function integrates the goal of maximizing the total achievable communication rate of the system, as well as the transmit power constraint, RIS phase shift unity mode constraint, and serial interference cancellation SIC decoding constraint.

7. A deep learning-based synesthetic network beamforming prediction system, characterized in that, Applications in the Intelligent Reflector RIS-assisted communication and sensing integrated ISAC vehicle-to-infrastructure (V2I) system include: Signal transceiver module: configured at the base station, used to send ISAC signals to multiple mobile vehicles in each time slot using a full-duplex mechanism and with the assistance of RIS in a non-orthogonal multiple access (NOMA) manner, and to receive echo signals reflected by the vehicles and transmitted back via RIS. Data preprocessing module: used to stack the echo signals received in the current time slot and several historical time slots into a three-dimensional tensor; Deep Neural Network (DNN) module: used to extract spatiotemporal features from the input echo signal tensor and directly output the base station transmit beamforming matrix and RIS phase shift matrix for the next time slot; Beamforming application module: used to apply the base station transmit beamforming matrix and RIS phase shift matrix output by the DNN module to the communication and sensing transmission in the next time slot.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images