A beam management method and device for environmental semantic aided communication

Abstract

This application discloses a beam management method and apparatus for environment semantic-assisted communication. It acquires raw image data from a base station camera, uses a generative model for repair and enhancement to generate an enhanced image; employs a lightweight detection network to extract environmental semantics, generating candidate bounding boxes and confidence scores; obtains the current optimal beam measured by the base station through beam scanning, maps it to a spatial azimuth angle, and projects it onto the image plane to obtain the expected position of the target vehicle; performs spatial likelihood matching between the expected position and the candidate bounding boxes, and identifies the target vehicle based on the confidence scores; tracks the target vehicle's trajectory to obtain temporal state information; inputs the temporal state information into a temporal prediction network to predict the future optimal beam and perform beam switching. This achieves all-weather stable beam management for vehicle-to-everything (V2X) millimeter-wave communication systems in low-light nighttime environments, solves the performance gap caused by uneven day-night data distribution, and reduces beam scanning overhead and access latency.

Description

Technical Field

[0001] This application relates to the field of vehicle-to-everything (V2X) communication, and in particular to a beam management method and apparatus for environmental semantic-assisted communication. Background Technology

[0002] In vehicular network millimeter-wave communication systems, beamforming is a key technology for achieving high-gain directional transmission. Traditional beam management relies on periodic full scans to select the optimal beam. However, as the number of beams increases, the training overhead and access latency of this method become insufficient to meet the low-latency communication requirements of high-speed mobile scenarios. To address this, a deep learning-based vision-assisted beam prediction technique has been proposed. This technique directly predicts the optimal beam direction from environmental images captured by base station cameras, thereby reducing beam scanning overhead.

[0003] However, this technology faces significant challenges in nighttime scenarios. Low light levels in connected vehicle environments at night result in insufficient brightness, significant noise, and blurred details in camera images, severely weakening the model's ability to extract effective visual features. Most existing image enhancement methods are geared towards general vision tasks and lack specific designs for beam prediction, failing to utilize wireless physical information such as beam direction and signal strength to guide the enhancement process. More critically, high-quality nighttime labeled data is scarce, causing a severe imbalance in model performance between day and night, making it difficult to ensure the stable and reliable operation of connected vehicle communication systems in all-weather scenarios. Summary of the Invention

[0004] This application provides a beam management method and apparatus for environmental semantic-assisted communication, which solves the problems of poor image quality, lack of task-oriented enhancement methods, and uneven distribution of day and night data leading to decreased beam prediction performance in visual-assisted beam prediction under low light conditions at night.

[0005] Firstly, this application provides a beam management method for environment semantic-assisted communication, including:

[0006] The system acquires raw image data containing the target vehicle collected by the base station camera, and uses a preset generative model to repair and enhance the raw image data to generate diverse enhanced images.

[0007] A lightweight detection network is used to extract environmental semantics from the enhanced image, generating candidate bounding boxes containing all candidate vehicles and the confidence scores of each candidate bounding box.

[0008] The current optimal beam measured by the target base station through beam scanning at the current moment is obtained, the current optimal beam is mapped to the corresponding spatial azimuth angle, and the spatial azimuth angle is projected onto the image plane in combination with the camera imaging geometry to obtain the expected position of the target vehicle.

[0009] Spatial likelihood matching is performed between the expected location and the candidate bounding boxes, and the target vehicle is identified from the candidate vehicles by combining the confidence scores.

[0010] The system tracks the target vehicle's trajectory in a continuous image sequence to obtain the target vehicle's corresponding temporal state information. The temporal state information is then input into a temporal prediction network based on a gated recurrent unit to predict the target vehicle's optimal future beam at a future time, and beam switching is performed based on the optimal future beam.

[0011] Secondly, this application provides a beam management device for environmental semantic-assisted communication, comprising:

[0012] The enhanced image generation module is configured to acquire raw image data containing the target vehicle collected by the base station camera, and use a preset generative model to repair and enhance the raw image data to generate diverse enhanced images.

[0013] The candidate bounding box generation module is configured to use a lightweight detection network to extract environmental semantics from the enhanced image and generate candidate bounding boxes containing all candidate vehicles and the confidence scores of each candidate bounding box.

[0014] The expected location determination module is configured to obtain the current optimal beam measured by the target base station through beam scanning at the current moment, map the current optimal beam to the corresponding spatial azimuth angle, and project the spatial azimuth angle onto the image plane in combination with the camera imaging geometry to obtain the expected location of the target vehicle.

[0015] The target vehicle recognition module is configured to perform spatial likelihood matching between the expected location and the candidate bounding box, and combine the confidence score to identify the target vehicle from the candidate vehicles.

[0016] The execution module is configured to perform trajectory tracking of the target vehicle in a continuous image sequence, obtain the temporal state information of the target vehicle, input the temporal state information into a temporal prediction network based on a gated recurrent unit, predict the future optimal beam of the target vehicle at a future time, and perform beam switching according to the future optimal beam.

[0017] Thirdly, this application provides a readable medium including executable instructions, which, when executed by a processor of an electronic device, cause the electronic device to perform any of the methods described in the first aspect.

[0018] Fourthly, this application provides an electronic device including a processor and a memory storing execution instructions, wherein when the processor executes the execution instructions stored in the memory, the processor performs the method as described in any of the first aspects.

[0019] This application provides a beam management method and apparatus for environmental semantic-assisted communication. It acquires raw image data containing a target vehicle from a base station camera, repairs and enhances the raw image data using a preset generative model to generate diverse enhanced images, and uses a lightweight detection network to extract environmental semantics from the enhanced images, generating candidate bounding boxes containing all candidate vehicles and their corresponding confidence scores. It obtains the current optimal beam measured by the target base station through beam scanning at the current moment, maps the current optimal beam to its corresponding spatial azimuth angle, and projects the spatial azimuth angle onto the image plane based on camera imaging geometry to obtain the expected position of the target vehicle. It performs spatial likelihood matching between the expected position and the candidate bounding boxes, and identifies the target vehicle from the candidate vehicles based on the confidence scores. It performs trajectory tracking of the target vehicle in a continuous image sequence to obtain the corresponding temporal state information of the target vehicle. It inputs the temporal state information into a temporal prediction network based on gated recurrent units to predict the future optimal beam of the target vehicle at a future time, and performs beam switching based on the future optimal beam. Further effects of the above-described non-conventional preferred method will be described below in conjunction with specific embodiments. It realizes all-weather stable beam management of vehicle-to-everything millimeter-wave communication system in low-light environment at night, effectively solves the performance gap caused by uneven data distribution between day and night, and significantly reduces beam scanning overhead and access latency. Attached Figure Description

[0020] To more clearly illustrate the embodiments of this application or the existing technical solutions, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a beam management method for environmental semantic-assisted communication provided in an embodiment of this application;

[0022] Figure 2 A flowchart illustrating another beam management method for environmental semantic-assisted communication provided in an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of the structure of a beam management device for environmental semantic-assisted communication provided in an embodiment of this application;

[0024] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] In vehicular network millimeter-wave communication systems, beamforming is a key technology for achieving high-gain directional transmission. Traditional beam management relies on periodic full scans to select the optimal beam. However, as the number of beams increases, the training overhead and access latency of this method become insufficient to meet the low-latency communication requirements of high-speed mobile scenarios. To address this, a deep learning-based vision-assisted beam prediction technique has been proposed. This technique directly predicts the optimal beam direction from environmental images captured by base station cameras, thereby reducing beam scanning overhead.

[0027] However, this technology faces significant challenges in nighttime scenarios. Low light levels in connected vehicle environments at night result in insufficient brightness, significant noise, and blurred details in camera images, severely weakening the model's ability to extract effective visual features. Most existing image enhancement methods are geared towards general vision tasks and lack specific designs for beam prediction, failing to utilize wireless physical information such as beam direction and signal strength to guide the enhancement process. More critically, high-quality nighttime labeled data is scarce, causing a severe imbalance in model performance between day and night, making it difficult to ensure the stable and reliable operation of connected vehicle communication systems in all-weather scenarios.

[0028] To address this issue, this application proposes a beam management method for environmental semantic-assisted communication, aiming to solve the problems of poor image quality, lack of task-oriented enhancement methods, and uneven day-night data distribution leading to decreased beam prediction performance in visual-assisted beam prediction under low-light conditions at night. In this embodiment, the beam management method for environmental semantic-assisted communication includes:

[0029] Step 101: Obtain the original image data containing the target vehicle collected by the base station camera, and use a preset generative model to repair and enhance the original image data to generate diverse enhanced images.

[0030] The original image data is input into the encoding module of the variational autoencoder, which compresses the original image data into a low-dimensional latent space to obtain the corresponding latent feature map. The beam index corresponding to the current optimal beam and the preset text prompt are obtained, and the beam index and text prompt are vectorized and fused to form a multimodal guidance vector. Based on the multimodal guidance vector, forward diffusion noise addition and backward denoising are performed on the latent feature map in the latent space to generate an enhanced feature map. The enhanced feature map is decoded back into the pixel space, and adaptive brightness adjustment is performed in combination with the brightness information of the original image data to generate an enhanced image.

[0031] In vehicle-to-everything (V2X) millimeter-wave communication systems, base station cameras continuously acquire road scene images as the raw input for visual-assisted beam prediction. However, in low-light conditions at night, the raw images often suffer from insufficient brightness, significant noise, and blurred edge details, which can lead to a significant drop in detection performance if directly used for subsequent semantic extraction. To address this, this embodiment introduces a pre-defined generative model to repair and enhance the raw image data, generating enhanced images with higher quality and more diverse scenes, providing reliable visual input for subsequent processing.

[0032] Specifically, the enhancement process of the generative model consists of four stages. First, the original image data is input into the encoding module of the variational autoencoder. Through encoding compression, the image is mapped from the high-dimensional pixel space to the low-dimensional latent space to obtain the corresponding latent feature map. The representation in the latent space removes redundant pixel information from the image, preserving the core semantic structure of the scene and providing a more compact operating space for subsequent diffusion generation.

[0033] The system obtains the beam index corresponding to the current optimal beam and a preset text prompt, and then fuses them into a multimodal guidance vector. The beam index carries the pointing information of the antenna beam at the current moment, while the text prompt describes the semantic constraints of the target scene. The multimodal guidance vector formed by fusing the two can unify the directional information in the wireless physical domain and the scene description in the visual semantic domain into the same conditional space, thereby guiding the subsequent generation process in a direction related to the beam prediction task, rather than generalized image enhancement.

[0034] Subsequently, conditioned on the multimodal guiding vector, a forward diffusion noise addition and reverse denoising generation process is performed on the latent feature map in the latent space. In the forward diffusion stage, Gaussian noise is progressively injected into the latent feature map to simulate image degradation; in the reverse denoising stage, under the constraints of the multimodal guiding vector, the enhanced feature map with richer details is gradually restored and generated. This process not only repairs noise and blur defects in the original image but also generates diverse scene variations in the latent space, effectively expanding the data distribution of nighttime scenes and alleviating the problem of day-night data imbalance.

[0035] Finally, the enhanced feature map is decoded back from the latent space to the pixel space, and adaptive brightness adjustment is performed by combining the brightness information from the original image data to generate the final enhanced image. The brightness adjustment step is introduced to avoid a large deviation between the decoded image and the overall lighting conditions of the original scene, ensuring that the enhanced image maintains visual consistency with the real scene, thus not affecting the normal operation of the subsequent semantic extraction network.

[0036] Step 102: Use a lightweight detection network to extract environmental semantics from the enhanced image, and generate candidate bounding boxes containing all candidate vehicles and the confidence scores of each candidate bounding box.

[0037] The lightweight detection network is the MobileNetV2-SSDLite network; the enhanced image is input into the MobileNetV2 backbone network, and the scale feature map of the enhanced image is extracted through depthwise separable convolution; candidate bounding boxes and confidence scores are generated on each scale feature map through preset anchor box regression.

[0038] After obtaining the enhanced image, it is necessary to extract environmental semantic information from it. Specifically, this involves detecting all possible candidate vehicles in the image and generating corresponding bounding boxes and confidence scores for each candidate target. Considering the stringent real-time requirements of vehicle-to-everything (V2X) systems, this embodiment can employ the lightweight detection network MobileNetV2-SSDLite to perform this task, significantly reducing computational overhead while ensuring detection accuracy.

[0039] The MobileNetV2-SSDLite network consists of two parts: the MobileNetV2 backbone network and the SSDLite detection head. In the feature extraction stage, the enhanced image is input into the MobileNetV2 backbone network, which uses depthwise separable convolution as its core computational unit. It decomposes standard convolution into two steps: depthwise convolution and pointwise convolution. This significantly reduces the number of parameters and computational cost while still effectively extracting multi-scale feature maps from the image. The introduction of multi-scale feature maps enables the network to simultaneously perceive both large and small targets, which is of great significance for vehicle detection at different distances.

[0040] In the target detection stage, the SSDLite detection head performs regression predictions on feature maps at various scales based on preset anchor boxes, generating the coordinate offset of candidate bounding boxes and their corresponding confidence scores for each anchor box location. The confidence score reflects the probability of a vehicle target existing within the candidate box and is a crucial basis for weighted fusion in the subsequent target recognition stage. After post-processing operations such as non-maximum suppression, the final output is a set of bounding boxes containing all candidate vehicles in the scene, along with the confidence scores for each box, for use in subsequent steps.

[0041] Step 103: Obtain the current optimal beam measured by the target base station through beam scanning at the current moment, map the current optimal beam to the corresponding spatial azimuth angle, and project the spatial azimuth angle onto the image plane in combination with the camera imaging geometry to obtain the expected position of the target vehicle.

[0042] Based on the mapping relationship between beam identifier and spatial azimuth, the current optimal beam is converted into the corresponding spatial azimuth; the intrinsic parameter matrix of the base station camera and the transformation matrix of the camera coordinate system relative to the antenna array coordinate system are obtained; using the intrinsic parameter matrix and the transformation matrix, the three-dimensional direction vector corresponding to the spatial azimuth is projected onto the image plane to obtain the expected position of the target vehicle in the image plane.

[0043] After detecting candidate vehicles, it is necessary to locate the target vehicle corresponding to the current communication link from among numerous candidate targets. To this end, this embodiment utilizes the beam information of the base station at the current moment to establish a spatial correspondence between the wireless signal domain and the image visual domain through geometric projection, predicting the approximate position of the target vehicle on the image plane, and providing prior constraints for subsequent target recognition.

[0044] Specifically, after the target base station obtains the current optimal beam through beam scanning at the current moment, it first converts the current optimal beam into the corresponding spatial azimuth angle based on the pre-established mapping relationship between beam identifiers and spatial azimuth angles. The beam identifier is a pre-defined number in the beam codebook used to uniquely identify the direction of each beam. Each beam identifier corresponds to the azimuth angle information of the antenna array under a specific direction, so the conversion can be completed directly by looking up a table.

[0045] After obtaining the spatial azimuth angle, it needs to be further projected from the antenna coordinate system to the image plane. This mainly relies on two sets of parameters: one is the intrinsic parameter matrix of the base station camera, which describes the imaging geometric parameters such as the camera's focal length and principal point coordinates; the other is the transformation matrix of the camera coordinate system relative to the antenna array coordinate system, which describes the rotation and translation relationship between the two coordinate systems.

[0046] Using the two sets of parameters mentioned above, the three-dimensional direction vector corresponding to the spatial azimuth angle is successively transformed by the coordinate system and projected through perspective, finally mapped onto the image plane to obtain the expected position of the target vehicle in the image. The expected position is essentially the projection point of the beam direction onto the image plane, reflecting the area in the image where the target vehicle currently communicating with the base station is most likely to appear.

[0047] Step 104: Perform spatial likelihood matching between the expected location and the candidate bounding boxes, and combine the confidence scores to identify the target vehicle from the candidate vehicles.

[0048] Based on the spatial uncertainties corresponding to the expected location, beamwidth, and vehicle size, a two-dimensional Gaussian distribution model is constructed. The probability value of the center point of each candidate bounding box falling within the two-dimensional Gaussian distribution model is calculated, and the probability value is used as the likelihood score of each candidate bounding box. The likelihood scores of each candidate bounding box are weighted and fused with the corresponding confidence scores to obtain a comprehensive matching score. The candidate vehicle corresponding to the candidate bounding box with the highest comprehensive matching score is identified as the target vehicle.

[0049] After obtaining the expected location of the target vehicle, spatial likelihood matching is performed between the expected location and the candidate bounding box set. Based on the confidence of each candidate box, the target vehicle is identified from the candidate vehicles.

[0050] The matching process first constructs a two-dimensional Gaussian distribution model based on the expected location. Due to the limited beamwidth and the physical size of the vehicle, there is a certain spatial uncertainty between the actual image location and the expected location of the target vehicle. Matching based solely on point-to-point distance is insufficient to accurately reflect this uncertainty. Therefore, a two-dimensional Gaussian distribution model is established with the expected location as the mean center and the spatial diffusion range, jointly determined by the beamwidth and vehicle size, as the covariance. This model quantifies the location uncertainty in the form of a probability distribution.

[0051] Based on this, the probability value of the center point of each candidate bounding box falling within the two-dimensional Gaussian distribution model is calculated, and this probability is used as the likelihood score of the candidate box. The higher the likelihood score, the more the image location of the candidate vehicle matches the beam projection expectation, and the more likely it is to be the target vehicle. Subsequently, the likelihood scores and confidence scores of each candidate box are weighted and fused to obtain a comprehensive matching score.

[0052] The comprehensive matching score takes into account both the reliability of visual detection and the spatial constraints of beam direction, making it more robust than a single metric. Finally, the candidate vehicle corresponding to the candidate bounding box with the highest comprehensive matching score is identified as the target vehicle, completing the target localization.

[0053] Step 105: Track the target vehicle's trajectory in a continuous image sequence to obtain the target vehicle's corresponding temporal state information; input the temporal state information into a temporal prediction network based on a gated recurrent unit to predict the target vehicle's optimal future beam at a future time, and perform beam switching based on the optimal future beam.

[0054] After target vehicle identification is completed, trajectory tracking of the target vehicle is performed in a continuous image sequence to obtain its temporal state information that changes over time. The temporal state information records the dynamic characteristics of the target vehicle, such as its position, speed, and direction of motion, in consecutive frames, reflecting the vehicle's motion patterns and serving as the input basis for predicting the optimal beam in the future.

[0055] The temporal prediction network has a two-layer gated recurrent unit structure. The first layer of recurrent units is used to extract the temporal dependency features of the temporal state information. The second layer of recurrent units is used to perform nonlinear transformation on the temporal dependency features and map them to the temporal feature space. The fully connected classification layer is used to map the temporal feature space to the prediction probability of each predefined beam and select the beam with the highest prediction probability as the future optimal beam.

[0056] The temporal state information is input into a temporal prediction network based on gated recurrent units (ROUs). The network then performs a mapping prediction from historical motion trajectories to the optimal future beam. The temporal prediction network adopts a two-layer gated recurrent unit structure. The first layer of recurrent units is responsible for extracting temporal dependency features from the input sequence, capturing the short-term and long-term dynamic patterns of vehicle motion. The second layer of recurrent units performs further nonlinear transformations on the extracted temporal dependency features, mapping them to a higher-level temporal feature space. Finally, a fully connected classification layer maps the temporal feature space to the prediction probabilities of each predefined beam, and outputs the beam with the highest prediction probability as the optimal future beam.

[0057] After obtaining the prediction result of the future optimal beam, the system performs beam switching in advance. Specifically, the base station sends the predicted future optimal beam to the millimeter-wave transceiver. The millimeter-wave transceiver retrieves the corresponding beam configuration parameters from the predefined beam codebook based on the future optimal beam and configures the antenna array to point in the direction corresponding to the parameters, thereby completing beam alignment before the target vehicle reaches the predicted position and achieving zero-overhead or low-overhead beam switching.

[0058] As can be seen from the above technical solutions, the beneficial effects of this embodiment are:

[0059] This application provides a beam management method for environmental semantic-assisted communication. The method acquires raw image data containing a target vehicle from a base station camera. Using a preset generative model, the raw image data is repaired and enhanced to generate diverse enhanced images. A lightweight detection network is used to extract environmental semantics from the enhanced images, generating candidate bounding boxes containing all candidate vehicles and their corresponding confidence scores. The current optimal beam, measured by the target base station through beam scanning at the current moment, is obtained and mapped to its corresponding spatial azimuth angle. This spatial azimuth angle is then projected onto the image plane using camera imaging geometry to obtain the expected position of the target vehicle. Spatial likelihood matching is performed between the expected position and the candidate bounding boxes, and the target vehicle is identified from the candidate vehicles based on the confidence scores. The target vehicle's trajectory is tracked in a continuous image sequence to obtain its corresponding temporal state information. This temporal state information is input into a temporal prediction network based on gated recurrent units to predict the target vehicle's future optimal beam at a future time, and beam switching is performed based on the future optimal beam. Further effects of the above-described non-conventional preferred method will be described below in conjunction with specific embodiments. It realizes all-weather stable beam management of vehicle-to-everything millimeter-wave communication system in low-light environment at night, effectively solves the performance gap caused by uneven data distribution between day and night, and significantly reduces beam scanning overhead and access latency.

[0060] Figure 1 The example shown is only a basic embodiment of a beam management method for environmental semantic-assisted communication according to this application. With certain optimizations and extensions, other preferred embodiments of a beam management method for environmental semantic-assisted communication can be obtained.

[0061] like Figure 2 The image shown is another specific embodiment of a beam management method for environmental semantic-assisted communication according to this application.

[0062] In this embodiment, a beam management method for environment semantic-assisted communication includes the following steps:

[0063] Step 201: Obtain the original image data containing the target vehicle collected by the base station camera, and use a preset generative model to repair and enhance the original image data to generate diverse enhanced images.

[0064] Step 202: Use a lightweight detection network to extract environmental semantics from the enhanced image, and generate candidate bounding boxes containing all candidate vehicles and the confidence scores of each candidate bounding box.

[0065] Step 203: Obtain the current optimal beam measured by the target base station through beam scanning at the current moment, map the current optimal beam to the corresponding spatial azimuth angle, and project the spatial azimuth angle onto the image plane in combination with the camera imaging geometry to obtain the expected position of the target vehicle.

[0066] Step 204: Perform spatial likelihood matching between the expected location and the candidate bounding boxes, and combine the confidence scores to identify the target vehicle from the candidate vehicles.

[0067] Step 205: Track the target vehicle's trajectory in a continuous image sequence to obtain the target vehicle's corresponding temporal state information; input the temporal state information into a temporal prediction network based on a gated recurrent unit to predict the target vehicle's optimal future beam at a future time, and perform beam switching based on the optimal future beam.

[0068] Step 206: Send the optimal future beam to the millimeter-wave transceiver of the target base station.

[0069] After the timing prediction network outputs the prediction result of the optimal future beam, the system needs to pass the prediction result to the execution layer to trigger the actual beam switching action. To this end, this embodiment sends the prediction result of the optimal future beam to the millimeter-wave transceiver of the target base station in the form of an instruction. This transmission process establishes an information path between the upper-layer prediction module and the lower-layer hardware execution module, enabling the beam decision based on vision and trajectory prediction to be applied to the actual antenna control system in a timely manner.

[0070] Since the prediction results are generated and transmitted before the target vehicle reaches the expected location, the millimeter-wave transceiver can know the beam direction required for the next moment in advance, thus gaining enough preparation time for subsequent pre-configuration operations and avoiding the risk of link interruption caused by beam switching lagging behind vehicle movement in the traditional method.

[0071] Step 207: The millimeter-wave transceiver retrieves the corresponding beam configuration parameters from the predefined beam codebook based on the optimal future beam, and configures the antenna array to point in the direction corresponding to the beam configuration parameters to complete the beam switching.

[0072] After receiving the instruction for the optimal future beam, the millimeter-wave transceiver retrieves the beam configuration parameters corresponding to that beam from the beam codebook. The beam codebook is a set of parameters pre-designed and stored by the system. Each beam entry corresponds to a complete set of antenna array excitation configurations, including parameters such as the phase weight and amplitude weight of each antenna element. These parameters together determine the actual radiation direction and beam shape of the antenna array.

[0073] The millimeter-wave transceiver adjusts the excitation signals of each element in the antenna array based on the retrieved beam configuration parameters, configuring the main lobe direction of the antenna array to correspond to the expected position of the target vehicle, thereby completing the beam switching. The entire switching process is completed before the target vehicle actually arrives at that position, achieving pre-alignment of the beam direction with the vehicle's trajectory and ensuring the continuity and stability of the millimeter-wave link during the vehicle's high-speed movement.

[0074] As can be seen from the above technical solution, the beneficial effects of this embodiment are as follows: by sending the future optimal beam output by the timing prediction network to the millimeter-wave transceiver in advance, and pre-configuring the antenna array according to the beam configuration parameters stored in the beam codebook, the time decoupling between beam switching action and vehicle movement trajectory is realized, so that the antenna pointing adjustment is completed before the target vehicle reaches the expected position. This eliminates the link interruption problem caused by beam decision lag in the traditional periodic full-scan method, significantly reduces beam switching latency, and improves the link continuity and communication reliability of the vehicle-to-everything (V2X) millimeter-wave communication system in high-speed mobile scenarios.

[0075] like Figure 3 The image shown is a specific embodiment of a beam management device for environment semantic-assisted communication according to this application. This embodiment of a beam management device for environment semantic-assisted communication is used to perform... Figures 1-2 A physical device for beam management in environment semantic-assisted communication is provided. Its technical solution is essentially the same as the embodiments described above, and the corresponding descriptions in the embodiments above also apply to this embodiment. The beam management device for environment semantic-assisted communication in this embodiment includes:

[0076] The enhanced image generation module 301 is configured to acquire original image data containing the target vehicle collected by the base station camera, and use a preset generative model to repair and enhance the original image data to generate diverse enhanced images.

[0077] The candidate bounding box generation module 302 is configured to use a lightweight detection network to extract environmental semantics from the enhanced image and generate candidate bounding boxes containing all candidate vehicles and the confidence scores of each candidate bounding box.

[0078] The expected location determination module 303 is configured to obtain the current optimal beam measured by the target base station through beam scanning at the current time, map the current optimal beam to the corresponding spatial azimuth angle, and project the spatial azimuth angle onto the image plane in combination with the camera imaging geometry to obtain the expected location of the target vehicle.

[0079] The target vehicle recognition module 304 is configured to perform spatial likelihood matching between the expected location and the candidate bounding box, and combine the confidence score to identify the target vehicle from the candidate vehicles.

[0080] The execution module 305 is configured to perform trajectory tracking of the target vehicle in a continuous image sequence, obtain the temporal state information corresponding to the target vehicle, input the temporal state information into a temporal prediction network based on a gated recurrent unit, predict the future optimal beam of the target vehicle at a future time, and perform beam switching according to the future optimal beam.

[0081] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include RAM, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk storage device. Of course, the electronic device may also include other hardware required for other services.

[0082] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0083] Memory is used to store instructions for execution. Specifically, instructions for execution are computer programs that can be executed. Memory can include main memory and non-volatile memory, and it provides the processor with execution instructions and data.

[0084] In one possible implementation, the processor reads the corresponding execution instructions from non-volatile memory into main memory and then executes them. Alternatively, it may obtain the corresponding execution instructions from other devices to form a beam management device for environment-semantic assisted communication at the logical level. The processor executes the execution instructions stored in the memory to implement the beam management method for environment-semantic assisted communication provided in any embodiment of this application.

[0085] The above is as stated in this application. Figure 3The method executed by the beam management device for environmental semantic-assisted communication provided in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0086] The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0087] This application also proposes a readable medium storing execution instructions. When these instructions are executed by a processor of an electronic device, the electronic device can perform a beam management method for environmental semantic-assisted communication provided in any embodiment of this application, specifically for executing, for example... Figure 1 or Figure 2 The method shown.

[0088] The electronic devices in the foregoing embodiments may be computers.

[0089] Those skilled in the art will understand that the embodiments of this application can be provided as methods or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or a combination of software and hardware.

[0090] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0091] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0092] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A beam management method for environment semantic-assisted communication, characterized in that, The method is applied to a vehicle-to-everything (V2X) millimeter-wave communication system, and the method includes: The system acquires raw image data containing the target vehicle collected by the base station camera, and uses a preset generative model to repair and enhance the raw image data to generate diverse enhanced images. A lightweight detection network is used to extract environmental semantics from the enhanced image, generating candidate bounding boxes containing all candidate vehicles and the confidence scores of each candidate bounding box. The current optimal beam measured by the target base station through beam scanning at the current moment is obtained, the current optimal beam is mapped to the corresponding spatial azimuth angle, and the spatial azimuth angle is projected onto the image plane in combination with the camera imaging geometry to obtain the expected position of the target vehicle. The expected location is spatially likely matched with the candidate bounding boxes, and the target vehicle is identified from the candidate vehicles by combining the confidence score. The target vehicle is tracked in a continuous image sequence to obtain the temporal state information corresponding to the target vehicle; the temporal state information is input into a temporal prediction network based on a gated recurrent unit to predict the future optimal beam of the target vehicle at a future time, and beam switching is performed according to the future optimal beam.

2. The method according to claim 1, characterized in that, The process of using a preset generative model to repair and enhance the original image data, generating diverse enhanced images, includes: The original image data is input into the encoding module of the variational autoencoder to compress the original image data into a low-dimensional latent space and obtain the corresponding latent feature map. Obtain the beam index and preset text prompt corresponding to the current optimal beam, and then perform vectorized fusion of the beam index and the text prompt to form a multimodal guidance vector; Using the multimodal guiding vector as a condition, forward diffusion noise addition and reverse denoising are performed on the latent feature map in the latent space to generate an enhanced feature map; The enhanced feature map is decoded back into pixel space, and adaptive brightness adjustment is performed by combining the brightness information of the original image data to generate the enhanced image.

3. The method according to claim 1, characterized in that, The lightweight detection network is the MobileNetV2-SSDLite network; the environmental semantic extraction of the enhanced image using the lightweight detection network to generate candidate bounding boxes containing all candidate vehicles and the confidence scores corresponding to each candidate bounding box includes: The enhanced image is input into the MobileNetV2 backbone network, and the scale feature map of the enhanced image is extracted through depthwise separable convolution. The candidate bounding boxes and the confidence scores are generated by regression on the feature maps at each scale using preset anchor boxes.

4. The method according to claim 1, characterized in that, The step of mapping the current optimal beam to the corresponding spatial azimuth angle, and projecting the spatial azimuth angle onto the image plane in combination with the camera imaging geometry to obtain the expected position of the target vehicle includes: Based on the mapping relationship between the beam identifier and the spatial azimuth angle, the current optimal beam is converted into the corresponding spatial azimuth angle; Obtain the intrinsic parameter matrix of the base station camera and the transformation matrix of the camera coordinate system relative to the antenna array coordinate system; Using the intrinsic parameter matrix and the transformation matrix, the three-dimensional direction vector corresponding to the spatial azimuth angle is projected onto the image plane to obtain the expected position of the target vehicle in the image plane.

5. The method according to claim 4, characterized in that, The step of performing spatial likelihood matching between the expected location and the candidate bounding box, and combining the confidence score to identify the target vehicle from the candidate vehicles includes: Based on the spatial uncertainty of the expected location and beamwidth corresponding to the vehicle size, a two-dimensional Gaussian distribution model is constructed. Calculate the probability value of the center point of each candidate bounding box falling within the two-dimensional Gaussian distribution model, and use the probability value as the likelihood score of each candidate bounding box; The likelihood scores of each candidate bounding box are weighted and fused with the corresponding confidence scores to obtain a comprehensive matching score. The candidate vehicle corresponding to the candidate bounding box with the highest comprehensive matching score is identified as the target vehicle.

6. The method according to claim 1, characterized in that, The time-series prediction network is a two-layer gated recurrent unit structure, wherein the first layer recurrent unit is used to extract the time-series dependency features of the time-series state information; The second-layer recurrent unit is used to perform nonlinear transformation on the time-dependent features and map them to the time-series feature space; the fully connected classification layer is used to map the time-series feature space to the predicted probabilities of each predefined beam, and to take the beam with the highest predicted probability as the future optimal beam.

7. The method according to claim 6, characterized in that, The step of performing beam switching based on the future optimal beam includes: The future optimal beam is transmitted to the millimeter-wave transceiver of the target base station; The millimeter-wave transceiver retrieves the corresponding beam configuration parameters from a predefined beam codebook based on the future optimal beam, and configures the antenna array to point in a direction corresponding to the beam configuration parameters in order to complete the beam switching.

8. A beam management device for environmental semantic-assisted communication, characterized in that, include: The enhanced image generation module is configured to acquire raw image data containing the target vehicle collected by the base station camera, and use a preset generative model to repair and enhance the raw image data to generate diverse enhanced images. The candidate bounding box generation module is configured to use a lightweight detection network to extract environmental semantics from the enhanced image and generate candidate bounding boxes containing all candidate vehicles and the confidence scores of each candidate bounding box. The expected location determination module is configured to obtain the current optimal beam measured by the target base station through beam scanning at the current time, map the current optimal beam to the corresponding spatial azimuth angle, and project the spatial azimuth angle onto the image plane in combination with the camera imaging geometry to obtain the expected location of the target vehicle. The target vehicle recognition module is configured to perform spatial likelihood matching between the expected location and the candidate bounding boxes, and combine the confidence score to identify the target vehicle from the candidate vehicles; The execution module is configured to perform trajectory tracking of the target vehicle in a continuous image sequence and obtain the temporal state information corresponding to the target vehicle. The timing state information is input into a timing prediction network based on a gated cyclic unit to predict the optimal future beam for the target vehicle at a future time, and beam switching is performed based on the optimal future beam.

9. A computer-readable storage medium storing a computer program, characterized in that, The computer program is used to execute the beam management method for environmental semantic-assisted communication as described in any one of claims 1-7.

10. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the beam management method for environmental semantic-assisted communication as described in any one of claims 1-7.

Images