Millimeter wave beam prediction method of industrial wireless system assisted by multi-modal environment semantics

Abstract

The invention relates to an industrial wireless system millimeter wave beam prediction method assisted by multi-mode environment semantics, and belongs to the technical field of industrial wireless communication. According to the method, for a millimeter wave wireless communication process from a base station to a user in a factory, absolute position environment semantics of the user in a global positioning system and relative position environment semantics relative to the base station are extracted and obtained through a global positioning module deployed in the user and a camera of the base station; and inputting into a lightweight efficient neural network prediction model, and finally outputting a beam index to complete millimeter wave beam prediction. According to the method, the actual communication process of the millimeter wave communication system in the industrial wireless environment is considered, the efficient prediction of the beam is realized on the premise of low overhead of system resources, and the requirements of high-mobility and low-delay industrial wireless communication scenes are met.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial wireless communications and relates to a millimeter wave beam prediction method for an industrial wireless system assisted by multimodal environment semantics. Background Art

[0002] Millimeter waves, with their advantages of high-speed data transmission, low latency, low interference, and large-scale device connectivity, provide strong support for the advent of the Industrial 4.0 era. The beam prediction mechanism aims to guide and control the state of the antenna array through accurate prediction based on the current environment of the communication target, complete directional reception or transmission of signals, and achieve efficient and reliable signal transmission. It is one of the key technologies to ensure the performance of millimeter wave communication systems.

[0003] In industrial wireless systems such as 5G / 6G, considering the high mobility and high reliability application scenarios in the industrial field, it is necessary to ensure sufficient signal reception and transmission power to adapt to the complex industrial communication environment and meet the stable industrial communication needs. Therefore, large antenna arrays are a key component in this communication system. However, as the number of antennas increases, the number of beamforming vectors pointing in different directions also increases, and the training and adjustment of these vectors requires a lot of system overhead. Therefore, how to use semantic information such as vision and location in the communication environment to invent an efficient beam prediction method suitable for millimeter wave communication in industrial wireless systems, while reducing system overhead and avoiding high communication delays, thereby supporting application scenarios with high mobility and low latency such as vehicles and robots in factories, has become an important challenge facing the development of the field of industrial wireless communications. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a millimeter wave beam prediction method for an industrial wireless system assisted by multimodal environmental semantics, which realizes lightweight and high-precision millimeter wave beam prediction in industrial scenarios by extracting and utilizing various environmental semantics in communication scenarios and combining them with efficient prediction models.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for millimeter wave beam prediction of an industrial wireless system assisted by multimodal environment semantics, specifically comprising the following steps:

[0007] S1: Based on the millimeter wave wireless communication process and channel characteristics, a millimeter wave wireless communication model between users and base stations in the factory is constructed, the downlink signal is calculated, and the optimal beam index is obtained by combining the millimeter wave beamforming theory;

[0008] S2: By installing a global positioning module on the user side and deploying a camera on the base station side, the original parameters of the user environment are obtained in real time, and data processing and target extraction are performed on them, thereby obtaining the user's relative position environment semantics and absolute position environment semantics;

[0009] S3: For the two user environment semantics obtained in step S2, define the task content of beam prediction, determine the optimization goal of machine learning, obtain the beam prediction optimization formula, and build a lightweight and efficient neural network model based on different user environment semantics, output the predicted beam index, and guide beamforming.

[0010] Further, in step S1, a millimeter wave wireless communication model between users in the factory and the base station is constructed, which specifically includes the following steps:

[0011] S11: Consider a vehicle-infrastructure communication scenario consisting of a mobile user (vehicle), a base station and a related environment, where the base station is equipped with an M-element linear antenna array for receiving user signals and an RGB camera for capturing user images, and the mobile user (vehicle) is equipped with a single antenna transmitter for sending signals to the base station and a GPS receiver for locating the current mobile user (vehicle) position; the entire communication process and communication system operate in the millimeter wave band and use an OFDM transmission method based on K subcarriers. At the same time, the base station uses a predefined beamforming codebook in Q represents the number of beamforming vectors; let the transmission channel of the kth subcarrier at time t be The transmission content is x, and there is Gaussian white noise v in the channel k [t], then the signal y received by the downlink at time t k [t] is expressed by the following formula:

[0012]

[0013] S12: In order to ensure stable transmission of downlink signals, a beamforming vector that can maximize the received signal-to-noise ratio (SNR) should be selected, which is uniquely indicated by the index i[t]∈{1,...,Q} in the predefined codebook. The optimal beam index i ＊ [t] can be expressed by the following formula:

[0014]

[0015] Wherein, K represents the number of subcarriers.

[0016] Further, step S2 specifically includes the following steps:

[0017] S21: The global positioning module installed on the user side continuously obtains the coordinates of the user in the global positioning system at a certain moment.pos = [la,lo], as the original data of the user environment, where la and lo represent the longitude and latitude of the user respectively;

[0018] Through the camera deployed at the base station, the user's environment is continuously captured at a certain moment. As the original data of the user environment, C, H, and W represent the number of channels, height, and width of the photo, respectively;

[0019] S22: Obtained user coordinate position data g pos , perform minimum-maximum normalization operations to ensure that the data is scaled to a smaller range to improve the training effect of the neural network model, making the model converge faster, and finally output the user's position vector at time t

[0020] S23: For the obtained user environment photo r img , the target recognition module is used to identify the user and extract the user bounding box, and the mask image of the user at time t is output respectively and the bounding box center vector

[0021] S24: Finally, the semantics of the environment where the user is at time t is obtained It is expressed as:

[0022]

[0023] Among them, m[t] and b[t] are the user's relative position environment semantics, indicating the user's absolute position in the global positioning system; g[t] is the user's absolute position environment semantics, indicating the user's relative position relative to the base station.

[0024] Further, step S3 specifically includes the following steps:

[0025] S31: To obtain the optimal beam index i in step S12 ＊ [t], the task of beam prediction is defined as the mapping function f Θ , based on the semantics extracted from the user’s environment at time t Real-time estimation of the best beam index The whole process is expressed as:

[0026]

[0027] Among them, Θ represents the relevant parameters constituting the mapping function;

[0028] S32: To obtain the mapping function f in S31 Θ, adopts a machine learning model and trains it using a dataset that contains both multi-modal sensor data and communication data. Represents a dataset consisting of feature-labeled samples, where J is the total number of available samples; the goal of machine learning is to maximize the probability of accurate prediction of a given sample, and the whole process is expressed as:

[0029]

[0030] in, represents the probability of an accurate prediction;

[0031] S33: Based on the beam prediction task defined in S31 and the machine learning objectives in S32, an improved LeNet model based on the convolutional block attention module (CBAM) is constructed, referred to as the CBAM-LeNet model; the user mask image in the user relative position environment semantics Input it into the CBAM-LeNet model, modify the output layer dimension of the LeNet model, and connect it to the merging module;

[0032] S34: Bounding box center vector in user relative position environment semantics and the position vector in the user's absolute position environment semantics It is merged with the output of the CBAM-LeNet model in step S33 in the merging module and input into the constructed six-layer fully connected network, and finally the predicted beam index is output Guides the beamforming process.

[0033] Further, in step S3, the constructed lightweight and efficient neural network model includes a CBAM-LeNet model, a merging module, and a six-layer fully connected network;

[0034] The CBAM-LeNet model embeds the convolutional attention module CBAM between the first convolution-pooling layer and the second convolution-pooling layer of the LeNet network, thereby increasing the model's attention to the image space and channels, and then modifies the output layer dimension of the LeNet model and connects it to the merging module;

[0035] The merging module merges the bounding box center vector b[t] in the user's relative position environment semantics and the position vector g[t] in the user's absolute position environment semantics with the output of the CBAM-LeNet model, and inputs them into the constructed six-layer fully connected network.

[0036] The beneficial effects of the present invention are that the present invention can significantly reduce the computational complexity and storage overhead of beam selection in industrial wireless systems, improve beam alignment accuracy and reduce beam energy efficiency loss, and has better adaptability to scenarios with high mobility characteristics and low latency requirements in industrial wireless communications.

[0037] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0039] Figure 1 A schematic diagram of user semantic extraction in millimeter wave wireless communication scenarios for industrial wireless systems;

[0040] Figure 2 This is a schematic diagram of the neural network model beam prediction;

[0041] Figure 3 This is a flow chart of the millimeter wave beam prediction method for an industrial wireless system assisted by multimodal environment semantics of the present invention. DETAILED DESCRIPTION

[0042] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0043] See also Figure 1 to Figure 3 The present invention provides a millimeter wave beam prediction method for an industrial wireless system assisted by multimodal environmental semantics. For millimeter wave wireless communication scenarios in industrial scenarios, the method identifies and extracts original environmental data including user photos taken by base stations and GPS positioning information, thereby obtaining the user's relative position environmental semantics and absolute position environmental semantics, inputs them into a neural network model, and outputs a beam index uniquely corresponding to a beamforming vector, thereby ultimately guiding the beamforming process of the millimeter wave, realizing directional emission of the beam, and completing the communication process.

[0044] Figure 1 This is a schematic diagram of user semantic extraction in the millimeter wave communication scenario of industrial wireless system. Figure 1 As shown in the figure, the base station provides a millimeter-wave-based wireless communication service for a mobile user (vehicle) in a factory scene, where the factory scene includes roads, trees, factory buildings, workshops and other related facilities; the base station is equipped with an RGB camera to continuously capture user images and an M-element uniform antenna array to receive millimeter-wave signals transmitted by the user; the mobile user (vehicle) is equipped with a GPS receiver to access the global positioning service to obtain positioning information and a single-antenna transmitter to transmit millimeter-wave signals to the base station; the global positioning module installed at the user end is used to continuously obtain the coordinates of the user in the global positioning system at a certain moment g pos , and perform minimum-maximum normalization operations on it to ensure that the data is scaled to a smaller range, and finally output the user's position vector g[t] at time t, which represents the user's absolute position environment semantics; through the camera deployed at the base station, continuously take pictures to obtain the user's environment photos r at a certain moment img The target recognition module is used to identify the user and extract the user bounding box, and the mask image m[t] and the bounding box center vector b[t] of the user at time t are output respectively, which represent the user's relative position environment semantics.

[0045] Figure 2 Schematic diagram of the neural network model beam prediction. For the user mask image m[t] in the user relative position environment semantics, it is input into the improved LeNet model based on the attention mechanism (CBAM-LeNet). The model embeds the attention module CBAM between the first convolution-pooling layer and the second convolution-pooling layer of the LeNet network, thereby increasing the model's attention to the image space and channel, and then modifies the output layer dimension of the LeNet model and connects it to the merging module; for the bounding box center vector b[t] in the user relative position environment semantics and the position vector g[t] in the user absolute position environment semantics, it is merged with the output of the CBAM-LeNet model in the merging module and input into the constructed six-layer fully connected network, and finally outputs the predicted optimal beam index.

[0046] Figure 3 The flowchart of the millimeter wave beam prediction method of the industrial wireless system assisted by multimodal environment semantics of the present invention specifically includes the following steps:

[0047] V1: The beam prediction process starts.

[0048] V2: Establish a millimeter-wave based wireless communication system model in industrial scenarios.

[0049] V3~V4: The base station continuously captures the user image r img At the same time, the user obtains the coordinate position g through the GPS receiver pos , as the original data of the user environment.

[0050] V5~V9: Determine whether it is GPS coordinate data g pos If it is, perform the minimum-maximum normalization operation on it and output the user position vector g[t] as the user's absolute position environment semantics; if it is not GPS coordinate data, it is the user's picture data r img , then target recognition is performed on it, and the user mask image m[t] and the user bounding box center vector b[t] are output as the user relative position environment semantics.

[0051] V10: Merge relative position environment semantics and absolute position environment semantics to obtain user environment semantics

[0052] V11~V13: Determine whether it is the user position vector g[t] or the bounding box vector b[t]. If so, it is input into the six-layer fully connected network. If not, it is the mask image m[t], which is input into the CBAM-LeNet model, and the output of the model is input into the six-layer fully connected network.

[0053] V14: Output predicted best beam index

[0054] V15: The beam prediction process ends.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A millimeter wave beam prediction method for industrial wireless systems assisted by multimodal environment semantics, characterized in that: The method specifically comprises the following steps: S1: Based on the millimeter wave wireless communication process and channel characteristics, a millimeter wave wireless communication model between users and base stations in the factory is constructed, the downlink signal is calculated, and the optimal beam index is obtained by combining the millimeter wave beamforming theory; S2: By installing a global positioning module on the user side and deploying a camera on the base station side, the original parameters of the user environment are obtained in real time, and data processing and target extraction are performed on them, thereby obtaining the user's relative position environment semantics and absolute position environment semantics; S3: For the two user environment semantics obtained in step S2, define the task content of beam prediction, determine the optimization goal of machine learning, obtain the beam prediction optimization formula, and build a lightweight and efficient neural network model based on different user environment semantics, output the predicted beam index, and guide beamforming.

2. The millimeter wave beam prediction method for an industrial wireless system according to claim 1, characterized in that: In step S1, a millimeter wave wireless communication model between users in the factory and the base station is constructed, which specifically includes the following steps: S11: Consider a vehicle-to-infrastructure communication scenario consisting of a mobile user, a base station, and a related environment. The base station is equipped with an M-element linear antenna array for receiving user signals and an RGB camera for capturing user images. The mobile user is equipped with a single antenna transmitter for sending signals to the base station and a GPS receiver for locating the current mobile user position. The entire communication process and communication system operate in the millimeter wave band and use an OFDM transmission method based on K subcarriers. At the same time, the base station uses a predefined beamforming codebook. in Q represents the number of beamforming vectors; let the transmission channel of the kth subcarrier at time t be The transmission content is x, and there is Gaussian white noise v in the channel k [t], then the signal y received by the downlink at time t k [t] is expressed by the following formula: S12: In order to ensure stable transmission of downlink signals, a beamforming vector that can maximize the received signal-to-noise ratio should be selected, which is uniquely indicated by the index i[t]∈{1,...,Q} in the predefined codebook. The optimal beam index i ＊ [t] is expressed by the following formula: Wherein, K represents the number of subcarriers.

3. The millimeter wave beam prediction method for an industrial wireless system according to claim 2, characterized in that: Step S2 specifically includes the following steps: S21: The global positioning module installed on the user side continuously obtains the user's coordinates in the global positioning system at a certain moment. pos = [la,lo], as the original data of the user environment, where la and lo represent the longitude and latitude of the user respectively; Through the camera deployed at the base station, the user's environment is continuously captured at a certain moment. As the original data of the user environment, C, H, and W represent the number of channels, height, and width of the photo, respectively; S22: Obtained user coordinate position data g pos , perform minimum-maximum normalization operations to ensure that the data is scaled to a small range to improve the training effect of the neural network model, making the model converge faster, and finally output the user's position vector at time t S23: For the obtained user environment photo r img , the target recognition module is used to identify the user and extract the user bounding box, and the mask image of the user at time t is output respectively and the bounding box center vector S24: Finally, the semantics of the environment where the user is at time t is obtained It is expressed as: Among them, m[t] and b[t] are the user's relative position environment semantics, indicating the user's absolute position in the global positioning system; g[t] is the user's absolute position environment semantics, indicating the user's relative position relative to the base station.

4. The millimeter wave beam prediction method for an industrial wireless system according to claim 3, characterized in that: Step S3 specifically includes the following steps: S31: To obtain the optimal beam index i in step S12 ＊ [t], the task of beam prediction is defined as the mapping function f Θ , based on the semantics extracted from the user’s environment at time t Real-time estimation of the best beam index The whole process is expressed as: Among them, Θ represents the relevant parameters constituting the mapping function; S32: To obtain the mapping function f in S31 Θ , adopts a machine learning model and trains it using a dataset that contains both multi-modal sensor data and communication data. Represents a dataset consisting of feature-labeled samples, where J is the total number of available samples; the goal of machine learning is to maximize the probability of accurate prediction of a given sample, and the whole process is expressed as: in, represents the probability of an accurate prediction; S33: Based on the beam prediction task defined in S31 and the machine learning goal in S32, an improved LeNet model based on convolutional block attention module is constructed, referred to as CBAM-LeNet model, where CBAM stands for convolutional attention module; user mask image in the user relative position environment semantics Input it into the CBAM-LeNet model, modify the output layer dimension of the LeNet model, and connect it to the merging module; S34: Bounding box center vector in user relative position environment semantics and the position vector in the user's absolute position environment semantics It is merged with the output of the CBAM-LeNet model in step S33 in the merging module and input into the constructed six-layer fully connected network, and finally the predicted beam index is output Guides the beamforming process.

5. The millimeter wave beam prediction method for an industrial wireless system according to claim 4, characterized in that: In step S3, the constructed lightweight and efficient neural network model includes a CBAM-LeNet model, a merging module, and a six-layer fully connected network; The CBAM-LeNet model embeds the convolutional attention module CBAM between the first convolution-pooling layer and the second convolution-pooling layer of the LeNet network, thereby increasing the model's attention to the image space and channels, and then modifies the output layer dimension of the LeNet model and connects it to the merging module; The merging module merges the bounding box center vector b[t] in the user's relative position environment semantics and the position vector g[t] in the user's absolute position environment semantics with the output of the CBAM-LeNet model, and inputs them into the constructed six-layer fully connected network.

Images