Image semantic feature and pilot signal mapping method and device based on reinforcement learning

Abstract

The application discloses a kind of based on reinforcement learning's image semantic feature and pilot signal mapping method and device, obtain the important element of the semantic feature of image to be sent and the non-uniform pilot pattern of communication system;With the layout of non-uniform pilot pattern and important element as state, with the position of important element in non-uniform pilot pattern as action, important element and non-uniform pilot pattern are established using reinforcement learning method mapping relationship;Wherein, the reward factor of action in reinforcement learning method is calculated according to the distance reward of important element and nearest pilot, the density reward of pilot density in the range where important element is located, and the mutual information value of important element;The application effectively improves the transmission reliability of high-value semantic information, solves the problem that semantic perception and pilot allocation are disconnected in traditional communication system.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of semantic communication, and particularly relates to an image semantic feature and pilot signal mapping method and device based on reinforcement learning. BACKGROUND

[0002] With the evolution of 6G communication technology towards semantic communication, the traditional "bit-centered" communication system faces significant challenges in the transmission of image, video and other semantic information. Traditional pilot allocation schemes usually adopt a uniform allocation mode, and only design pilot sequences from the perspective of physical layer channel estimation requirements, without considering the differences in semantic features extracted by the sending end, resulting in low image reconstruction quality of key semantic symbols such as faces and edges in the image due to the mismatch between pilot resource configuration and semantic importance.

[0003] Although the current pilot optimization scheme based on reinforcement learning can dynamically adjust the pilot density according to the channel state, it lacks semantic layer feature perception ability and cannot establish a correlation between semantic symbol importance and pilot allocation strategy, making it difficult to guarantee the transmission reliability of high-value semantic regions.

[0004] At the same time, in the existing semantic communication system, the non-uniform pilot pattern design mostly adopts a fixed mode, without dynamic optimization combining fading channel data information and reinforcement learning algorithm, which is difficult to adapt to the fast fading and short time-varying characteristics of 6G millimeter wave channels, resulting in the inability of important semantic symbols to be effectively mapped to pilot dense areas, and the recovery accuracy of key semantic data at the receiving end is severely affected. At present, there is a lack of cross-layer coordination mechanism between semantic layer feature extraction, importance evaluation and physical layer pilot allocation, which has become the core bottleneck restricting the realization of "semantic fidelity" and "resource efficiency" collaborative optimization of the semantic communication system. SUMMARY

[0005] The purpose of the present application is to provide an image semantic feature and pilot signal mapping method and device based on reinforcement learning, to establish a correlation between pilots and semantic features, and to realize the joint optimization of semantic fidelity and resource efficiency.

[0006] The present application discloses an image semantic feature and pilot signal mapping method based on reinforcement learning, comprising the following steps:

[0007] Obtain the important elements of the semantic features of the image to be sent and the non-uniform pilot pattern of the communication system;

[0008] Take the layout of the non-uniform pilot pattern and the important elements as the state, and the position of the important elements in the non-uniform pilot pattern as the action, and use a reinforcement learning method to establish the mapping relationship between the important elements and the non-uniform pilot pattern;

[0009] In reinforcement learning, the reward factor for an action is calculated based on the distance reward between the important element and the nearest pilot, the density reward of the pilot density within the range of the important element, and the mutual information value of the important element.

[0010] Furthermore, the reward factor is calculated as follows:

[0011] The base reward is calculated based on distance reward, density reward, and mutual information value;

[0012] When the number of pilots within the range of an important element is greater than the pilot number threshold and the distance between the important element and the nearest pilot is less than or equal to the distance threshold, the product of the basic reward and the additional reward factor is used as the final reward factor; where the additional reward factor is greater than 1.

[0013] Furthermore, when the number of pilots within the range of an important element is less than the pilot number threshold or the distance between the important element and the nearest pilot is greater than the distance threshold, the basic reward will be used as the final reward factor.

[0014] Furthermore, the basic reward calculation method is as follows:

[0015] R base =I(y)×(R) d +R de ),

[0016] Among them, R base Let I(y) represent the basic reward, I(y) represent the mutual information value of the important element y, and R represent the basic reward. d R represents the distance reward. de This indicates a density reward.

[0017] Furthermore, distance rewards are based on Calculate; where D min This indicates the distance between the important element and the nearest pilot.

[0018] Furthermore, density rewards are based on Calculate; where r represents the radius of the region containing important elements in the non-uniform pilot pattern, and N pilot This indicates the number of pilot frequencies within the range of important elements.

[0019] Furthermore, the method for generating non-uniform pilot patterns is as follows:

[0020] Acquire the time-frequency response matrix of the noisy channel;

[0021] The time-frequency response matrix is ​​input into the encoder in the trained autoencoder network to obtain a non-uniform pilot pattern.

[0022] Furthermore, the encoder includes an attention fusion module, which consists of a context extraction submodule, a factor prediction submodule, and a feature recalibration submodule connected in sequence.

[0023] The context extraction submodule extracts global information from the feature map through global average pooling and concatenates it with the channel signal-to-noise ratio to form context information.

[0024] The factor prediction submodule uses two fully connected layers combined with ReLU and Sigmoid activation functions to predict channel-wise scaling factors based on context information.

[0025] The feature recalibration submodule is used to multiply the output features of the feature learning module by a scaling factor to generate weighted attention features.

[0026] Furthermore, the loss function of the autoencoder network is:

[0027]

[0028] in, Let H denote the loss function of the autoencoder network, K denote the number of samples of the time-frequency response matrix, k∈K, H k E represents the set of time-frequency response matrices after K samplings. θ (·) represents the encoder, θ represents the encoder parameters, and D φ (·) represents the decoder, φ represents the decoder parameters, and ||·|| F Let f denote the Frobenius norm, λ denote the regularization parameter, vec(·) denote the matrix vectorization operation, and |||1 denote the 1-norm.

[0029] The present invention also discloses an image semantic feature and pilot signal mapping device based on reinforcement learning, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the above-mentioned image semantic feature and pilot signal mapping method based on reinforcement learning.

[0030] The beneficial effects of this invention are: by using reinforcement learning to map important elements of semantic features to the non-uniform pilot pattern of the communication system, this invention can achieve intelligent optimization allocation of pilot resources, effectively improve the transmission reliability of high-value semantic information, and construct a cross-layer collaborative optimization framework from physical layer resource allocation to application layer semantic fidelity, thus solving the problem of the disconnect between semantic perception and pilot allocation in traditional communication systems. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the semantic communication system framework in an embodiment of the present invention;

[0032] Figure 2 forFigure 1 Flowchart of OFDM transmission and OFDM reception in China;

[0033] Figure 3 This is a schematic diagram of the deep joint source-channel codec network architecture based on the attention mechanism implemented in this invention.

[0034] Figure 4 This is a schematic diagram of the framework of the attention fusion module in an embodiment of the present invention;

[0035] Figure 5 This is a schematic diagram of the RCB module in an embodiment of the present invention;

[0036] Figure 6 This is a schematic diagram of the RTCB module in an embodiment of the present invention;

[0037] Figure 7 This is a schematic diagram of the structure of the self-encoder with a non-uniform pilot pattern in an embodiment of the invention.

[0038] Figure 8 The graph shows the performance analysis of peak signal-to-noise ratio (PSNR) versus signal-to-noise ratio for different methods used in the embodiments of the present invention.

[0039] Figure 9 The graph shows the performance analysis of multi-scale structural similarity (MS-SSIM) versus signal-to-noise ratio using different methods in the embodiments of the present invention. Detailed Implementation

[0040] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0041] In existing semantic communication systems, non-uniform pilot pattern designs mostly adopt fixed patterns and do not combine fading channel data information with reinforcement learning algorithms for dynamic optimization. This makes it difficult to adapt to the fast fading and short time-varying characteristics of 6G millimeter wave channels, resulting in important semantic symbols not being effectively mapped to dense pilot areas, and severely affecting the recovery accuracy of key semantic data at the receiving end.

[0042] Currently, the lack of a cross-layer collaborative mechanism between semantic layer feature extraction, importance assessment, and physical layer pilot allocation has become a core bottleneck restricting the synergistic optimization of semantic fidelity and resource efficiency in semantic communication systems. Therefore, how to construct a cross-layer collaborative mechanism between semantic features and pilot allocation to achieve joint optimization of semantic fidelity and resource efficiency has become a technical problem to be solved.

[0043] To address the problem of semantic features being disconnected from pilot allocation in existing semantic communication systems, and the inability to balance transmission reliability and resource efficiency, this invention proposes a non-uniform pilot semantic communication system design based on reinforcement learning. The aim is to achieve highly reliable transmission of important semantic symbols and efficient utilization of pilot resources through cross-layer collaborative optimization.

[0044] This invention discloses a method for mapping image semantic features to pilot signals based on reinforcement learning, comprising the following steps: acquiring important elements of the semantic features of the image to be transmitted and the non-uniform pilot pattern of the communication system; using the layout of the non-uniform pilot pattern and important elements as the state and the position of the important elements in the non-uniform pilot pattern as the action, establishing a mapping relationship between important elements and the non-uniform pilot pattern using a reinforcement learning method; wherein, the reward factor of the action in the reinforcement learning method is calculated based on the distance reward between the important element and the nearest pilot, the density reward of the pilot density within the range of the important element, and the mutual information value of the important element.

[0045] Specifically, this invention discloses a design for a non-uniform pilot semantic communication system based on reinforcement learning. The communication system framework of this invention adopts a cross-layer collaborative design, such as... Figure 1 and Figure 2 As shown, the system is divided into three main parts: the transmitter, the transmission channel, and the receiver. It forms a closed-loop optimization link through semantic feature extraction, reinforcement learning pilot decision-making, non-uniform pilot transmission, and semantic reconstruction feedback. Breaking through the limitations of traditional layered communication designs, the system achieves end-to-end collaborative optimization from physical layer resource allocation to application layer semantic fidelity. Its core innovation lies in the deep integration of semantic features and reinforcement learning, dynamically adapting to the semantic importance of the information source and the time-varying characteristics of the channel, thereby improving the reliability of high-value information transmission.

[0046] First, the original image X is input and converted into semantic features Y using a Joint Source Channel Coding (JSCC) encoder:

[0047] Y = f(X,θ) (1)

[0048] Where θ represents the trainable parameters of the JSCC encoder f(·).

[0049] The correlation between semantic features is calculated using mutual information, and the formula is as follows:

[0050]

[0051] Where y represents an element of the semantic features of the image, x represents a pixel in an image, P(x,y) is the joint probability of taking values ​​of both x and y, and p(x) and p(y) are their respective marginal probability densities. By calculating the mutual information values ​​between each symbol and the key semantic concept, the important elements in the important semantic features are identified.

[0052] In this communication mode, multiple images are usually transmitted at once, and each image has its own semantic features. After calculating the mutual information value of the elements of the semantic features, the multiple elements are sorted according to the size of the mutual information value, and finally the mutual information value greater than the mutual information value threshold is selected as the important element of the image.

[0053] In communication systems, it is also necessary to generate the system's non-uniform pilot pattern. Specifically, the method for generating the non-uniform pilot pattern is as follows: acquire the time-frequency response matrix of the noisy channel; input the time-frequency response matrix into the encoder in the trained autoencoder network to obtain the non-uniform pilot pattern.

[0054] As one implementation method, in this invention, the time-frequency response matrix of the noisy channel is H, and the set of time-frequency response matrices H is obtained by sampling it K times. k ={H1,H2,...,H K Then, it is fed into the autoencoder network.

[0055] The self-encoder network consists of encoder E. θ (·) and decoder D φ Composed of (·), it minimizes the reconstruction error by optimizing the objective, and its loss function is:

[0056]

[0057] in, Let H denote the loss function of the autoencoder network, K denote the number of samples of the time-frequency response matrix, k∈K, H k E represents the set of time-frequency response matrices after K samplings. θ (·) represents the encoder, θ represents the encoder parameters, and D φ (·) represents the decoder, φ represents the decoder parameters, and ||·|| F λ is the Frobenius norm, λ is the regularization parameter, vec(·) is the matrix vectorization operation, and |||1 represents the 1 norm.

[0058] By training an autoencoder network, channel feature compression and reconstruction are learned, and a non-uniform pilot pattern ρ∈{0,1} is output. M×N , where M represents the time domain, N represents the frequency domain, and ρ(m,n)=1 indicates that a pilot signal is inserted at the time domain symbol m and the frequency domain subcarrier n.

[0059] The deep joint source-channel autoencoder network architecture based on the attention mechanism proposed in this invention is as follows: Figure 3 As shown, it mainly consists of a Feature Learning (FL) module and an Attention Fusion (AFB) module, with RCB and RTCB being the basic components for building the FL module.

[0060] The Feature Learning (FL) module consists of convolutional layers, GDN layers, and PReLU activation layers. Its function is to extract and transform features from the input image. In the encoder, it is responsible for converting image data into an abstract feature representation, realizing source coding; in the decoder, it reconstructs the received feature signals into an image, completing the recovery process after channel decoding.

[0061] like Figure 4 As shown, the attention fusion module (AFB) consists of a context extraction submodule, a factor prediction submodule, and a feature recalibration submodule connected in sequence, and is designed based on a channel-wise soft attention mechanism.

[0062] The context extraction submodule extracts global information from the feature map through global average pooling and concatenates it with the channel signal-to-noise ratio to form context information; the factor prediction submodule uses two fully connected layers combined with ReLU and Sigmoid activation functions to predict the scaling factor for each channel based on the context information; the feature recalibration submodule is used to multiply the output features of the feature learning module with the scaling factor to generate weighted attention features.

[0063] The overall function of AFB is to adaptively adjust the feature weights according to the channel SNR conditions, thereby enhancing the model's adaptability under different SNR environments.

[0064] like Figure 5 As shown, the RCB module, or residual convolutional block, helps to enhance the joint source-channel coding capability through residual structure, while improving the convergence speed of the model training process, enabling the model to learn image features more effectively and improving coding and decoding performance.

[0065] like Figure 6 As shown, the RTCB module, or Residual Transposed Convolutional Block, is mainly used in the decoder. The transposed convolution operation can restore low-dimensional feature data to high-dimensional image data, while the residual structure further optimizes this restoration process, helping the model to better reconstruct the original image and improve the quality of image reconstruction.

[0066] The architecture of the autoencoder network proposed in this invention is as follows: Figure 7 As shown, the Concrete-AE autoencoder consists of two parts: an encoder and a decoder. The encoder is a neural network used to compress the input data into a low-dimensional latent space representation. The Concrete-AE encoder uses a Concrete selector layer, with H as the input and m as the output vector. j (That is, the output of the j-th channel of the encoder), specifically:

[0067]

[0068] Where, α jThese are the importance weights of the features, initialized to very small positive numbers, and becoming sparser with training iterations; g j m is a d-dimensional random variable sampled from a Gumbel distribution; T is a temperature parameter, and as T approaches 0, m j There is a certain probability that it will become a one-hot vector. A specific feature is selected, where z represents the index of the feature, and α... z Let g represent the learnable importance weight of the z-th feature. z Let represent a random variable sampled from the Gumbel distribution for the z-th feature.

[0069] The encoder acts as a feature selector, generating random linear combinations of input features during the training phase. These combinations gradually converge smoothly to a discrete combination of Z features, identifying the Z pilot positions with the richest information content, and generating an approximately optimal pilot pattern for a specific channel model.

[0070] The decoder uses a conventional deep neural network (DNN) to reconstruct the low-dimensional features output by the encoder. The deviation between the channel estimation result from the decoder output and the original noisy channel is quantified by calculating the 2-norm of the difference. The average deviation is then calculated over all samples, and the loss function is minimized to update the encoder parameters α. j And decoder weights w, to optimize the autoencoder model.

[0071] After obtaining the key elements and the non-uniform pilot pattern, reinforcement learning can be used to map the two together.

[0072] In reinforcement learning, the agent continuously learns and makes decisions based on the set state space, action space, and reward function, thereby optimizing the pilot allocation strategy.

[0073] State space: The state space should be a vector containing multi-dimensional information, comprehensively reflecting the state of the communication system. The set denoted as S consists of important elements from the semantic features determined by mutual information values. sem The time-frequency response matrix H and the generated non-uniform pilot pattern ρ∈{0,1} are learned by the autoencoder network. M×N Then the state space at time t is defined as: s t =[s sem ,vec(ρ)]∈S, where S is the state space.

[0074] Action space: Action space includes s sem The relationship between vec(ρ) and the action selection strategy is as follows: When selecting an action, an ε-greedy strategy is employed. The action with the highest Q-value in the current state is selected with probability 1-ε; otherwise, an action is randomly selected with probability ∈ ε.

[0075] Reward function: The reward takes into account the distance to the nearest pilot (the closer the distance, the higher the reward), pilot density, and semantic importance. Here, factors such as distance and density are used to comprehensively measure the reward for semantic reconstruction, channel correlation, and resource utilization.

[0076] In this invention, the non-uniform pilot pattern is regarded as a rectangular coordinate system with time as the horizontal axis and subcarrier ordinal number as the vertical axis, and the above-mentioned distance is the Manhattan distance in this coordinate system.

[0077] The Q-learning algorithm approximates the optimal strategy by iteratively updating the Q-function:

[0078] Q new (s,a)←Q old (s,a)+α[R+γmax a′ Q old (s′,a′)-Q old (s,a)] (5)

[0079] Among them, Q old (s,a) is the value estimate before performing action a in the current state s, Q. new (s,a) is the updated value estimate, max a′ Q old (s′,a′) is the maximum value estimate before the update among all possible actions in the next state, R is the reward factor obtained by the current action, α is the learning rate, which controls the degree to which the newly obtained reward updates the old Q value, s′ is the next state after placing the data, γ represents the discount factor, and a′ represents the action corresponding to the next state. Through continuous interaction and updates, it converges to the optimal policy to maximize the long-term cumulative reward.

[0080] In this invention, the reward factor is calculated as follows: a basic reward is calculated based on distance reward, density reward, and mutual information value; when the number of pilots within the range of an important element is greater than a pilot number threshold and the distance between the important element and the nearest pilot is less than or equal to a distance threshold, the product of the basic reward and the additional reward factor is used as the final reward factor; wherein, the additional reward factor is greater than 1. When the number of pilots within the range of an important element is less than a pilot number threshold or the distance between the important element and the nearest pilot is greater than a distance threshold, the basic reward is used as the final reward factor.

[0081] The calculation of the reward factor R takes into account the placement location of important elements and the distance to the nearest pilot, the density of pilots, and the importance of important elements (i.e., mutual information value), and performs more detailed classification of pilot dense areas to calculate the reward.

[0082] In this embodiment of the invention, let (s,h) be the placement position of the important element d in the rectangular coordinate system, and let (s,h) be the pilot position closest to (s,h). p ,h p If ), then the distance between the two is D((s,h),(s) p ,h p ))=|ss p |+|hh p Distance reward is defined as: Where D min This indicates the distance to the nearest pilot (calculated using Manhattan distance).

[0083] Secondly, within a region of radius r centered at position (s,h), the number of pilots is N. pilot The density reward is Where r represents the radius of the region containing important elements in the non-uniform pilot pattern, and N pilot This indicates the number of pilot frequencies within the range of important elements.

[0084] The basic reward is the product of the mutual information value I(y) of the important element y and the sum of the distance reward and the density reward:

[0085] R base =I(d)×(R) d +R de (6)

[0086] Regarding the final reward, it is based on the number of pilot signals N. pilot Divide the rewards into different tiers. As a specific example, the following approach could be adopted:

[0087] If N pilot ≥8 (i.e., scenarios with denser pilots) and D min If ≤1, then the final reward R = β1 × R base β1 is the first additional reward factor.

[0088] If 7≤N pilot <8 and D min If ≤1, then R=β2×R base β2 is the second additional reward factor.

[0089] If 5≤N pilot <7 and D min If ≤1, then R=β3×R base β3 is the third additional reward factor. It should be noted that β1, β2, and β3 are all numbers greater than 1, and decrease sequentially.

[0090] If D is not satisfied min ≤1 or N pilot If <5, then R = Rbase .

[0091] Thus, by explicitly binding the "location of pilot-dense regions" with the rewards of "highly important data" through the reward function, Q-learning learns this binding relationship so that the placement of important data (location selection) gradually converges to pilot-dense regions. In other words, the final output is the association information between important elements of semantic features and pilot locations in the time-frequency grid.

[0092] In one embodiment, the receiver performs channel estimation and decodes the data using semantic index information, typically employing the least squares (LS) method. The channel estimation formula is as follows: Where Y′ is the received signal matrix, and ρ is the pilot matrix, i.e., the non-uniform pilot pattern. It is the pseudo-inverse of ρ. This represents the channel state information estimated using the least squares method. Simultaneously, it is combined with semantic index information from the transmitter, which indicates the location distribution of important elements in the semantic features. This information identifies the location of semantically important parts of the data. During decoding, based on the semantic index information, high-semantic-value parts are prioritized for processing and recovery, improving decoding accuracy and protecting important semantics.

[0093] The receiver feeds back the calculated semantic reconstruction metrics (such as PSNR and MS-SSIM values) to the reinforcement learning module at the transmitter. Based on this feedback, the reinforcement learning module adjusts the parameters θ of the policy network using the policy gradient method.

[0094]

[0095] Where J(θ) is the objective function, p(s) is the state distribution, and π θ (a|s) is the policy function, and R(s,a) is the reward value. The pilot allocation policy is continuously optimized through iteration to achieve cross-layer closed-loop optimization. It is the gradient operator with respect to the parameter θ. It expresses expectation.

[0096] Furthermore, to verify the effectiveness of the method of the present invention, experiments were conducted comparing the method of the present invention with existing methods, as well as categorizing the method of the present invention. In the experiments, the experimental conditions were identical for all methods; only the correlation methods used differed.

[0097] like Figure 8 As shown, the performance analysis of peak signal-to-noise ratio (PSNR) for different transmission methods under different signal-to-noise ratio conditions is presented.

[0098] The diamond pilot refers to the transmission of signals without distinguishing the importance of semantic symbols, and directly using the traditional diamond pilot pattern.

[0099] Entropy model + diamond pilot + sorting refers to using existing semantic feature measurement methods (entropy model) to determine the importance of semantic symbols, and combining it with traditional diamond pilot pattern transmission.

[0100] Mutual information model + diamond pilot + sorting, mutual information model + non-uniform pilot + sorting, and mutual information model + reinforcement learning + non-uniform pilot + sorting all use the mutual information theory model as the basis for judging the importance of semantic symbols, and then combine them with traditional diamond pilot patterns, non-uniform pilot patterns, and the combination of non-uniform pilot patterns and reinforcement learning for transmission.

[0101] from Figure 8 As can be seen, the performance index of image reconstruction quality gradually increases with the increase of signal-to-noise ratio (SNR). The method of this invention (i.e., mutual information model + reinforcement learning + non-uniform pilot + ranking), which is based on semantic communication implementation method combining reinforcement learning and non-uniform pilot, has a much higher performance than traditional methods, proving the effectiveness and feasibility of this invention.

[0102] like Figure 9 As shown in the figure, the performance analysis of MS-SSIM (Multi-Scale Structural Similarity) versus signal-to-noise ratio (SNR) for different transmission methods is presented. The comparison curves in the figure intuitively demonstrate that the communication system based on semantic features and reinforcement learning pilot allocation, under different SNRs, can more effectively preserve image structural features compared to traditional and other improved transmission methods, highlighting its performance advantages.

[0103] The present invention also discloses an image semantic feature and pilot signal mapping device based on reinforcement learning, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the above-mentioned image semantic feature and pilot signal mapping method based on reinforcement learning.

[0104] Those skilled in the art will understand that the steps described in the embodiments disclosed in this invention can be implemented either by electronic hardware or by a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application scenario and design constraints of the technical solution. Those skilled in the art can implement the above functions in different ways to meet different application needs, and such implementations should not be considered as departing from the scope of protection of this invention.

[0105] In the embodiments provided by this invention, the described method may also be implemented in other ways. For example, the description of the above embodiments is only illustrative in nature. Multiple modules or components may be integrated or integrated into other systems, and certain features or related steps may be omitted.

[0106] In summary, the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail based on the foregoing embodiments, those skilled in the art can still improve the technical solutions described in the embodiments or make equivalent substitutions for some technical features. As long as these improvements or substitutions do not cause the corresponding technical solutions to deviate from the core essence and protection scope of the technical solutions of the embodiments of the present invention, they should all be included within the protection scope of the present invention.

[0107] In the above embodiments, the descriptions of each embodiment have different focuses. If there are any parts in a certain embodiment that are not described in detail or recorded, you can refer to the relevant descriptions of other embodiments.

Claims

1. A method for mapping image semantic features and pilot signals based on reinforcement learning, characterized in that, The method comprises the following steps: obtaining an important element of a semantic feature of an image to be sent and a non-uniform pilot pattern of a communication system; using a reinforcement learning method to establish a mapping relationship between the important element and the non-uniform pilot pattern, taking the layout of the important element and the non-uniform pilot pattern as a state and the position of the important element in the non-uniform pilot pattern as an action; wherein the reward factor of the action in the reinforcement learning method is calculated according to a distance reward of the important element and the nearest pilot, a density reward of the pilot density in the range where the important element is located, and a mutual information value of the important element; the reward factor calculation method is: calculating a basic reward based on the distance reward, the density reward, and the mutual information value; when the number of pilots in the range where the important element is located is greater than a pilot number threshold and the distance between the important element and the nearest pilot is less than or equal to a distance threshold, taking the product of the basic reward and an additional reward factor as the final reward factor; wherein the additional reward factor is greater than 1; the basic reward calculation method is: ， wherein, denotes a base reward, denotes a mutual information value of the important element y, denotes a distance reward, denotes a density reward; The distance reward is calculated according to wherein, denotes the distance of the important element from the nearest pilot. the non-uniform pilot pattern generation method is: collecting a time-frequency response matrix of a noisy channel; inputting the time-frequency response matrix into an encoder in a trained autoencoder network to obtain the non-uniform pilot pattern; the encoder comprises an attention fusion module, which is composed of a context extraction submodule, a factor prediction submodule, and a feature recalibration submodule connected in sequence; the context extraction submodule extracts global information of a feature map through global average pooling and concatenates the channel signal-to-noise ratio to form context information; the factor prediction submodule uses two layers of fully connected layers combined with ReLU and Sigmoid activation functions to predict the scaling factor of each channel according to the context information; the feature recalibration submodule is used to multiply the output feature of the feature learning module by the scaling factor to generate weighted attention features. 2.The method of claim 1, wherein, when the number of pilots in the range where the important element is located is less than a pilot number threshold or the distance between the important element and the nearest pilot is greater than a distance threshold, taking the basic reward as the final reward factor. 3.The method of claim 2, wherein, The density reward is according to calculating; wherein, represents a radius of a range in which the important element is located in the non-uniform pilot pattern, represents a number of pilots in the range in which the important element is located. 4.The method of claim 3, wherein, the loss function of the autoencoder network is: ， wherein, denotes a loss function of the autoencoder network, K denotes a number of samples of the time-frequency response matrix, , denotes a set of K-sampled time-frequency response matrices, denotes an encoder, 0 denotes an encoder parameter, denotes a decoder, denotes a decoder parameter, denotes a Frobenius norm, denotes a regularization parameter, denotes a matrix vectorization operation, denotes a 1-norm.

5. An apparatus for mapping image semantic features and pilot signals based on reinforcement learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, the processor executes the computer program to realize the reinforcement learning-based image semantic feature and pilot signal mapping method according to any one of claims 1-4.

Images