A method and system for allocating communication system resources based on semantic information

Abstract

The present invention provides a method and system for allocating resources of a communication system based on semantic information, and relates to the technical field of wireless communication. The method provided by the present invention processes an image to be transmitted to obtain a semantic importance score of each semantic feature vector corresponding to the image to be transmitted; then determines the semantic importance weighted information volume of each semantic feature vector; sorts multiple semantic feature vectors from large to small according to the semantic importance weighted information volume to obtain a first sequence; and according to the first sequence, sequentially determines the channel with the smallest probability of communication interruption corresponding to each semantic feature vector, and can use a transmission information volume greater than the required transmission information volume. The method provided by the present invention can improve information transmission efficiency by determining the importance scores of multiple semantic vectors included in the image to be transmitted, and then allocating channels with low interruption probabilities to semantic vectors with high importance based on the importance scores of multiple semantic vectors.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a communication system resource allocation method and system based on semantic information. Background Art

[0002] Visible light communication (VLC) is a wireless communication technology that uses visible light electromagnetic bands for communication, which has a wide spectrum resource. Usually, the visible light communication system is based on multiple light-emitting diodes (LEDs) for communication. The data to be communicated is encoded in the light emitted by the LEDs, and the photodetector or image sensor is used to receive the modulated signal and decode the data.

[0003] However, during the use of the visible light communication system, since each light-emitting diode has different channel resources, it is necessary to allocate channel resources, that is, to allocate corresponding channel resources to the data to be transmitted in order to improve communication efficiency and channel utilization.

[0004] In related technologies, resource allocation is usually implemented based on pre-set constraints (such as illumination constraints, chromaticity constraints, amplitude constraints, and signal-to-noise ratio constraints). However, this approach does not take into account the importance and priority of information content, resulting in insufficient efficiency in application scenarios with high critical communication requirements, and there is a problem of low communication efficiency. In addition, resource allocation based on pre-set constraints has poor flexibility in application scenarios with intensive and rapidly changing transmission data. Summary of the invention

[0005] The embodiments of the present invention provide a method and system for allocating resources in a communication system based on semantic information. In a multi-color visible light communication system, resource allocation can be dynamically adjusted according to the semantic importance of data content, thereby improving communication efficiency, optimizing resource utilization, and enhancing the adaptability and response speed of the system.

[0006] To achieve the above object, the embodiments of the present invention adopt the following technical solutions:

[0007] In a first aspect, a method for allocating resources of a communication system based on semantic information is provided, which is applied to a communication system. The communication system includes multiple channels, each of which has a corresponding channel capacity and a communication interruption probability. The method includes: processing an image to be transmitted to obtain multiple semantic feature vectors corresponding to the image to be transmitted and a semantic importance score corresponding to each semantic feature vector; converting each semantic feature vector into a binary bit vector to obtain the amount of transmission information corresponding to each semantic feature vector; determining the product of the semantic importance score corresponding to each semantic feature vector and the amount of transmission information as the semantic importance weighted information amount of each semantic feature vector; sorting the multiple semantic feature vectors from large to small according to the semantic importance weighted information amount of each semantic feature vector to obtain a first sequence; according to the first sequence, determining in turn a unique channel corresponding to each semantic feature vector, wherein the unique channel is a channel with the smallest probability of communication interruption among the remaining multiple channels and whose usable transmission information amount is greater than the transmission information amount, wherein the usable transmission information amount of each channel is the difference between the channel capacity of each channel and the used transmission information amount.

[0008] In a possible implementation of the first aspect, the image to be transmitted is processed to obtain multiple semantic feature vectors corresponding to the image to be transmitted and a semantic importance score corresponding to each semantic feature vector, including: forward propagating the image to be transmitted based on a convolutional neural network to obtain multiple semantic feature vectors and a prediction confidence score corresponding to a target task category of each semantic feature vector, wherein the convolutional neural network includes x convolutional layers, and x is a positive integer; using the prediction confidence score corresponding to the target task category of each semantic feature vector as a loss function, performing back propagation to obtain a gradient corresponding to the xth convolutional layer among the x convolutional layers; determining a class activation map corresponding to the target task category based on the gradient corresponding to the xth convolutional layer and the feature map output by each convolutional layer; and normalizing the class activation map corresponding to the target task category to obtain a semantic importance score corresponding to each semantic feature vector.

[0009] In a possible implementation of the first aspect, when the unique channel corresponding to each semantic feature vector is determined in turn according to the first sequence, the above method includes: for any semantic feature vector, determining whether there is a channel among multiple channels whose available transmission information amount is greater than or equal to the transmission information amount of any semantic feature vector; in the case that there is a channel among multiple channels whose available transmission information amount is greater than the transmission information amount of any semantic feature vector, determining the target channel with the lowest probability of communication interruption among the channels whose available transmission information amount is greater than the transmission information amount of any semantic feature vector as the unique channel corresponding to any semantic feature vector; and updating the available transmission information amount of the target channel according to the difference between the available transmission information amount of the target channel and the transmission information amount of any semantic feature vector.

[0010] In a possible implementation manner of the first aspect, the method further includes: if there is no channel among multiple channels whose available transmission information amount is greater than or equal to the transmission information amount of any semantic feature vector, deleting any semantic feature vector.

[0011] In a possible implementation manner of the first aspect, the method further includes: when there is no channel among multiple channels that can use a transmission amount of information greater than a transmission amount of information of any semantic feature vector, storing any semantic feature vector in a cache queue.

[0012] In a possible implementation of the first aspect, when semantic feature vectors are stored in a cache queue, multiple semantic feature vectors are sorted from large to small according to the semantic importance weighted information amount of each semantic feature vector to obtain a first sequence, including: determining the importance weighted information amount corresponding to the semantic feature vectors stored in the cache queue; sorting multiple semantic feature vectors from large to small according to the semantic importance weighted information amount of each semantic feature vector and the importance weighted information amount corresponding to the semantic feature vectors stored in the cache queue to obtain the first sequence.

[0013] The beneficial effects of the present invention are as follows: the method provided by the embodiment of the present invention can ensure the transmission efficiency of important information by determining the importance scores of multiple semantic vectors included in the image to be transmitted, and then allocating channels with low interruption probability to semantic vectors with high importance based on the importance scores of multiple semantic vectors. It can also be understood that the method provided by the embodiment of the present invention can quickly and accurately determine the importance of the information corresponding to each semantic vector for the target task category, and dynamically select different channels for different semantic vectors according to the importance of the information, which can effectively reduce semantic distortion, thereby improving communication efficiency, optimizing resource utilization, and thus enhancing the adaptability and response speed of the system. In addition, the method provided by the embodiment of the present invention can solve the problem of low communication efficiency in application scenarios with high critical communication requirements, and has good flexibility in application scenarios with intensive and rapidly changing transmission data, and can meet the communication needs in different usage scenarios.

[0014] In a second aspect, an embodiment of the present invention provides a communication system, which includes: multiple channels, each channel having a corresponding channel capacity and a communication interruption probability; a semantic processing unit, which is used to process an image to be transmitted to obtain multiple semantic feature vectors corresponding to the image to be transmitted and a semantic importance score corresponding to each semantic feature vector; a vector conversion unit, which is used to convert each semantic feature vector into a binary bit vector to obtain the amount of transmission information corresponding to each semantic feature vector; an information determination unit, which is used to determine the product of the semantic importance score corresponding to each semantic feature vector and the amount of transmission information as the semantic importance weighted information amount of each semantic feature vector; a vector sorting unit, which is used to sort the multiple semantic feature vectors from large to small according to the semantic importance weighted information amount of each semantic feature vector to obtain a first sequence; a channel determination unit, which is used to determine the unique channel corresponding to each semantic feature vector in turn according to the first sequence, wherein the unique channel is the channel with the smallest probability of communication interruption among the remaining multiple channels and the amount of available transmission information is greater than the amount of transmission information, wherein the amount of available transmission information of each channel is the difference between the channel capacity of each channel and the amount of used transmission information.

[0015] In a possible implementation of the second aspect, the semantic processing unit is specifically used to: perform forward propagation on the image to be transmitted based on a convolutional neural network to obtain multiple semantic feature vectors and prediction confidence scores corresponding to the target task category of each semantic feature vector, wherein the convolutional neural network includes x convolutional layers, and x is a positive integer; use the prediction confidence score corresponding to the target task category of each semantic feature vector as a loss function, perform back propagation, and obtain the gradient corresponding to the xth convolutional layer among the x convolutional layers; determine the class activation map corresponding to the target task category based on the gradient corresponding to the xth convolutional layer and the feature map output by each convolutional layer; normalize the class activation map corresponding to the target task category to obtain the semantic importance score corresponding to each semantic feature vector.

[0016] In a possible implementation of the second aspect, when the channel determination unit is used to determine, in sequence according to the first sequence, the unique channel corresponding to each semantic feature vector, the channel determination unit is specifically used to: for any semantic feature vector, determine whether there is a channel among multiple channels whose available transmission information amount is greater than or equal to the transmission information amount of any semantic feature vector; in the case that there is a channel among multiple channels whose available transmission information amount is greater than the transmission information amount of any semantic feature vector, determine the target channel with the smallest probability of communication interruption among the channels whose available transmission information amount is greater than the transmission information amount of any semantic feature vector as the unique channel corresponding to any semantic feature vector; and update the available transmission information amount of the target channel according to the difference between the available transmission information amount of the target channel and the transmission information amount of any semantic feature vector.

[0017] In a possible implementation of the second aspect, the channel determination unit is further used to: delete any semantic feature vector if there is no channel among multiple channels whose available transmission information amount is greater than or equal to the transmission information amount of any semantic feature vector.

[0018] It can be understood that the beneficial effects that can be achieved by the system of the second aspect provided above can refer to the beneficial effects in the first aspect and any possible design thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of the architecture of a communication system provided by an embodiment of the present invention is shown;

[0020] Figure 2 A schematic diagram of the architecture of a resource allocation module provided by an embodiment of the present invention is shown;

[0021] Figure 3 A flow chart of a method for allocating resources in a communication system based on semantic information according to an embodiment of the present invention;

[0022] Figure 4 A flowchart of another communication system resource allocation method based on semantic information according to an embodiment of the present invention;

[0023] Figure 5 The present invention is a flowchart of another method for allocating resources in a communication system based on semantic information according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The technical solution in the embodiment of the present invention will be described below in conjunction with the accompanying drawings in the embodiment of the present invention. Among them, in the description of the present invention, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in the present invention is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. And, in the description of the present invention, unless otherwise specified, "multiple" means two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items.

[0025] In addition, in order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, the words "first", "second", etc. are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art can understand that the words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not necessarily limit the difference.

[0026] Meanwhile, in the embodiments of the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way for easy understanding.

[0027] Visible light communication (VLC) is a wireless communication technology that uses visible light electromagnetic bands for communication, which has a wide spectrum resource. Usually, the visible light communication system is based on multiple light-emitting diodes (LEDs) for communication. The data to be communicated is encoded in the light emitted by the LEDs, and the photodetector or image sensor is used to receive the modulated signal and decode the data.

[0028] However, during the use of the visible light communication system, since each light-emitting diode has different channel resources, it is necessary to allocate channel resources, that is, to allocate corresponding channel resources to the data to be transmitted in order to improve communication efficiency and channel utilization.

[0029] In related technologies, resource allocation is usually implemented based on pre-set constraints (such as illumination constraints, chromaticity constraints, amplitude constraints, and signal-to-noise ratio constraints). However, this approach does not take into account the importance and priority of information content, resulting in insufficient efficiency in application scenarios with high critical communication requirements, and there is a problem of low communication efficiency. In addition, resource allocation based on pre-set constraints has poor flexibility in application scenarios with intensive and rapidly changing transmission data.

[0030] In view of this, an embodiment of the present invention provides a communication system resource allocation method based on semantic information, which is applied to a communication system, wherein the communication system includes multiple channels, each of which has a corresponding channel capacity and a communication interruption probability. The above method includes: processing an image to be transmitted to obtain multiple semantic feature vectors corresponding to the image to be transmitted and a semantic importance score corresponding to each semantic feature vector; converting each semantic feature vector into a binary bit vector to obtain the amount of transmission information corresponding to each semantic feature vector; determining the product of the semantic importance score corresponding to each semantic feature vector and the amount of transmission information as the semantic importance weighted information amount of each semantic feature vector; sorting the multiple semantic feature vectors from large to small according to the semantic importance weighted information amount of each semantic feature vector to obtain a first sequence; according to the first sequence, determining the unique channel corresponding to each semantic feature vector in turn, wherein the unique channel is the channel with the smallest probability of communication interruption among the remaining multiple channels and the amount of available transmission information is greater than the amount of transmission information, wherein the amount of available transmission information of each channel is the difference between the channel capacity of each channel and the amount of used transmission information.

[0031] The method provided by the embodiment of the present invention can ensure the transmission efficiency of important information by determining the importance scores of multiple semantic vectors included in the image to be transmitted, and then allocating channels with low interruption probability to semantic vectors with high importance based on the importance scores of the multiple semantic vectors. It can also be understood that the method provided by the embodiment of the present invention can quickly and accurately determine the importance of the information corresponding to each semantic vector for the target task category, and dynamically select different channels for different semantic vectors according to the importance of the information, which can effectively reduce semantic distortion, thereby improving communication efficiency, optimizing resource utilization, and enhancing the adaptability and response speed of the system. In addition, the method provided by the embodiment of the present invention can solve the problem of low communication efficiency in application scenarios with high critical communication requirements, and has good flexibility in application scenarios with intensive and rapidly changing transmission data, and can meet the communication needs in different usage scenarios.

[0032] In some embodiments, the communication system resource allocation method based on semantic information provided in the embodiment of the present invention can be executed by the communication system 100. In the scenario where the communication system is a visible light communication system, Figure 1The schematic diagram of the architecture of a communication system provided by an embodiment of the present invention is shown. The communication system 100 includes a transmitting end 110 and a receiving end 120. The transmitting end 110 includes a resource allocation module 111, a transmitting module 112 and a plurality of light emitting diodes 113, and the receiving end 120 includes a receiving module 121 and a data processing module 122. Among them, the resource allocation module 111 is connected to the transmitting module 112, and the transmitting module 112 is connected to the plurality of light emitting diodes 113. The receiving module 121 is connected to the data processing module 122. The resource allocation module 111 is used to receive an image to be transmitted, and to determine the channels corresponding to the plurality of light emitting diodes 113 used to transmit the image to be transmitted, and the transmitting module 112 is used to send the binary bit vector corresponding to the image to be transmitted to the receiving module 121 through the channels corresponding to the plurality of light emitting diodes 113. The receiving module 121 is used to receive the binary bit vector corresponding to the image to be transmitted. The data processing module 122 is used to process the binary bit vector corresponding to the image to be transmitted to obtain the image to be transmitted.

[0033] Specifically, each of the plurality of light emitting diodes 113 corresponds to a channel, and each channel has a corresponding channel capacity and communication interruption probability. The channel capacity corresponding to each channel is used to characterize the maximum amount of data that can be transmitted per unit time by each channel, that is, the capacity of data that can be transmitted per unit time by each channel.

[0034] In a possible implementation, the channel capacity corresponding to each channel is the product of the theoretical channel capacity of each channel and a conservative factor, with the unit being bit / ms.

[0035] It should be noted that the embodiment of the present invention does not impose any particular restrictions on the specific determination method and unit of the channel capacity corresponding to each channel. For example, the unit of the channel capacity may also be Mb / ms.

[0036] It should be understood that the above visible light communication system is only an example. The communication system provided in the embodiment of the present invention may also be a radio frequency communication system, or a communication system with any multiple channels configured. The embodiment of the present invention does not impose any particular restrictions on the specific type and implementation method of the communication system.

[0037] For further information, see Figure 2 , Figure 2The schematic diagram of the architecture of a resource allocation module provided by an embodiment of the present invention is shown. The resource allocation module 111 includes a semantic processing unit 1111 , a vector conversion unit 1112 , an information determination unit 1113 , a vector sorting unit 1114 and a channel determination unit 1115 . The semantic processing unit 1111 is used to process the image to be transmitted to obtain multiple semantic feature vectors corresponding to the image to be transmitted and the semantic importance score corresponding to each semantic feature vector; the vector conversion unit 1112 is used to convert each semantic feature vector into a binary bit vector to obtain the transmission information amount corresponding to each semantic feature vector; the information determination unit 1113 is used to determine the product of the semantic importance score corresponding to each semantic feature vector and the transmission information amount as the semantic importance weighted information amount of each semantic feature vector; the vector sorting unit 1114 is used to sort the multiple semantic feature vectors from large to small according to the semantic importance weighted information amount of each semantic feature vector to obtain a first sequence; the channel determination unit 1115 is used to determine the unique channel corresponding to each semantic feature vector in turn according to the first sequence, wherein the unique channel is the channel with the smallest probability of communication interruption among the remaining multiple channels and the available transmission information amount is greater than the transmission information amount, wherein the available transmission information amount of each channel is the difference between the channel capacity of each channel and the used transmission information amount.

[0038] For example, the channel capacity of channel 1 is 300 bit / ms. After channel 1 is determined as the only channel of semantic feature vector A, the transmission data volume of semantic feature vector A is 10 bits. At this time, the used transmission information volume of channel 1 is 10 bits, and the available transmission information volume becomes 290 bit / ms.

[0039] Optionally, the semantic processing unit 1111 is specifically used to: perform forward propagation on the image to be transmitted based on a convolutional neural network to obtain multiple semantic feature vectors and prediction confidence scores corresponding to the target task category of each semantic feature vector, wherein the convolutional neural network includes x convolutional layers, and x is a positive integer; use the prediction confidence score corresponding to the target task category of each semantic feature vector as a loss function, perform back propagation, and obtain the gradient corresponding to the x-th convolutional layer among the x convolutional layers; determine the class activation map corresponding to the target task category based on the gradient corresponding to the x-th convolutional layer and the feature map output by each convolutional layer; normalize the class activation map corresponding to the target task category to obtain the semantic importance score corresponding to each semantic feature vector.

[0040] Optionally, the channel determination unit 1115 is further used to: determine whether there is a channel among multiple channels whose available transmission information amount is greater than or equal to the transmission information amount of any semantic feature vector, where any semantic feature vector is any one of the multiple semantic feature vectors included in the first sequence; in the case that there is a channel among multiple channels whose available transmission information amount is greater than the transmission information amount of any semantic feature vector, determine the target channel with the lowest probability of communication interruption among the channels whose available transmission information amount is greater than the transmission information amount of any semantic feature vector as the only channel corresponding to any semantic feature vector; and update the available transmission information amount of the target channel according to the difference between the available transmission information amount of the target channel and the transmission information amount of any semantic feature vector.

[0041] Optionally, the channel determination unit 1115 is further configured to: delete any semantic feature vector if there is no channel in the multiple channels whose available transmission information amount is greater than the transmission information amount of any semantic feature vector.

[0042] Optionally, the channel determination unit 1115 is further configured to: if there is no channel among the multiple channels whose available transmission information amount is greater than the transmission information amount of any semantic feature vector, store any semantic feature vector in a cache queue.

[0043] Optionally, when semantic feature vectors are stored in the cache queue, the vector sorting unit 1114 is specifically used to: determine the importance-weighted information amount corresponding to the semantic feature vectors stored in the cache queue; sort multiple semantic feature vectors from large to small according to the semantic importance-weighted information amount of each semantic feature vector and the importance-weighted information amount corresponding to the semantic feature vectors stored in the cache queue to obtain a first sequence.

[0044] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the communication system 100 or the resource allocation module 111. In other embodiments of the present invention, the communication system 100 or the resource allocation module 111 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0045] The following describes a method for allocating resources in a communication system based on semantic information provided by an embodiment of the present invention in conjunction with the accompanying drawings.

[0046] Figure 3 A flowchart of a communication system resource allocation method based on semantic information provided by an embodiment of the present invention. Optionally, the method may be Figure 1 The communication system 100 shown is executed by Figure 2 The resource allocation module 111 shown is executed. The method may include the following steps:

[0047] S1. Processing the image to be transmitted to obtain a plurality of semantic feature vectors corresponding to the image to be transmitted and a semantic importance score corresponding to each semantic feature vector.

[0048] The image to be transmitted is obtained by the information source and needs to be sent from the transmitting end 110 to the receiving end 120 through the communication system 100 so that the information sink connected to the receiving end can receive the image information.

[0049] Specifically, the resource allocation module 111 processes the image to be transmitted by using a gradient-weighted class activation mapping (Grad-CAM) method to obtain a class activation map (importance map) corresponding to the image to be transmitted, thereby obtaining a semantic importance score corresponding to each semantic feature vector.

[0050] It should be understood that for different task categories, the semantic importance scores corresponding to each semantic feature vector included in the image to be transmitted are different. The resource allocation module 111 determines the class activation map (importance map) corresponding to the image to be transmitted in the scenario of the target task category based on the target task category input by the user, thereby obtaining the semantic importance scores corresponding to each semantic feature vector. In other words, for the same semantic information, it has different semantic importances under different task categories. For example, the target task category can be intelligent driving, and the semantic importance scores corresponding to the semantic feature vectors of the areas corresponding to people and vehicles in the image to be transmitted are high. The target task category can also be pipeline defect detection, and the semantic importance scores corresponding to the semantic feature vectors of the defective areas of the pipeline in the image to be transmitted are high.

[0051] In one example, the image a to be transmitted is processed to obtain three semantic feature vectors corresponding to the image to be transmitted and the semantic importance scores corresponding to the three semantic feature vectors. The three semantic feature vectors are semantic feature vector A, semantic feature vector B, and semantic feature vector C. The semantic importance score of semantic feature vector A is 35, the semantic importance score of semantic feature vector B is 60, and the semantic importance score of semantic feature vector C is 25.

[0052] In some embodiments, see Figure 4 The above S1 specifically includes the following steps:

[0053] S11. Based on the convolutional neural network, forward propagation is performed on the image to be transmitted to obtain multiple semantic feature vectors and the prediction confidence score corresponding to the target task category of each semantic feature vector.

[0054] Among them, the convolutional neural network includes x convolutional layers, x is a positive integer;

[0055] Specifically, the image to be transmitted is input into the convolutional neural network, and forward propagation (i.e., semantic information extraction) is performed through the convolutional neural network to obtain multiple semantic feature vectors (SFVs) and the prediction confidence score corresponding to each semantic feature vector in the target task category. Among them, the target task category is the conditional parameter input by the user.

[0056] S12. Use the prediction confidence score corresponding to the target task category of each semantic feature vector as the loss function, perform back propagation, and obtain the gradient corresponding to the xth convolutional layer among the x convolutional layers.

[0057] Specifically, for the prediction confidence score y of the target task category c c , determine the feature map output by the xth convolutional layer among x convolutional layers The corresponding gradient

[0058] S13. Determine a class activation map corresponding to the target task category according to the gradient corresponding to the x-th convolutional layer and the feature map output by each convolutional layer.

[0059] Specifically, the feature map output by the xth convolutional layer is The corresponding gradient Perform global average pooling to obtain the weight of the feature map output by each convolutional layer Then the weights of the feature maps output by each convolutional layer are Perform weighted summation and pass through the activation function ReLU to obtain the class activation map corresponding to the target task category

[0060] It should be understood that the activation function ReLU is used to retain features that have a positive impact on the target task category c and remove features that have a negative impact.

[0061] Among them, the weight The formula for determining is:

[0062]

[0063] Class Activation Map The formula for determining is:

[0064]

[0065] The ReLU function is used to retain features that have a positive impact on the category and remove features that have a negative impact.

[0066] S14. Normalize the class activation map corresponding to the target task category to obtain the semantic importance score corresponding to each semantic feature vector.

[0067] Specifically, the class activation map That is, a two-dimensional matrix of the importance of each semantic feature vector to the target task category, and a class activation map After normalization, the semantic importance score of each semantic feature vector can be obtained.

[0068] From the above, it can be seen that the method provided by the embodiment of the present invention uses the learned perceptual image patch similarity (Learned Perceptual Image Patch Similarity, LPIPS) as an evaluation index. Since LPIPS is learned through a deep learning algorithm, it can better simulate human visual perception and better meet the requirements of semantic transmission. LPIPS represents the perceptual difference between images. The smaller the value, the smaller the perceptual difference between images and the better the image quality. Therefore, the method provided by the embodiment of the present invention can quickly and accurately determine the semantic importance scores corresponding to the multiple semantic feature vectors included in the image to be transmitted under the target task category, and then determine the corresponding channel for the information corresponding to each semantic feature vector based on the semantic importance score corresponding to each semantic feature vector. Thereby reducing the semantic distortion of information, ensuring that more important information is transmitted more stably at the perceptual level, thereby reducing the learned perceptual image block similarity.

[0069] It should be noted that the method provided in the embodiment of the present invention can also process data to be transmitted in other formats to obtain multiple semantic feature vectors corresponding to the data to be transmitted, as well as the semantic importance score corresponding to each semantic feature vector. The data to be transmitted can be text data, voice data, video data or image data. The embodiment of the present invention does not specifically limit the specific application scenario of the communication system resource allocation method based on semantic information and the specific data form of the data to be transmitted.

[0070] S2. Convert each semantic feature vector into a binary bit vector to obtain the amount of transmission information corresponding to each semantic feature vector.

[0071] Based on the above example, semantic feature vector A, semantic feature vector B and semantic feature vector C are converted into binary bit vectors. The amount of information transmitted by semantic feature vector A is 100 bits, the amount of information transmitted by semantic feature vector B is 50 bits, and the amount of information transmitted by semantic feature vector C is 200 bits.

[0072] S3. Determine the product of the semantic importance score corresponding to each semantic feature vector and the amount of transmitted information as the semantic importance weighted information amount of each semantic feature vector.

[0073] In combination with the above example, the semantic importance weighted information amount of semantic feature vector A is 3500, the semantic importance weighted information amount of semantic feature vector B is 3000, and the semantic importance weighted information amount of semantic feature vector C is 5000.

[0074] S4. Sort the multiple semantic feature vectors from large to small according to the semantic importance weighted information amount of each semantic feature vector to obtain a first sequence.

[0075] For example, when the semantic importance weighted information amount of semantic feature vector A is 3500, the semantic importance weighted information amount of semantic feature vector B is 3000, and the semantic importance weighted information amount of semantic feature vector C is 5000, since 5000>3500>3000, the first sequence is semantic feature vector C, semantic feature vector A, and semantic feature vector B, respectively.

[0076] S5. According to the first sequence, determine the unique channel corresponding to each semantic feature vector in turn, wherein the unique channel is a channel with the smallest probability of communication interruption among the remaining multiple channels and a channel whose usable transmission information amount is greater than the transmission information amount.

[0077] The amount of available transmission information for each channel is the difference between the channel capacity of each channel and the amount of used transmission information.

[0078] Exemplarily, the channel capacity of channel 1 is 300 bit / ms. When channel 1 is not used to transmit any semantic feature vector, the used transmission information amount of channel 1 is 0, and the available transmission information amount is 300 bit / ms. When channel 1 is used to transmit a semantic feature vector D with a transmission information amount of 100 bits within a unit time of 1 ms, the used transmission information amount of channel 1 is 100 bit / ms, and the available transmission information amount is 300 bit / ms-100 bit / ms=200 bit / ms.

[0079] Specifically, the resource allocation module 111 sorts the multiple channels from small to large according to the communication interruption probability of the multiple channels to obtain a second sequence, and then, when the amount of available transmission information of the channel is greater than or equal to the amount of transmission data of the semantic feature vector, the channels sorted in front of the second sequence are sequentially allocated to the semantic feature vectors sorted in front of the first sequence, and each semantic feature vector corresponds to a channel. When the amount of available transmission information of the target channel is less than the amount of transmission data of the target semantic feature vector, the channel sorted next to the target channel and whose amount of available transmission information is greater than or equal to the amount of transmission data of the target semantic feature vector is determined as the only channel of the target semantic feature vector, where the target semantic feature vector is any one of the multiple semantic feature vectors.

[0080] In a possible implementation, when the resource allocation module 111 executes the above S5, see Figure 5 , specifically including the following steps:

[0081] S51, for any semantic feature vector, determining whether there is a channel among multiple channels whose available transmission information amount is greater than or equal to the transmission information amount of any semantic feature vector;

[0082] S52, when there is a channel whose available transmission information amount is greater than the transmission information amount of any semantic feature vector among multiple channels, determine the target channel with the lowest communication interruption probability among the channels whose available transmission information amount is greater than the transmission information amount of any semantic feature vector as the only channel corresponding to any semantic feature vector;

[0083] S53: Update the available transmission information amount of the target channel according to the difference between the available transmission information amount of the target channel and the transmission information amount of any semantic feature vector.

[0084] In one example, the communication system includes three channels, namely channel 1, channel 2 and channel 3. The channel capacity of channel 1 is 350 bit / ms, and the probability of communication interruption is 5%. The channel capacity of channel 2 is 300 bit / ms, and the probability of communication interruption is 10%. The channel capacity of channel 3 is 300 bit / ms, and the probability of communication interruption is 20%. In combination with the above example, when any of the above semantic feature vectors is semantic feature vector C, the resource allocation module 111 first determines whether there are channels among multiple channels whose available transmission information amount is greater than or equal to the transmission information amount of semantic feature vector C, namely, channel 1, channel 2 and channel 3. The resource allocation module 111 determines the target channel with the smallest probability of communication interruption among the channels whose available transmission information amount is greater than the transmission information amount of semantic feature vector C as the only channel corresponding to semantic feature vector C, that is, determines channel 1 as the only channel corresponding to semantic feature vector C.

[0085] Since channel 1 is not used to transmit other semantic feature vectors at this time, the available transmission information amount of channel 1 is equal to the channel capacity. The resource allocation module 111 determines that the difference between the available transmission information amount of channel 1 corresponding to the semantic feature vector C and the transmission information amount of the semantic feature vector C is 150. That is, after determining that the only channel of the semantic feature vector C is channel 1, the resource allocation module 111 updates the available transmission information amount of channel 1 to 150 bit / ms.

[0086] Then, according to the first sequence, the resource allocation module 111 determines whether there are channels among the multiple channels whose available transmission information amount is greater than or equal to the transmission information amount of the semantic feature vector A, namely, channel 1, channel 2 and channel 3. The resource allocation module 111 determines the target channel with the lowest probability of communication interruption among the channels whose available transmission information amount is greater than the transmission information amount of the semantic feature vector A as the only channel corresponding to the semantic feature vector A, that is, determines channel 1 as the only channel corresponding to the semantic feature vector A. Since the available transmission information amount of channel 1 is equal to 150 bit / ms at this time. The resource allocation module 111 determines that the difference between the available transmission information amount of channel 1 corresponding to the semantic feature vector A and the transmission information amount of the semantic feature vector A is 50 bit / ms. That is, after determining the only channel of the semantic feature vector A as channel 1, the resource allocation module 111 updates the available transmission information amount of channel 1 to 50 bit / ms.

[0087] Finally, the resource allocation module 111 determines, according to the first sequence, whether there are channels among the multiple channels whose available transmission information amount is greater than or equal to the transmission information amount of the semantic feature vector B, namely, channel 2 and channel 3. Since the probability of communication interruption of channel 2 is less than that of channel 3, the resource allocation module 111 determines channel 2 as the only channel corresponding to the semantic feature vector B. Since the available transmission information amount of channel 2 is equal to 300 at this time. The resource allocation module 111 determines that the difference between the available transmission information amount of channel 2 and the transmission information amount of the semantic feature vector B is 250. That is, after determining that the only channel of the semantic feature vector B is channel 2, the resource allocation module 111 updates the available transmission information amount of channel 2 to 50bit / ms.

[0088] In a possible implementation, the embodiment of the present invention further includes the following steps:

[0089] If there is no channel among the multiple channels whose available transmission information amount is greater than the transmission information amount of the target semantic feature vector, the target semantic feature vector is deleted.

[0090] In an example, when the amount of available transmission information of channel 1 is 50 bit / ms, the amount of available transmission information of channel 2 is 50 bit / ms, the amount of available transmission information of channel 3 is 50 bit / ms, and the amount of transmission information of semantic feature vector D is 100 bits, the semantic feature vector D is deleted.

[0091] In another possible implementation, the embodiment of the present invention further includes the following steps:

[0092] If there is no channel among the multiple channels that can use a transmission information amount greater than the transmission information amount of the target semantic feature vector, the target semantic feature vector is stored in the cache queue.

[0093] In another example, when the available transmission information amount of channel 1 is 50 bit / ms, the available transmission information amount of channel 2 is 50 bit / ms, the available transmission information amount of channel 3 is 50 bit / ms, and the transmission information amount of semantic feature vector D is 100 bits, the semantic feature vector D is stored in the cache queue.

[0094] It should be noted that the above examples are only for illustrative purposes. In the embodiment of the present invention, the number of semantic feature vectors and the number of channels may be more or less, and the embodiment of the present invention does not impose any special limitation on this.

[0095] It can be seen from the above S1-S5 that the method provided by the embodiment of the present invention can ensure the transmission efficiency of important information by determining the importance scores of multiple semantic vectors included in the image to be transmitted, and then allocating channels with low interruption probability to semantic vectors with high importance based on the importance scores of multiple semantic vectors. It can also be understood that the method provided by the embodiment of the present invention can quickly and accurately determine the importance of information corresponding to each semantic vector for the target task category, and dynamically select different channels for different semantic vectors according to the importance of the information, which can effectively reduce semantic distortion, thereby improving communication efficiency, optimizing resource utilization, and enhancing the adaptability and response speed of the system. In addition, the method provided by the embodiment of the present invention can solve the problem of low communication efficiency in application scenarios with high critical communication requirements, and has good flexibility in application scenarios with intensive and rapidly changing transmission data, and can meet the communication needs in different usage scenarios.

[0096] In a possible implementation manner of the first aspect, when the semantic feature vector is stored in the cache queue, the above S4 specifically includes the following steps:

[0097] Determine the importance weighted information amount corresponding to the semantic feature vectors stored in the cache queue; sort the multiple semantic feature vectors from large to small according to the semantic importance weighted information amount of each semantic feature vector and the importance weighted information amount corresponding to the semantic feature vectors stored in the cache queue to obtain a first sequence.

[0098] It should be understood that the manner of determining the importance weighted information amount corresponding to the semantic feature vector stored in the cache queue is the same as that in the above embodiment, and will not be described in detail here.

[0099] The method provided by the embodiment of the present invention can avoid information loss by storing semantic feature vectors in a cache queue and determining and sending the corresponding channel for each semantic feature vector when the channel is idle, thereby improving information transmission efficiency and meeting user usage needs in different usage scenarios.

[0100] To facilitate understanding of the present solution, the following is an explanation of determining a unique channel for each semantic feature vector in an embodiment of the present invention based on an example. After the multiple semantic feature vectors corresponding to the image to be transmitted are converted into binary bit vectors, the transmission information amount I of each semantic feature vector is obtained. m , in the case where one LED corresponds to one channel, a m,n ∈{0,1}, when the mth semantic feature vector is assigned to the channel corresponding to the nth LED, a m,n =1; otherwise a m,n = 0. Assuming that each semantic feature vector can and can only be assigned to one channel, then:

[0101]

[0102] Assume that the theoretical channel capacity of the channel corresponding to the nth light-emitting diode is C n , then in unit time:

[0103]

[0104] Due to C n As time changes, the upper bound of the constraint also changes. Assuming that the channel is stationary in unit time, the modified constraint is

[0105]

[0106] E(C n ) represents the unit time C n The mean of , β(0<β<1) is the conservative factor.

[0107] It should be noted that the conservative factor can be set according to actual scenarios, and the embodiment of the present invention does not impose any particular limitation on this.

[0108] Since the channel is stable within a unit time, the probability of information interruption in the nth channel is for:

[0109]

[0110] SNR n is the current signal-to-noise ratio of the nth channel, γ th is the SNR threshold.

[0111] For the mth semantic feature vector assigned to the nth channel, define its semantic importance loss l m,n :

[0112]

[0113] The total semantic importance loss L total :

[0114]

[0115] Therefore, the channel determination of the semantic feature vector can be expressed as minimizing L total The optimization problem is as follows:

[0116]

[0117] And the constraint condition is (subject to);

[0118]

[0119] This optimization problem is a 0-1 integer programming problem, which is a generalization of the knapsack problem. It is NP-hard and it is difficult to find the global optimal solution in a limited time. Therefore, a heuristic algorithm is used to find the local optimal solution in a reasonable time.

[0120] Taking the greedy algorithm as an example to solve the optimization problem, the specific steps are as follows:

[0121] Determine the available transmission information amount R of each LED corresponding channel according to the current state n =βE(C n ), information interruption probability Determine the semantic importance weighted information of each semantic feature vector, where for each semantic feature vector, calculate the semantic importance weighted information w m I m ; Weight the semantic importance information w m I m Sort from large to small to obtain a first sequence including multiple semantic feature vectors, according to the information interruption probability of multiple channels Sort from small to large to get the second sequence. Based on the first sequence, according to the amount of transmitted information I corresponding to the sorted semantic feature vector m , in turn (w m I m Try to assign it to the channel n included in the second sequence (from large to small) From small to large), if R n ≥I m , then the mth semantic feature vector is assigned to the nth channel. If R n <I m , then it is assigned to the next channel in sequence. Repeat the above steps in sequence until all semantic feature vectors are traversed, check whether all semantic feature vectors have been assigned, and if there are unassigned semantic feature vectors, delete them or store them in the cache queue until the next transmission.

[0122] Through this method, information can be allocated more reasonably, and the resource allocation of channels corresponding to different color LEDs can be completed according to semantic importance, ensuring that information with high semantic importance is transmitted on better channels first, minimizing the overall semantic importance loss. Although the greedy algorithm cannot guarantee the global optimal solution, it can converge to a high-quality suboptimal solution in a reasonable time in practical applications, providing a near-optimal solution.

[0123] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A communication system resource allocation method based on semantic information, characterized in that: Applied to a communication system, the communication system includes a plurality of channels, each channel has a corresponding channel capacity and a communication interruption probability, the method includes: Processing the image to be transmitted to obtain a plurality of semantic feature vectors corresponding to the image to be transmitted and a semantic importance score corresponding to each semantic feature vector; Convert each semantic feature vector into a binary bit vector to obtain the amount of transmitted information corresponding to each semantic feature vector; The product of the semantic importance score corresponding to each semantic feature vector and the amount of transmitted information is determined as the semantic importance weighted information amount of each semantic feature vector; The plurality of semantic feature vectors are sorted from large to small according to the semantic importance weighted information amount of each semantic feature vector to obtain a first sequence; According to the first sequence, the unique channel corresponding to each semantic feature vector is determined in turn, wherein the unique channel is the channel with the smallest probability of communication interruption among the remaining multiple channels and the amount of usable transmission information is greater than the amount of transmission information, wherein the amount of usable transmission information of each channel is the difference between the channel capacity of each channel and the amount of used transmission information.

2. The method according to claim 1, characterized in that The processing of the image to be transmitted to obtain a plurality of semantic feature vectors corresponding to the image to be transmitted and a semantic importance score corresponding to each semantic feature vector includes: Based on the convolutional neural network, forward propagation is performed on the image to be transmitted to obtain multiple semantic feature vectors and prediction confidence scores corresponding to the target task category of each semantic feature vector, wherein the convolutional neural network includes x convolutional layers, and x is a positive integer; The prediction confidence score corresponding to the target task category of each semantic feature vector is used as the loss function, and back-propagation is performed to obtain the gradient corresponding to the xth convolutional layer among the x convolutional layers; Determine the class activation map corresponding to the target task category based on the gradient corresponding to the xth convolutional layer and the feature map output by each convolutional layer; The class activation map corresponding to the target task category is normalized to obtain the semantic importance score corresponding to each semantic feature vector.

3. The method according to claim 2, characterized in that When determining the unique channel corresponding to each semantic feature vector in sequence according to the first sequence, the method includes: For any semantic feature vector, determining whether there is a channel among the multiple channels whose available transmission information amount is greater than or equal to the transmission information amount of any semantic feature vector; In the case that there is a channel whose available transmission information amount is greater than the transmission information amount of any of the semantic feature vectors among the multiple channels, a target channel with the lowest probability of communication interruption among the channels whose available transmission information amount is greater than the transmission information amount of any of the semantic feature vectors is determined as the only channel corresponding to any of the semantic feature vectors; The available transmission information amount of the target channel is updated according to the difference between the available transmission information amount of the target channel and the transmission information amount of any semantic feature vector.

4. The method according to claim 3, characterized in that The method further comprises: If there is no channel among the multiple channels whose available transmission information amount is greater than or equal to the transmission information amount of any semantic feature vector, the any semantic feature vector is deleted.

5. The method according to claim 3, characterized in that: The method further comprises: If there is no channel among the multiple channels that can use a transmission amount of information greater than the transmission amount of any semantic feature vector, the any semantic feature vector is stored in a cache queue.

6. The method according to claim 5, characterized in that In the case where the semantic feature vectors are stored in the cache queue, the plurality of semantic feature vectors are sorted from large to small according to the semantic importance weighted information amount of each semantic feature vector to obtain a first sequence, including: Determining the importance weighted information amount corresponding to the semantic feature vector stored in the cache queue; According to the semantic importance weighted information volume of each semantic feature vector and the importance weighted information volume corresponding to the semantic feature vectors stored in the cache queue, multiple semantic feature vectors are sorted from large to small to obtain a first sequence.

7. A communication system, characterized in that: The communication system comprises: A plurality of channels, each channel having a corresponding channel capacity and a communication interruption probability; A semantic processing unit, used for processing the image to be transmitted to obtain a plurality of semantic feature vectors corresponding to the image to be transmitted and a semantic importance score corresponding to each semantic feature vector; A vector conversion unit, used to convert each semantic feature vector into a binary bit vector to obtain a transmission information amount corresponding to each semantic feature vector; An information determination unit, used to determine the product of the semantic importance score corresponding to each semantic feature vector and the amount of transmitted information as the semantic importance weighted information amount of each semantic feature vector; A vector sorting unit, used for sorting the multiple semantic feature vectors from large to small according to the semantic importance weighted information amount of each semantic feature vector to obtain a first sequence; A channel determination unit is used to determine, in accordance with the first sequence, a unique channel corresponding to each semantic feature vector in turn, wherein the unique channel is a channel with the smallest probability of communication interruption among the remaining multiple channels and whose usable transmission information amount is greater than the transmission information amount, wherein the usable transmission information amount of each channel is the difference between the channel capacity of each channel and the used transmission information amount.

8. The system according to claim 7, characterized in that The semantic processing unit is specifically used for: Based on the convolutional neural network, forward propagation is performed on the image to be transmitted to obtain multiple semantic feature vectors and prediction confidence scores corresponding to the target task category of each semantic feature vector, wherein the convolutional neural network includes x convolutional layers, and x is a positive integer; The prediction confidence score corresponding to the target task category of each semantic feature vector is used as the loss function, and back propagation is performed to obtain the gradient corresponding to the xth convolutional layer among the x convolutional layers; Determine the class activation map corresponding to the target task category based on the gradient corresponding to the xth convolutional layer and the feature map output by each convolutional layer; The class activation map corresponding to the target task category is normalized to obtain the semantic importance score corresponding to each semantic feature vector.

9. The system according to claim 8, characterized in that When the channel determination unit is used to determine the unique channel corresponding to each semantic feature vector in sequence according to the first sequence, the channel determination unit is specifically used to: For any semantic feature vector, determining whether there is a channel among the multiple channels whose available transmission information amount is greater than or equal to the transmission information amount of any semantic feature vector; In the case that there is a channel whose available transmission information amount is greater than the transmission information amount of any of the semantic feature vectors among the multiple channels, a target channel with the lowest probability of communication interruption among the channels whose available transmission information amount is greater than the transmission information amount of any of the semantic feature vectors is determined as the only channel corresponding to any of the semantic feature vectors; The available transmission information amount of the target channel is updated according to the difference between the available transmission information amount of the target channel and the transmission information amount of any semantic feature vector.

10. The system according to claim 9, characterized in that The channel determination unit is further configured to: If there is no channel among the multiple channels whose available transmission information amount is greater than or equal to the transmission information amount of any semantic feature vector, the any semantic feature vector is deleted.

Images