Edge multi-task semantic communication method and device, computer equipment and storage medium

Abstract

The invention discloses an edge multi-task semantic communication method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining the neuromorphic data of each path of user signal, carrying out the semantic feature extraction of the neuromorphic data based on a pulse neural network, and carrying out the coding, and obtaining a corresponding low-dimensional feature vector; performing power distribution and superposition on each low-dimensional feature vector to obtain a superposed signal; and detecting the receiving signal power of the user signals in the superposed signals, and sequentially separating and decoding each path of user signals from the superposed signals according to the size of the receiving signal power of the user signals so as to complete a corresponding reasoning task. The pulse neural network is adopted to replace a traditional artificial neural network, the quantization step is avoided, the accuracy of reasoning task classification is improved, the superposed user signals are separated and decoded according to the received signal power of the user signals, and the decoding efficiency is improved. The problem that resources allocated to users with poor signal information in a traditional multiple access technology are few is solved.

Description

Technical Field

[0001] The present application relates to the field of information and communication technology, and in particular to an edge multi-task semantic communication method, apparatus, computer equipment and storage medium. Background Art

[0002] Neural network-based intelligent IoT systems are increasingly being used to perform edge reasoning tasks. In IoT sensing and computing applications, wireless edge sensing devices offload the collected sensor data to nearby edge servers with strong computing power to perform subsequent reasoning tasks. Due to the increasing number of mobile devices and more complex multimedia IoT applications, limited communication bandwidth has become the main bottleneck for sensor data offloading. Task-oriented semantic communication has the characteristics of extracting task-related semantic information and discarding redundant information. Its advantage is that it significantly reduces the size of communication data while maintaining high reasoning accuracy, which enables multiple edge devices to complete data transmission more efficiently under the same bandwidth constraints.

[0003] In the process of transmitting semantic features in semantic communication technology, the semantic features will be affected by the noise in the wireless environment, which means that when the wireless environment changes, the distribution of sensor data collected by the edge server and the signal-to-noise ratio of the wireless channel will change. At the same time, when multiple sensing devices send semantic features to the same edge server at the same time, the problem of co-channel interference becomes particularly prominent. To solve this problem, multiple access technology is widely used in wireless communication systems to improve system capacity and transmission efficiency. Multiple access technology divides resources into non-interfering parts to avoid mutual interference between signals. Although multiple access technology performs well in traditional communications, in actual multi-user communication scenarios, users with poor channel conditions may obtain fewer resources or be ignored by the system priority, resulting in their performance being affected. In addition, in order to be compatible with modern digital communication standards, traditional neural networks will add additional quantization modules to the system, which will cause problems of non-differentiability and convergence in the training stage and back-propagation stage, resulting in reduced classification accuracy of the edge server.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the purpose of the present application is to provide an edge multi-task semantic communication method, apparatus, computer equipment and storage medium to solve the problems existing in the existing multi-semantic communication technology that users with poor channel conditions may obtain fewer resources or be ignored by the system priority, and the traditional neural network uses a quantization module, resulting in a decrease in the classification accuracy of the edge server.

[0006] The technical solution of this application is as follows: In a first aspect, the present application provides an edge multi-task semantic communication method, which includes: Obtain the neuromorphic data of each user signal, extract semantic features from the neuromorphic data based on the spiking neural network, and encode the data to obtain the corresponding low-dimensional feature vector; Power is allocated to each low-dimensional feature vector and then superimposed to obtain a superimposed signal; The received signal power of each user signal in the superimposed signal is detected, and each user signal is separated from the superimposed signal in turn according to the size of the received signal power of each user signal and decoded to complete the corresponding reasoning task.

[0007] In a further configuration of the present application, the step of detecting the received signal power of each user signal in the superimposed signal, and sequentially separating and decoding each user signal from the superimposed signal according to the size of the received signal power of each user signal to complete the corresponding reasoning task includes: Detecting the received signal power of each user signal in the superimposed signal, and decoding the user signal with the highest received signal power among the user signals; Reconstructing the user signal with the highest received signal power in the superimposed signal and subtracting it from the superimposed signal; The user signal with the largest received signal power in the current superimposed signal is re-detected, decoded and task inference is performed, and then further reconstructed and the reconstructed features are subtracted from the superimposed signal until all user signals are decoded.

[0008] In a further configuration of the present application, the steps of obtaining the neural morphological data of each user signal, extracting semantic features from the neural morphological data using a spiking neural network, and encoding the semantic features to obtain a corresponding low-dimensional feature vector include: Representing the acquired neuromorphic data as an integer vector; wherein the neuromorphic data is collected by multiple edge sensing devices; Extracting semantic features through a feature extractor of a spiking neural network and expanding the extracted semantic features; The expanded semantic features are input into the feature encoder of the pulse neural network for encoding and then output as a low-dimensional feature vector.

[0009] In a further configuration of the present application, the step of allocating power to each low-dimensional feature vector and superimposing them to obtain a superimposed signal comprises: When superimposing the user signals of various channels, the sizes of the low-dimensional feature vectors corresponding to the user signals of various channels are adjusted so that the user signals of various channels can be transmitted and superimposed on the same channel at the same time.

[0010] In a further configuration of the present application, the step of reconstructing the user signal with the highest received signal power in the superimposed signal and subtracting it from the superimposed signal comprises: Reconstructing the semantic features using a feature decoder to obtain a decoded signal; the decoded signal includes semantic information corresponding to the encoded user signal; Encoding the decoded signal to simulate binary reconstruction features; The reconstructed feature is subtracted from the superimposed signal.

[0011] In a further configuration of the present application, mean square error is used as a loss function to calculate the difference between the output and the unique hot encoding of the corresponding label during training, and the edge multi-task semantic communication method also includes: The feature extractor and feature encoder of each user signal are trained based on the mean square error loss function. The mean square error between the predicted label and the true label is calculated through the output of each user signal and the one-hot vector to update the network parameters of the semantic feature extractor and the semantic feature encoder. The feature decoder is trained to obtain a loss function of the feature decoder; wherein the loss function of the feature decoder is composed of the decoded signal and the semantic features extracted from the neuromorphic data; The classifier of each user signal is trained, and the mean square error between the predicted label and the true label is calculated through the output of each user and the one-hot vector.

[0012] A further configuration of the present application is that, in the pulse neural network, the activation function used in forward propagation is a step function, and back propagation uses a continuous and approximately shaped function as an alternative gradient function.

[0013] In a second aspect, the present application also provides an edge multi-task semantic communication device, which includes: A feature extraction and encoding unit is used to obtain the neuromorphic data of each user signal, and use a pulse neural network to extract semantic features from the neuromorphic data and encode them to obtain a corresponding low-dimensional feature vector; A signal superposition unit, used for allocating power to each low-dimensional feature vector and superimposing them to obtain a superposition signal; The signal separation and decoding unit is used to detect the received signal power of each user signal in the superimposed signal, and to separate and decode each user signal from the superimposed signal in turn according to the size of the received signal power of each user signal to complete the corresponding inference task.

[0014] In a third aspect, the present application also provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it is used to implement the steps in the edge multi-task semantic communication method as described above.

[0015] In a fourth aspect, the present application also provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, it is used to implement the steps in the edge multi-task semantic communication method as described above.

[0016] The present application provides an edge multi-task semantic communication method, device, computer equipment and storage medium, the method comprising: obtaining the neuromorphic data of each user signal, extracting semantic features of the neuromorphic data based on a pulse neural network and encoding the data to obtain a corresponding low-dimensional feature vector; allocating power to each low-dimensional feature vector and superimposing them to obtain a superimposed signal; detecting the received signal power of each user signal in the superimposed signal, and separating and decoding each user signal from the superimposed signal in turn according to the size of the received signal power of each user signal to complete the corresponding reasoning task. The present application uses a pulse neural network to replace the traditional artificial neural network, avoiding the quantization step, improving the accuracy of the reasoning task classification, and separating and decoding the user signals directly superimposed together according to the power size of the user signal, thus solving the problem of fewer resources allocated to users with poor signal information in traditional multiple access technology, and taking into account both efficiency and fairness. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary personnel in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0018] Figure 1 It is a flowchart of the edge multi-task semantic communication method in this application.

[0019] Figure 2 This is a comparison chart of the data set processing results of the spiking neural network and the artificial neural network.

[0020] Figure 3 It is a schematic diagram of an edge multi-task semantic communication method in one embodiment of the present application.

[0021] Figure 4 This is a performance comparison chart of a static image dataset and a dynamic gesture recognition dataset under different feature dimensions in an embodiment of the present application.

[0022] Figure 5 This is a performance comparison chart between a pulse neural network and an artificial neural network in one embodiment of the present application.

[0023] Figure 6 This is a performance comparison chart of whether or not serial interference cancellation technology is used in an embodiment of the present application.

[0024] Figure 7 It is a principle block diagram of the edge multi-task semantic communication device in this application. DETAILED DESCRIPTION

[0025] The present application provides an edge multi-task semantic communication method, device, computer equipment and storage medium. In order to make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0026] In the embodiments and the scope of the patent application, unless the text specifically defines the article, "a", "an", "the" and "the" may also include plural forms. If there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0027] It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when an element is said to be "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can also be intermediate elements. In addition, the "connection" or "coupling" used here can include wireless connection or wireless coupling. The term "and / or" used here includes all or any unit and all combinations of one or more associated listed items.

[0028] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0029] In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0030] The inventors have found that 5G / 6G communications and artificial intelligence have become the representatives of a new round of information technology revolution and industrial transformation. In the sensing and computing applications of the Internet of Things, wireless edge sensing devices offload the collected sensing data to nearby edge servers with strong computing power to perform subsequent reasoning tasks. The current mainstream semantic communication technology is: first, the edge sensing device extracts semantic features from the collected raw sensing data through a semantic encoder based on an artificial neural network, and then transmits the extracted semantic features to the edge server through a wireless channel. Finally, the edge server completes the corresponding reasoning task through the received semantic features.

[0031] In the process of transmitting semantic features in semantic communication technology, the semantic features will be affected by the noise in the wireless environment, which means that when the wireless environment changes, the distribution of sensor data collected by the edge server and the signal-to-noise ratio of the wireless channel will change. At the same time, when multiple sensing devices send semantic features to the same edge server at the same time, the problem of co-channel interference becomes particularly prominent. In order to solve this problem, multiple access technology is widely used in wireless communication systems to improve system capacity and transmission efficiency. Traditional multiple access technologies include frequency division multiple access, time division multiple access, and code division multiple access. Frequency division multiple access divides the channel into different frequency bands and allocates them to different devices to avoid co-channel interference; time division multiple access divides time into multiple time slots, and devices send data in their respective time slots to reduce interference in the same time period; code division multiple access is based on spread spectrum code technology, allowing multiple devices to send data on the same frequency at the same time, and distinguishing device signals through unique codes. Multiple access technology divides resources into non-interfering parts to avoid mutual interference between signals. Although multiple access technology performs well in traditional communications, in actual multi-user communication scenarios, users with poor channel conditions may receive fewer resources or be ignored by the system priority, which cannot balance efficiency and fairness, resulting in performance being affected. In addition, in order to be compatible with modern digital communication standards, traditional neural networks will add additional quantization modules to the system, which will cause non-differentiability and convergence problems in the training and back-propagation stages, resulting in reduced classification accuracy of edge servers.

[0032] In addition, most existing studies directly consider the transmission of continuous signals from the output of traditional neural networks. Traditional neural networks usually use weighted summation and nonlinear activation functions to simulate neuronal behavior. Neurons generally transmit signals through floating-point numbers. This computing mechanism is significantly different from the discrete pulse signal transmission of biological neural systems. Although there are currently technologies such as replacing neural functions and semantic perception codebooks to solve related problems, there is still a certain degree of performance degradation compared to analog communication. In addition, traditional neural networks have high computational costs and require a lot of computing resources to perform high-precision floating-point calculations. The computing power of edge sensing devices is usually limited. Once the network layers are deep or the data scale is large, it will be difficult for edge sensing devices to extract relevant features with low latency.

[0033] In summary, the existing semantic edge reasoning methods have the following technical defects: 1. The multiple access technology used in traditional multi-user semantic communication scenarios allocates fewer resources to users with poor channel information, which seriously affects their performance; 2. During the communication process of traditional artificial neural networks, the edge perception devices consume a lot of energy and need to quantize the data before unloading it to the edge server, resulting in poor final classification accuracy of the edge server.

[0034] In response to the above technical problems, the present application provides an edge multi-task semantic communication method, device, computer equipment and storage medium, which adopts a neuromorphic neural network to replace the traditional artificial neural network, avoids the quantization step, improves the accuracy of reasoning task classification, and reduces the computational complexity, thereby reducing energy consumption, and separates and decodes user signals that are directly superimposed on each other according to the power size of the user signals. This solves the problem of traditional multiple access technology in which fewer resources are allocated to users with poor signal information, and takes into account both efficiency and fairness.

[0035] Please also see Figures 1 to 6 , the present application provides a preferred embodiment of an edge multi-task semantic communication method.

[0036] In some embodiments, Figure 1 As shown, the present application provides an edge multi-task semantic communication method, which includes the steps of: S100, obtaining neuromorphic data of each user signal, extracting semantic features from the neuromorphic data based on a spiking neural network, and encoding the data to obtain a corresponding low-dimensional feature vector; Specifically, neuromorphic data is data with time information, which is collected by edge sensing devices. Unlike artificial neural networks (ANNs), which use continuous-valued signals as input and output, spiking neural networks (SNNs) transmit information through discrete-time pulse signals, which is closer to the actual behavior of biological neurons. The basic computing unit of a spiking neural network is a spiking neuron, that is, a neuron that can only output 0 or 1. Each neuron receives a signal from the upper layer of neurons and accumulates it to its own membrane potential. When the membrane potential exceeds a set threshold, the neuron generates a pulse and passes it to the next layer of neurons. At the same time, its membrane potential is immediately reset to the reset potential, simulating the discharge behavior of biological neurons. The neuron model commonly used in spiking neural networks is the Leaky Integrate-and-Fire (LIF) model, which can be described by the following three states of charging, discharging, and resetting: ; Among them, the above formula represents the state update of the neuron, represents the membrane potential of the neuron after charging and before discharging at time t, V represents the membrane potential, represents the membrane time constant, represents the membrane potential before discharge, Represents the input of the neuron.

[0037] ; In the above formula, Indicates the firing state of synaptic neurons. When the membrane potential Exceeding a certain threshold at time t When , the synaptic neuron will discharge, which is expressed as , otherwise the neuron remains silent, i.e. .

[0038] ; The above formula indicates that after the neuron fires a spike, the membrane potential will return to ,in, Represents the membrane potential after a neuron fires.

[0039] In a spiking neural network, since the output of a neuron is only 0 or 1, a step function is presented during the forward propagation process, and the corresponding derivative is a unit impulse function. If the system directly uses the unit impulse function as the gradient function, the entire training process will be very unstable. To solve this problem, the general training method of a spiking neural network is to use a continuous and similarly shaped function as a substitute gradient function in the back propagation, such as the Sigmoid function and the arctan function. In this embodiment, the arctan function is used. and its derivatives The corresponding ones are as follows: ; .

[0040] Since a spiking neural network is used, the activation function of the neurons in the network uses a step function, and the output of each stage is a binary signal, which allows it to avoid the quantization step. At the same time, the morphological architecture and event-driven characteristics of the spiking neural network (only outputs 0 and 1) significantly reduce the computational complexity, and can process data sets containing time information without increasing the complexity of the network.

[0041] Commonly used datasets for pulse neural networks include gesture recognition (DVSGesture), digital classification (N-MNIST, MNIST-DVS), and image recognition (CIFAR-DVS, N-Caltech101). In this implementation, the datasets use two neuromorphic datasets: N-MNIST (static image dataset) and DVSGesture128 (dynamic gesture recognition dataset). N-MNIST is a pulse version of the MNIST dataset, and the data is collected by an asynchronous time image sensor. The DVS128Gesture dataset is 11 types of gesture data collected under three different lighting conditions, including clapping, waving arms, rotating arms, etc., recorded by a DVS128 camera. Each data sample contains a set of events, and each event is represented by four parameters: , where t is the timestamp, x and y are the horizontal and vertical coordinates of the event, respectively, and p is the polarity of the event. The event representation is as follows: ; in, represents an event, n is a positive integer, express Time events.

[0042] In order to convert the original event stream into frame data, a time window is set and the events are integrated within the window to generate a frame of data. In this embodiment, the frame sizes of the N-MNIST and DVS128Gesture datasets are set to 5 and 20 respectively. The integrated frame data can be expressed by the following formula: ; in, Indicates frame data, express Time events.

[0043] In order to compare the performance with the ANN of the same network structure and this embodiment, the data set is processed into a reasonable input format. Therefore, the corresponding data set will set the frame size to 1, that is, the original event stream is integrated. Figure 2 As shown, Figure 2 The first row in corresponds to the ANN data set result. Figure 2 The second and third rows in the figure are the results of the SNN dataset. It can be observed that the static image dataset in the first column can still be identified as the number "0" by its outline after processing, while the second and third columns are the results of the dynamic gesture recognition dataset after processing. The details of the arm movement cannot be distinguished from the image alone. Figure 2 The SNN in the proposed method can clearly distinguish the direction of the arm swinging because of the accompanying time information.

[0044] A low-dimensional feature vector refers to a set of vectors extracted from the original high-dimensional data through a specific algorithm, which can effectively represent the main features of the data. These vectors not only reduce the dimension of the data, but also retain the key information of the data, which is convenient for subsequent data processing and analysis tasks. One edge sensing device collects one user signal. In this embodiment, two edge sensing devices are used for illustration. In this embodiment, event-driven cameras are used to observe neuromorphic data with time information, and feature extraction of neuromorphic data based on pulse neural network is performed to obtain semantic features and encode them to obtain low-dimensional feature vectors, such as Figure 3 shown.

[0045] S200, performing power allocation on each low-dimensional feature vector and superimposing the vectors to obtain a superimposed signal; Specifically, please combine Figure 3, because the low-dimensional feature vector obtained after semantic extraction and encoding of neuromorphic data based on the pulse neural network is a binary signal, no additional normalization or quantization steps are required, and power allocation can be performed directly. For example, when there are two users, the signal of user 1 is recorded as the first user signal, and the signal of user 2 is recorded as the second user signal. The received signal power of the first user signal is greater than the received signal power of the second user signal. Afterwards, each user signal sends its own feature vector in a shared broadband.

[0046] S300, detecting the received signal power of each user signal in the superimposed signal, and separating and decoding each user signal from the superimposed signal in turn according to the received signal power of each user signal to complete the corresponding inference task.

[0047] Specifically, the superimposed signal refers to a signal in which various user signals are superimposed together, for example, a signal in which the first user signal and the second user signal are superimposed together. The user signals are collected by edge sensing devices, and the number of edge sensing devices can be 2, 3, 4, etc. The received signal power refers to the product of the transmission power of the user signal and the channel coefficient of the channel. Usually, a higher power is matched to users with better channel conditions. After the edge server receives the superimposed signal, it classifies the user signals according to the size of the received signal power, and completes the decoding and separation of the user signals in order from large to small according to the size of the received signal power. For example, when the received signal power of the first user signal is greater than the received signal power of the second user signal, the first user signal is first decoded and the corresponding reasoning task is completed, and then the first user signal is subtracted from the superimposed signal before the second user signal is decoded and the corresponding reasoning task is completed.

[0048] In the above technical scheme, the present application provides a non-orthogonal multi-access digital semantic communication method for edge multi-task reasoning, which uses a pulse neural network to replace the traditional artificial neural network, avoids the quantization step, improves the accuracy of reasoning task classification, and reduces the computational complexity, thereby reducing energy consumption. And according to the power of the user signal, the user signals directly superimposed together are separated and decoded, which solves the problem of fewer resources allocated to users with poor signal information in traditional multiple access technology, and takes into account both efficiency and fairness.

[0049] In some embodiments, the step of obtaining the neural morphological data of each user signal, extracting semantic features from the neural morphological data using a spiking neural network, and encoding the neural morphological data to obtain a corresponding low-dimensional feature vector comprises: S110, representing the acquired neuromorphic data as an integer vector; wherein the neuromorphic data is collected by multiple edge sensing devices; S120, extracting semantic features through a feature extractor of a spiking neural network and expanding the extracted semantic features; S130, inputting the expanded semantic features into the feature encoder of the pulse neural network for encoding and then outputting a low-dimensional feature vector.

[0050] In this embodiment, the integer vector refers to a vector whose input vector is limited to integers, which can improve the calculation speed and save computing resources and time during operation. When extracting the collected neuromorphic data, the collected neuromorphic data needs to be represented as an integer vector. After extracting the semantic features, the feature extractor expands the semantic features and inputs them into the fully connected feature encoder of the pulse neural network, and finally outputs a low-dimensional feature vector.

[0051] Taking two user signals as an example, first, the edge sensing device Represents the kth user, and the neuromorphic data collected by the device is represented as an integer vector , where k=1,2. The data set of this embodiment uses two neuromorphic data sets, N-MNIST and DVSGesture, where the first user signal and the second user signal are classification tasks of the N-MNIST and DVSGesture data sets, respectively, that is, user 1 is responsible for the static digital classification task, and user 2 is responsible for the dynamic gesture recognition task. The corresponding network models are shown in Tables 1 and 2: Table 1:

[0052] Table 2:

[0053] Among them, the feature extractor It consists of convolutional layers, where k=1,2, represents the feature extractor, is the network parameter of the feature extractor, extracting semantic features After that, it is expanded and input into the feature encoder of the pulse neural network In which the feature encoder of the spiking neural network is is a fully connected semantic feature encoder, is the network parameter of the feature encoder, and the feature dimension of the final output They are low-dimensional feature vectors of 5×6 and 20×24 respectively.

[0054] In some embodiments, see Figure 1 Combined with Figure 3 The step of allocating power to each low-dimensional feature vector and superimposing them to obtain a superimposed signal comprises: S210: When superimposing the user signals of various channels, the sizes of the low-dimensional feature vectors corresponding to the user signals of various channels are adjusted so that the user signals of various channels can be transmitted and superimposed on the same channel at the same time.

[0055] Specifically, when the edge sensing device sends compressed semantic features to the edge server in the wireless channel, it is necessary to superimpose the signals of each user in the channel, so the size of each semantic feature needs to be adjusted so that each user signal can be transmitted in the same channel. Taking two user signals as an example, the first user signal and the second user signal need to be transmitted and superimposed in the same channel, and the feature size of the two user signals needs to be adjusted, so that user 1 can transmit 16 data while user 2 can transmit 1 data.

[0056] In this embodiment, the signal between the edge sensing device and the edge server is a Rayleigh channel, so the edge server receives the input signal of two user signals (i.e., superimposed signal) at the receiving end, which can be expressed as: ; Where Y represents the superimposed signal, represents the channel coefficient, represents the transmission power of the kth user, , represents the maximum transmission power, , N represents additive white Gaussian noise (AWGN), where Represents a Gaussian distribution.

[0057] In some embodiments, see Figure 1 Combined with Figure 3 The steps of detecting the received signal power of each user signal in the superimposed signal, and separating and decoding each user signal from the superimposed signal in turn according to the received signal power of each user signal to complete the corresponding reasoning task include: S310, detecting the received signal power of each user signal in the superimposed signal, and decoding the user signal with the highest received signal power among the user signals; S320, reconstructing the user signal with the highest received signal power in the superimposed signal and subtracting it from the superimposed signal; S330, re-detecting the user signal with the largest received signal power in the current superimposed signal, performing decoding and task inference, and then further performing reconstruction processing, subtracting the reconstruction feature from the superimposed signal, until all user signals are decoded.

[0058] Specifically, when the edge server receives the superimposed signal, it detects the inference task of the near-end user with higher received signal power, and can classify the user signals according to the received signal power of each user signal.

[0059] When classifying and decoding user signals and performing task reasoning, in order to recover the semantic features of the user from the mixed signal superimposed by different users and noise and perform reasoning tasks, this embodiment uses serial interference cancellation technology (Serial Interference Cancellation, SIC) to solve the problem of co-channel interference and optimize reasoning performance. The edge processor will first process the user signal with the highest received signal power, that is, decode the user signal with the highest received signal power, and the decoding and task reasoning are completed by the classifier corresponding to the user signal. At the same time, the user signal with the highest received signal power in the current superimposed signal is reconstructed so that the user signal with the highest received signal power can be subtracted from the input superimposed signal to obtain the signals of the remaining users, complete signal separation, and reduce the interference level of the remaining signals. After completing the decoding and task reasoning of a user signal, repeat the above process, that is, re-detect the user signal with the relatively largest received signal power in the current superimposed signal, and after decoding and task reasoning, further reconstruct the signal and subtract the reconstructed features from the superimposed signal. Through iteration, until all user signals complete decoding and task reasoning, such as Figure 3 As shown, the output of user 1 is the number 3, and the output of user 2 is a waving action.

[0060] For further information, see Figure 1 Combined with Figure 3 The step of reconstructing the user signal with the highest received signal power in the superimposed signal and subtracting it from the superimposed signal comprises: S321, reconstructing semantic features using a feature decoder to obtain a decoded signal; the decoded signal includes semantic information corresponding to the encoded user signal; S322, encoding the decoded signal to simulate and obtain binary reconstruction features; S323: Subtract the reconstructed feature from the superimposed signal.

[0061] Specifically, different user signals have corresponding classifiers to complete decoding and task reasoning. This embodiment uses two user signals to illustrate the principle of signal reconstruction. In this embodiment, the first user signal The received signal power is greater than the second user signal The received signal power is explained as follows. For the first user signal, the second user signal is the noise of the first user signal. Then the edge server will first detect the task of the first user signal. Then the first user signal will be decoded and task reasoning will be performed. In this process, Parameterized Classifier The output of the first user signal is determined as: ; in, The predicted label for task reasoning of the first user signal, For the classifier The edge server then uses the network parameters given by Parameterized feature decoder to reconstruct the original feature vector , the original feature vector is the semantic feature extracted on the edge perception device: ; in, is a decoded signal, which includes the encoded semantic information sent by the first user, represents the feature extractor, For feature extractor Then, we use the feature encoder with the same structure as the edge sensing device This signal is encoded to simulate the binary signal sent by the transmitter , the binary signal is the low-dimensional feature vector: ; in, To reconstruct features, Representation feature encoder After the signal is reconstructed, the edge server subtracts the reconstructed feature from the superimposed signal Y , get the second user signal : ; Where P1 is the transmission power of the first user signal. Directly perform the following classification tasks: ; in, A classifier representing the second user signal The network parameters, represents the predicted label of the second user signal.

[0062] In this way, the interference from the first user signal can be eliminated, and only the second user signal is affected by the channel noise.

[0063] In some embodiments, during training, the mean square error is used as a loss function to calculate the difference between the output and the one-hot encoding of the corresponding label, and the edge multi-task semantic communication method further includes: S400, training the feature extractor and feature encoder of each user signal based on the mean square error loss function, calculating the mean square error between the predicted label and the true label through the output of each user signal and the one-hot vector, so as to update the network parameters of the feature extractor and the feature encoder.

[0064] Specifically, in this embodiment, for the neuromorphic dataset, this embodiment defines is the simulation time step of the kth task, and is defined is the number of categories for this task, where For labeled tasks, omitting the subscript will output Expressed as where each time step corresponds to an output, and then All outputs in time are summed and averaged to obtain a length of Output , and its corresponding structure is Finally, the edge server selects the output The maximum value in The corresponding index is used as the inference label of user k At the same time, by creating a The zero vector of the label y is set to 1, and the other positions are set to 0 to obtain the one-hot vector corresponding to the k users. The training result expects that the neuron corresponding to label y will show the strongest excitatory activity, while the activities of other neurons will remain inactive.

[0065] This embodiment uses mean squared error (MSE) as the loss function to calculate the difference between the final output and the one-hot encoding of the corresponding label. The mean squared error loss function is defined as follows: ; in, is the mean square error loss function.

[0066] In order to extract effective semantic features, the edge perception device needs to be trained in advance. This embodiment uses the gradient descent algorithm for training. Taking two users as an example, the feature extractor and feature encoder of the first user signal are first trained. Based on the mean square error loss function, the output of the first user signal is compared with the one-hot vector Calculate predicted labels With the true label The mean square error between , through training mean square error To update ,in, are the network parameters of the feature extractor of the first user signal, is the network parameter of the feature encoder of the first user signal. The training method of the feature extractor and feature encoder of the second user signal is the same as that of the feature extractor and feature encoder of the first user signal. Both use the mean square error (MSE) as the loss function. Calculate the corresponding mean square error , through training mean square error To update ,in, are the network parameters of the feature extractor of the second user signal, are network parameters of the feature encoder of the second user signal.

[0067] S500, training a feature decoder to obtain a loss function of the feature decoder; wherein the loss function of the decoder is composed of a decoded signal and a semantic feature extracted from the neuromorphic data; Specifically, when training the edge server, it is necessary to train the feature decoder and the classifier corresponding to each user. First, the feature decoder is trained. The loss function of the feature decoder can be composed of the decoded signal (reconstructed feature) and the semantic feature (original feature vector): ; in, represents the loss function of the feature decoder, represents the semantic feature of the first user signal, The decoded signal representing the first user signal. It should be noted that when training the feature decoder, the feature extractor and feature encoder of each trained model need to be loaded into the system, and the corresponding model gradient is closed and training is not allowed to ensure that the edge perception device can extract effective semantic features for transmission. Among them, the feature encoder of the first user signal in the edge server directly uses the feature encoder of the first user signal, which can ensure that the edge server can effectively restore the original features of the first user signal.

[0068] S600 , training the classifier of each user signal, and calculating the mean square error between the predicted label and the true label through the output of each user and the one-hot vector.

[0069] Specifically, in the edge server, the training method of the classifier corresponding to each user signal is similar to the training steps of the feature extractor and feature encoder, which is to calculate the mean square error of the inference task of two users. Then the mean square error of the first user signal and the second user signal calculated can be expressed as , Then, the total loss function for training the feature decoder and classifier in the edge server can be expressed as: ; in, is the total training loss function of the feature decoder and classifier.

[0070] This application uses the N-MNIST and DVSGesture datasets for effect verification. The first user signal and the second user signal are responsible for performing the reasoning tasks of N-MNIST and DVSGesture respectively. In order to reflect the effectiveness and fairness of this application, the signal-to-noise ratio is set to follow the exponential distribution and random sampling is performed. The average signal-to-noise ratios of the first user signal and the second user signal are set to be and , and in The output feature of user 1 is fixed to 5×6. The performance of the two datasets under different feature output dimensions is shown in the figure. Figure 4 shown.

[0071] In order to compare the performance differences between ANN and SNN in this system, it is necessary to process the input data corresponding to ANN and sum the neuromorphic data set in the time dimension to generate a static image. The pixel intensity of the image is proportional to the pulse rate of the pixel. The specific conversion process is as follows: Each polarity channel It can be represented as a set of pulse sequences, in channel In the pulse sequence, The pixels are labeled: ; Each element represents a pulse on the frame, and then the pulses in this frame are summed and counted: ; in, Represents the integrated frame data.

[0072] To ensure fairness, the network structure in ANN is kept the same as that in SNN, and the activation function of ANN is replaced by LeakyReLU. Figure 5It is observed that in SNN, the edge server can well recover the signals of user 1 and user 2 and correctly classify them, and the classification accuracy is not significantly attenuated compared to the single-user classification accuracy. In ANN, however, the edge server cannot well separate the signals of the two users from the superimposed signals, and the final classification results show that the two users interfere with each other, so that when the feature dimension of user 2 is increased, the classification accuracy of user 1 is affected.

[0073] In order to verify the effectiveness of SIC technology, a comparative test was conducted between a system using SIC technology and a system not using SIC technology. In the system not using SIC technology, the edge server directly classifies the received superimposed signals. Figure 6 The experimental results show that in the system without SIC technology, although the accuracy of user 2 is close to the classification accuracy of using SIC technology, the accuracy of user 1 is far from that of using SIC technology, which also proves the effectiveness of SIC technology.

[0074] The following table shows the performance of this application under different signal-to-noise ratios, reflecting that this application has good anti-interference ability.

[0075] .

[0076] From the above simulation experiments, it can be seen that the present application has strong robustness in the face of channel fading and communication noise. The edge server can still decode the transmitter features in sequence through SIC technology and maintain a high classification accuracy. The system simulation results show that in the static image recognition task, the reasoning accuracy of the pulse neural network (SNN) is improved by 13% to 35% compared with the artificial neural network (ANN); in the dynamic gesture recognition task, the accuracy is improved by 19% to 52%. The system can still maintain a relatively good classification accuracy under low signal-to-noise ratio conditions. When the average signal-to-noise ratio of static digital recognition users and dynamic gesture recognition users is set to -5dB and -10dB respectively, the corresponding accuracy is 96.40% and 94.44%. This verifies the effectiveness of this method in adapting to channels and anti-noise performance.

[0077] In some embodiments, Figure 7 As shown, the present application also provides an edge multi-task semantic communication device, which includes: The feature extraction and encoding unit is used to obtain the neural morphological data of each user signal, and use a pulse neural network to extract semantic features from the neural morphological data and encode them to obtain a corresponding low-dimensional feature vector; the specific details are as described in an embodiment of an edge multi-task semantic communication method, which will not be repeated here.

[0078] The signal superposition unit is used to allocate power to each low-dimensional feature vector and superimpose them to obtain a superposition signal; the specific details are as described in an embodiment of an edge multi-task semantic communication method, which will not be repeated here.

[0079] The signal separation and decoding unit is used to detect the received signal power of each user signal in the superimposed signal, and to separate and decode each user signal from the superimposed signal in turn according to the received signal power of each user signal to complete the corresponding reasoning task. The specific details are as described in an embodiment of an edge multi-task semantic communication method, which will not be repeated here.

[0080] In some embodiments, the present application further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it is used to implement the following steps: Obtain the neuromorphic data of each user signal, extract semantic features from the neuromorphic data based on a pulse neural network and encode them to obtain a corresponding low-dimensional feature vector; the details are as described in an embodiment of an edge multi-task semantic communication method, which will not be repeated here.

[0081] Power is allocated to each low-dimensional feature vector and then superimposed to obtain a superimposed signal; the details are as described in an embodiment of an edge multi-task semantic communication method, which will not be repeated here.

[0082] The received signal power of each user signal in the superimposed signal is detected, and each user signal is separated and decoded from the superimposed signal in turn according to the received signal power of each user signal to complete the corresponding reasoning task. The specific details are as described in an embodiment of an edge multi-task semantic communication method, which will not be repeated here.

[0083] In some embodiments, the present application further provides a storage medium having a computer program stored thereon, wherein the computer program is used to implement the following steps when executed by a processor: Obtain the neuromorphic data of each user signal, extract semantic features from the neuromorphic data based on a pulse neural network and encode them to obtain a corresponding low-dimensional feature vector; the details are as described in an embodiment of an edge multi-task semantic communication method, which will not be repeated here.

[0084] Power is allocated to each low-dimensional feature vector and then superimposed to obtain a superimposed signal; the details are as described in an embodiment of an edge multi-task semantic communication method, which will not be repeated here.

[0085] The received signal power of each user signal in the superimposed signal is detected, and each user signal is separated and decoded from the superimposed signal in turn according to the received signal power of each user signal to complete the corresponding reasoning task. The specific details are as described in an embodiment of an edge multi-task semantic communication method, which will not be repeated here.

[0086] In summary, the edge multi-task semantic communication method, device, computer equipment and storage medium provided by the present application have the following beneficial effects: By collecting different reasoning tasks and extracting semantic features through multiple edge sensing devices, and performing classification tasks on the edge server, the accuracy of multi-task semantic edge reasoning is improved; Using pulse neural networks instead of traditional artificial neural networks enables edge sensing devices to directly distribute power to signals without normalization and quantization steps, which can effectively avoid the loss of feature information caused by the quantization stage. Compared with traditional artificial neural networks, it can better process data sets containing time information without increasing network complexity, improve the accuracy of reasoning task classification, and reduce computational complexity, thereby reducing energy consumption; The receiving end uses serial interference cancellation technology to effectively separate and recover the directly superimposed signals, effectively solving the problem of traditional multiple access technology allocating fewer resources to users with poor channel information, and taking into account both efficiency and fairness.

[0087] It should be understood that the application of the present application is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to this application.

Claims

1. An edge multi-task semantic communication method, characterized in that: include: Obtain the neuromorphic data of each user signal, extract semantic features from the neuromorphic data based on the spiking neural network, and encode the data to obtain the corresponding low-dimensional feature vector; Power is allocated to each low-dimensional feature vector and then superimposed to obtain a superimposed signal; The received signal power of each user signal in the superimposed signal is detected, and each user signal is separated from the superimposed signal in turn according to the size of the received signal power of each user signal and decoded to complete the corresponding reasoning task.

2. The edge multi-task semantic communication method according to claim 1, characterized in that: The step of detecting the received signal power of each user signal in the superimposed signal, and separating and decoding each user signal from the superimposed signal in sequence according to the received signal power of each user signal to complete the corresponding reasoning task includes: Detecting the received signal power of each user signal in the superimposed signal, and decoding the user signal with the highest received signal power among the user signals; Reconstructing the user signal with the highest received signal power in the superimposed signal and subtracting it from the superimposed signal; The user signal with the largest received signal power in the current superimposed signal is re-detected, decoded and task inference is performed, and then further reconstructed and the reconstructed features are subtracted from the superimposed signal until all user signals are decoded.

3. The edge multi-task semantic communication method according to claim 1, characterized in that: The steps of obtaining the neural morphological data of each user signal, extracting semantic features from the neural morphological data using a pulse neural network, and encoding the neural morphological data to obtain a corresponding low-dimensional feature vector include: Representing the acquired neuromorphic data as an integer vector; wherein the neuromorphic data is collected by multiple edge sensing devices; Extracting semantic features through a feature extractor of a spiking neural network and expanding the extracted semantic features; The expanded semantic features are input into the feature encoder of the pulse neural network for encoding and then output as a low-dimensional feature vector.

4. The edge multi-task semantic communication method according to claim 1, characterized in that: The step of allocating power to each low-dimensional feature vector and superimposing them to obtain a superimposed signal comprises: When superimposing the user signals of various channels, the sizes of the low-dimensional feature vectors corresponding to the user signals of various channels are adjusted so that the user signals of various channels can be transmitted and superimposed on the same channel at the same time.

5. The edge multi-task semantic communication method according to claim 2, characterized in that: The step of reconstructing the user signal with the highest received signal power in the superimposed signal and subtracting it from the superimposed signal comprises: Reconstructing the semantic features using a feature decoder to obtain a decoded signal; the decoded signal includes semantic information corresponding to the encoded user signal; Encoding the decoded signal to simulate binary reconstruction features; The reconstructed feature is subtracted from the superimposed signal.

6. The edge multi-task semantic communication method according to claim 1, characterized in that: During training, the mean square error is used as the loss function to calculate the difference between the output and the unique hot encoding of the corresponding label. The edge multi-task semantic communication method also includes: The feature extractor and feature encoder of each user signal are trained based on the mean square error loss function. The mean square error between the predicted label and the true label is calculated through the output of each user signal and the one-hot vector to update the network parameters of the feature extractor and feature encoder. The feature decoder is trained to obtain a loss function of the feature decoder; wherein the loss function of the feature decoder is composed of the decoded signal and the semantic features extracted from the neuromorphic data; The classifier of each user signal is trained, and the mean square error between the predicted label and the true label is calculated through the output of each user and the one-hot vector.

7. The edge multi-task semantic communication method according to claim 1, characterized in that: In spiking neural networks, the activation function used in forward propagation is a step function, and backpropagation uses a continuous and similarly shaped function as an alternative gradient function.

8. An edge multi-task semantic communication device, characterized in that: include: A feature extraction and encoding unit is used to obtain the neuromorphic data of each user signal, and use a pulse neural network to extract semantic features from the neuromorphic data and encode them to obtain a corresponding low-dimensional feature vector; A signal superposition unit, used for allocating power to each low-dimensional feature vector and superimposing them to obtain a superposition signal; The signal separation and decoding unit is used to detect the received signal power of each user signal in the superimposed signal, and to separate and decode each user signal from the superimposed signal in turn according to the size of the received signal power of each user signal to complete the corresponding inference task.

9. A computer device, characterized in that: It comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it is used to implement the steps in the edge multi-task semantic communication method as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it is used to implement the steps in the edge multi-task semantic communication method as described in any one of claims 1-7.

Images