Edge multi-task semantic communication methods, devices, computer equipment and storage media

Abstract

This application discloses an edge multi-task semantic communication method, apparatus, computer device, and storage medium. The method includes: acquiring neuromorphic data of various user signals; extracting semantic features from the neuromorphic data based on a spiking neural network and encoding them to obtain corresponding low-dimensional feature vectors; allocating power to each low-dimensional feature vector and superimposing them to obtain a superimposed signal; detecting the received signal power of the user signals in the superimposed signal; and separating and decoding each user signal from the superimposed signal according to the magnitude of the received signal power to complete the corresponding inference task. This application uses a spiking neural network to replace the traditional artificial neural network, avoiding the quantization step, improving the accuracy of inference task classification, and separating and decoding the superimposed user signals according to the magnitude of the received signal power, solving the problem of limited resources allocated to users with poor signal information in traditional multiple access technology.

Description

Technical Field

[0001] This application relates to the field of information and communication technology, and in particular to an edge multi-task semantic communication method, apparatus, computer device and storage medium. Background Technology

[0002] Neural network-based intelligent IoT systems are increasingly used for edge inference tasks. In IoT sensing and computing applications, wireless edge sensing devices offload collected sensor data to nearby edge servers with higher computing power for subsequent inference tasks. Due to the increasing number of mobile devices and more complex multimedia IoT applications, limited communication bandwidth has become a major bottleneck for sensor data offloading. Task-oriented semantic communication, characterized by extracting task-related semantic information and discarding redundant information, has the advantage of significantly reducing communication data size while maintaining high inference accuracy. This allows multiple edge devices to complete data transmission more efficiently under the same bandwidth constraints.

[0003] During the transmission of semantic features in semantic communication technology, these features are affected by noise in the wireless environment. This means that changes in the wireless environment can alter the distribution of sensor data collected by the edge server and the signal-to-noise ratio of the wireless channel. Furthermore, co-channel interference becomes particularly prominent when multiple sensing devices simultaneously send semantic features to the same edge server. To address this issue, multiple access technology (MIG) is widely used in wireless communication systems to improve system capacity and transmission efficiency. MIG divides resources into non-interfering portions to avoid signal interference. While MIG performs well in traditional communications, in real-world multi-user communication scenarios, users with poor channel conditions may receive fewer resources or be ignored by the system, impacting their performance. Additionally, to ensure compatibility with modern digital communication standards, traditional neural networks incorporate additional quantization modules. This can cause non-differentiability and convergence issues during the training and backpropagation phases, leading to reduced classification accuracy of the edge server.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this application is to provide an edge multi-task semantic communication method, apparatus, computer device and storage medium to solve the problems in the existing multi-semantic communication technology where users with poor channel conditions may obtain fewer resources or be ignored by the system priority, and the classification accuracy of edge servers is reduced due to the use of quantization modules in traditional neural networks.

[0006] The technical solution of this application is as follows:

[0007] In a first aspect, this application provides an edge multi-task semantic communication method, which includes:

[0008] The neuromorphic data of various user signals are acquired, and semantic features are extracted and encoded based on the spiking neural network to obtain the corresponding low-dimensional feature vectors.

[0009] The superimposed signal is obtained by power allocation and superposition of each low-dimensional eigenvector.

[0010] The received signal power of each user signal in the superimposed signal is detected, and each user signal is separated from the superimposed signal and decoded in sequence according to the magnitude of the received signal power of each user signal to complete the corresponding inference task.

[0011] A further provision of this application states that the step of detecting the received signal power of each user signal in the superimposed signal, and sequentially separating and decoding each user signal from the superimposed signal according to the magnitude of the received signal power of each user signal to complete the corresponding inference task includes:

[0012] The received signal power of each user signal in the superimposed signal is detected, and the user signal with the highest received signal power among all user signals is decoded;

[0013] The user signal with the highest received signal power in the superimposed signal is reconstructed and then subtracted from the superimposed signal.

[0014] The user signal with the highest received signal power in the current superimposed signal is re-detected, and after decoding and task inference, further reconstruction processing is performed. The reconstruction features are subtracted from the superimposed signal until all user signals are decoded.

[0015] A further provision of this application states that the step of acquiring neuromorphic data of each user signal, and then using a spiking neural network to extract semantic features from the neuromorphic data and encode them to obtain corresponding low-dimensional feature vectors includes:

[0016] The acquired neuromorphic data is represented as an integer vector; the neuromorphic data is acquired through multiple edge sensing devices.

[0017] Semantic features are extracted using a feature extractor from a spiking neural network, and the extracted semantic features are then expanded.

[0018] The expanded semantic features are input into the feature encoder of the spiking neural network for encoding and output as a low-dimensional feature vector.

[0019] A further provision of this application states that the step of allocating power to each low-dimensional feature vector and then superimposing them to obtain a superimposed signal includes:

[0020] When superimposing user signals from different channels, the magnitude of the low-dimensional feature vectors corresponding to each user signal is adjusted so that the user signals can be transmitted and superimposed simultaneously on the same channel.

[0021] A further provision of this application states that the step of reconstructing the user signal with the highest received signal power in the superimposed signal and then adding it back to the signal subtracted from the superimposed signal includes:

[0022] A feature decoder is used to reconstruct semantic features to obtain a decoded signal; the decoded signal includes semantic information encoded from the corresponding user signal.

[0023] The decoded signal is encoded to simulate the reconstructed binary characteristics;

[0024] The reconstructed features are subtracted from the superimposed signal.

[0025] A further provision of this application is that, during training, mean squared error is used as the loss function to calculate the difference between the output and the one-hot encoding of the corresponding label. The edge multi-task semantic communication method also includes:

[0026] The feature extractor and feature encoder of each user signal are trained based on the mean square error loss function. The mean square error between the predicted label and the real label is calculated by the output of each user signal and the one-hot vector, so as to update the network parameters of the semantic feature extractor and the semantic feature encoder.

[0027] The feature decoder is trained to obtain its loss function; the loss function of the feature decoder is composed of the decoded signal and the semantic features extracted from the neuromorphic data.

[0028] The classifiers for each user signal are trained, and the mean square error between the predicted label and the true label is calculated by using the output of each user and the one-hot vector.

[0029] In a further embodiment of this application, in the spiking neural network, the activation function used for forward propagation is a step function, and the function used for backpropagation is a continuous function with an approximate shape as an alternative gradient function.

[0030] Secondly, this application also provides an edge multi-task semantic communication device, which includes:

[0031] The feature extraction and encoding unit is used to acquire the neuromorphic data of each user signal, and to extract semantic features from the neuromorphic data using a spiking neural network and then encode them to obtain the corresponding low-dimensional feature vectors.

[0032] The signal superposition unit is used to allocate power to each low-dimensional feature vector and then superimpose them to obtain a superimposed signal.

[0033] The signal separation and decoding unit is used to detect the received signal power of each user signal in the superimposed signal, and to separate and decode each user signal from the superimposed signal in sequence according to the magnitude of the received signal power of each user signal to complete the corresponding inference task.

[0034] Thirdly, this application also provides a computer device including a memory and a processor, wherein the memory stores a computer program, which, when executed by the processor, is used to implement the steps in the edge multitasking semantic communication method as described above.

[0035] Fourthly, this application also provides a storage medium having a computer program stored thereon, which, when executed by a processor, is used to implement the steps in the edge multitasking semantic communication method as described above.

[0036] This application provides an edge multi-task semantic communication method, apparatus, computer device, and storage medium. The method includes: acquiring neuromorphic data of various user signals; extracting semantic features from the neuromorphic data based on a spiking neural network and encoding them to obtain corresponding low-dimensional feature vectors; allocating power to each low-dimensional feature vector and superimposing them to obtain a superimposed signal; detecting the received signal power of each user signal in the superimposed signal; and separating and decoding each user signal from the superimposed signal according to the magnitude of the received signal power of each user signal to complete the corresponding inference task. This application uses a spiking neural network to replace the traditional artificial neural network, avoiding the quantization step, improving the accuracy of inference task classification, and separating and decoding user signals that are directly superimposed according to the power of the user signals. This solves the problem of limited resources allocated to users with poor signal information in traditional multiple access technology, balancing efficiency and fairness. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating the edge multi-task semantic communication method in this application.

[0039] Figure 2This is a comparison chart of the dataset processing results between spiking neural networks and artificial neural networks.

[0040] Figure 3 This is a schematic diagram of an edge multi-task semantic communication method in one embodiment of this application.

[0041] Figure 4 This is a performance comparison chart of a static image dataset and a dynamic gesture recognition dataset under different feature dimensions in one embodiment of this application.

[0042] Figure 5 This is a performance comparison chart of a spiking neural network and an artificial neural network in one embodiment of this application.

[0043] Figure 6 This is a performance comparison chart of whether or not serial interference cancellation technology is used in one embodiment of this application.

[0044] Figure 7 This is a schematic diagram of the edge multi-task semantic communication device in this application. Detailed Implementation

[0045] This application provides an edge multi-task semantic communication method, apparatus, computer device, and storage medium. To make the objectives, technical solutions, and effects of this application clearer and more explicit, the following detailed description is provided with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit the scope of this application.

[0046] In the implementation methods and scope of the claims, unless otherwise specified in the text, the terms "a," "an," "the," and "the" may also include plural forms. If the embodiments of this application involve descriptions of "first," "second," etc., such descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features.

[0047] It should be further understood that the term "comprising" as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when an element is referred to as "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements present. Furthermore, "connected" or "coupled" as used herein can include wireless connections or wireless coupling. The term "and / or" as used herein includes all or any units and all combinations thereof of one or more associatedly listed items.

[0048] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0049] Furthermore, the technical solutions of the various embodiments can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0050] The inventors' research revealed that 5G / 6G communication and artificial intelligence have become representative of a new round of information technology revolution and industrial transformation. In the sensing and computing applications of the Internet of Things (IoT), wireless edge sensing devices offload the collected sensor data to nearby edge servers with higher computing power to perform subsequent inference tasks. The current mainstream semantic communication technology works as follows: first, the edge sensing device extracts semantic features from the collected raw sensing data using a semantic encoder based on an artificial neural network; then, the extracted semantic features are transmitted to the edge server via a wireless channel; finally, the edge server completes the corresponding inference task using the received semantic features.

[0051] During the transmission of semantic features in semantic communication technology, these features are affected by noise in the wireless environment. This means that changes in the wireless environment can alter the distribution of sensor data collected by the edge server and the signal-to-noise ratio of the wireless channel. Furthermore, co-channel interference becomes particularly prominent when multiple sensing devices simultaneously send semantic features to the same edge server. To address this issue, multiple access technologies are widely used in wireless communication systems to improve system capacity and transmission efficiency. Traditional multiple access technologies include frequency division multiple access (FDMA), time division multiple access (TDMA), and code division multiple access (CDMA). FDMA divides the channel into different frequency bands and allocates them to different devices to avoid co-channel interference. TDMA divides time into multiple time slots, allowing devices to transmit data within their respective time slots, reducing interference during the same time period. CDMA, based on spreading code technology, allows multiple devices to transmit data simultaneously on the same frequency, distinguishing device signals using a unique code. Multiple access technologies divide resources into non-interfering portions to avoid mutual interference between signals. While multiple access technologies perform well in traditional communications, in real-world multi-user communication scenarios, users with poor channel conditions may receive fewer resources or be ignored by the system's priority, compromising both efficiency and fairness and thus impacting performance. Furthermore, to ensure compatibility with modern digital communication standards, traditional neural networks incorporate additional quantization modules, which can cause non-differentiability and convergence issues during training and backpropagation, leading to reduced classification accuracy on edge servers.

[0052] Furthermore, most existing research directly considers transmitting continuous signals from the output of traditional neural networks. Traditional neural networks typically use weighted summations and nonlinear activation functions to simulate neuronal behavior, with neurons generally transmitting signals via floating-point numbers. This computational mechanism differs significantly from the discrete, pulse-like signal transmission of biological nervous systems. Although current techniques such as replacing neural functions and semantic-aware codebooks exist to address related issues, they still exhibit some performance degradation compared to analog communication. In addition, traditional neural networks are computationally expensive, requiring significant computational resources for high-precision floating-point calculations. Edge sensing devices, however, typically have limited computational capabilities; once the network depth increases or the data volume becomes large, edge sensing devices struggle to extract relevant features with low latency.

[0053] In summary, existing semantic edge reasoning methods have the following technical shortcomings: 1. Traditional multi-user semantic communication scenarios use multiple access technology, which allocates fewer resources to users with poor channel information, resulting in a severe impact on their performance; 2. In traditional artificial neural networks, edge sensing devices consume a lot of energy during communication, and the data needs to be quantized before being unloaded to the edge server, resulting in poor final classification accuracy of the edge server.

[0054] To address the aforementioned technical problems, this application provides an edge multi-task semantic communication method, apparatus, computer device, and storage medium. It replaces the traditional artificial neural network with a neuromorphic neural network, avoiding quantization steps, improving the accuracy of inference task classification, and reducing computational complexity, thereby reducing energy consumption. Furthermore, it separates and decodes user signals that are directly superimposed based on the power of the user signals, thus solving the problem of limited resources allocated to users with poor signal information in traditional multiple access technologies, and balancing efficiency and fairness.

[0055] Please also refer to Figures 1 to 6 This application provides a preferred embodiment of an edge multi-task semantic communication method.

[0056] In some embodiments, such as Figure 1 As shown, this application provides an edge multi-task semantic communication method, which includes the following steps:

[0057] S100. Obtain the neuromorphic data of each user signal, extract semantic features from the neuromorphic data based on the spiking neural network and encode them to obtain the corresponding low-dimensional feature vectors.

[0058] Specifically, neuromorphic data is time-informed data acquired by edge sensing devices. Unlike Artificial Neural Networks (ANNs), which use continuous-value signals for input and output, Spiking Neural Networks (SNNs) transmit information through discrete-time pulse signals, thus more closely resembling the actual behavior of biological neurons. The basic computational unit of a spiking neural network is the spiking neuron, a neuron that can only output 0 or 1. Each neuron receives signals from neurons in the previous layer and accumulates them to its own membrane potential. When the membrane potential exceeds a set threshold, the neuron generates a pulse and transmits it to the next layer of neurons. Simultaneously, its membrane potential is immediately reset to the reset potential, simulating the firing behavior of biological neurons. A commonly used neuron model for spiking neural networks is the Leaky Integrate-and-Fire (LIF) model, which can be described by three states: charging, discharging, and resetting.

[0059] ;

[0060] The above formula represents the state update of the neuron, where V represents the membrane potential of a neuron after charging and before firing at time t. Indicates the membrane time constant. This represents the membrane potential before discharge. This represents the input to the neuron.

[0061] ;

[0062] In the above formula, This indicates the firing state of a synaptic neuron, when the membrane potential... When time t exceeds a specific threshold At this time, synaptic neurons fire, which is represented as Otherwise, the neurons remain silent, that is... .

[0063] ;

[0064] The above formula indicates that after the neuron fires a spike, the membrane potential will return to its normal value. ,in, This represents the membrane potential after a neuron fires.

[0065] In spiking neural networks, since the output of a neuron is only 0 or 1, the forward propagation process exhibits a step function, and its derivative is the unit impulse function. If the system directly uses the unit impulse function as the gradient function, the entire training process will be highly unstable. To address this issue, a common training method for spiking neural networks is to use a continuous function with a similar shape as a substitute gradient function during backpropagation, such as the sigmoid function or the arctan function. In this embodiment, the arctan function is used. and its derivative The corresponding information is as follows:

[0066] ;

[0067] .

[0068] Because a spiking neural network is used, the activation function of the neurons in the network is a step function, and the output of each stage is a binary signal, which allows it to avoid the quantization step. At the same time, the morphological architecture and event-driven nature of the spiking neural network (outputting only 0 and 1) significantly reduce computational complexity, enabling it to process datasets containing time information without increasing network complexity.

[0069] Commonly used datasets for spiking neural networks include gesture recognition (DVSGesture), digit classification (N-MNIST, MNIST-DVS), and image recognition (CIFAR-DVS, N-Caltech101). This implementation uses two neuromorphic datasets: N-MNIST (a static image dataset) and DVSGesture128 (a dynamic gesture recognition dataset). N-MNIST is the spiking version of the MNIST dataset, collected by an asynchronous time-series image sensor. The DVS128Gesture dataset contains 11 gesture data collected under three different lighting conditions using a DVS128 camera, including clapping, waving arms, and rotating arms. Each data sample contains a set of events, each represented by four parameters: Where t is the timestamp, x and y are the horizontal and vertical coordinates of the event respectively, and p is the polarity of the event. The event is represented as follows:

[0070] ;

[0071] in, Represents an event, where n is a positive integer. express Events of time.

[0072] To convert the raw event stream into frame data, a time window is set, and events are integrated within this window to generate a single frame. In this embodiment, the frame sizes of the N-MNIST and DVS128Gesture datasets are set to 5 and 20, respectively. The integrated frame data can be expressed by the following formula:

[0073] ;

[0074] in, Represents frame data, express Events of time.

[0075] To compare performance with ANNs of the same network structure and this embodiment, the dataset is processed into a reasonable input format. Therefore, the corresponding dataset will have its frame size set to 1, that is, the original event stream will be integrated. For example... Figure 2 As shown, Figure 2 The first row in the table corresponds to the dataset results for the ANN. Figure 2 The second and third rows show the SNN dataset results. It can be observed that the static image dataset in the first column, after processing, still allows for identification of the digit "0" based on its outline. However, the second and third columns show the results of processing the dynamic gesture recognition dataset; the details of the arm movements cannot be discerned from the images alone. Figure 2Because the SNN in the model includes time information, it can clearly distinguish the direction of the arm swing.

[0076] Low-dimensional feature vectors refer to a set of vectors extracted from original high-dimensional data using specific algorithms. These vectors effectively represent the main features of the data, reducing its dimensionality while retaining key information, facilitating subsequent data processing and analysis. One edge sensing device acquires one user signal; in this embodiment, two edge sensing devices are used. In this embodiment, event-driven cameras are used to observe neuromorphic data with temporal information. Based on a spiking neural network, semantic features are extracted from the neuromorphic data and encoded to obtain low-dimensional feature vectors, such as... Figure 3 As shown.

[0077] S200. Power allocation is performed on each low-dimensional eigenvector, and the superimposed signal is obtained.

[0078] Specifically, please combine Figure 3 Because the low-dimensional feature vectors obtained after semantic extraction and encoding of neuromorphic data based on spiking neural networks are binary signals, no additional normalization or quantization steps are required, and power allocation can be performed directly. For example, when there are two users, the signal of user 1 is denoted as the first user signal, and the signal of user 2 is denoted as the second user signal. The received signal power of the first user signal is greater than that of the second user signal. Subsequently, each user signal is transmitted in a shared broadband environment, and each feature vector is sent accordingly.

[0079] S300: Detect the received signal power of each user signal in the superimposed signal, and sequentially separate and decode each user signal from the superimposed signal according to the magnitude of the received signal power of each user signal to complete the corresponding inference task.

[0080] Specifically, the superimposed signal refers to the signal where various user signals are superimposed together, such as the signal where the first user signal and the second user signal are superimposed. These user signals are collected by edge sensing devices, which can be used for 2, 3, or 4 users. Received signal power refers to the product of the user signal's transmit power and the channel coefficient; generally, higher power is assigned to users with better channel conditions. After the edge server receives the superimposed signal, it classifies the user signals according to their received signal power and decodes and separates them sequentially from highest to lowest power. For example, if the received signal power of the first user signal is greater than that of the second user signal, the first user signal is first decoded and its corresponding inference task is completed. Then, the first user signal is subtracted from the superimposed signal before the second user signal is decoded and its corresponding inference task is completed.

[0081] In the above technical solutions, this application provides a non-orthogonal multiple access digital semantic communication method for edge multi-task inference. It replaces traditional artificial neural networks with spiking neural networks, avoiding quantization steps, improving the accuracy of inference task classification, and reducing computational complexity, thereby reducing energy consumption. Furthermore, it separates and decodes user signals that are directly superimposed based on their power levels, thus solving the problem of limited resources allocated to users with poor signal information in traditional multiple access technologies, achieving a balance between efficiency and fairness.

[0082] In some embodiments, the step of acquiring neuromorphic data of each user signal, extracting semantic features from the neuromorphic data using a spiking neural network, and encoding the data to obtain corresponding low-dimensional feature vectors includes:

[0083] S110. Represent the acquired neuromorphic data as an integer vector; wherein, the neuromorphic data is acquired through multiple edge sensing devices;

[0084] S120. Extract semantic features using a feature extractor from a spiking neural network and expand the extracted semantic features;

[0085] S130. The expanded semantic features are input into the feature encoder of the spiking neural network for encoding and output as a low-dimensional feature vector.

[0086] In this embodiment, an integer vector refers to an input vector that is limited to integers. This improves computation speed and saves computational resources and time. When extracting collected neuromorphic data, the collected neuromorphic data needs to be represented as an integer vector. After extracting semantic features, the feature extractor expands the semantic features and inputs them into the fully connected feature encoder of the spiking neural network, finally outputting a low-dimensional feature vector.

[0087] Taking two user signals as an example, firstly, the edge sensing device... Representing the k-th user, the neuromorphic data collected by the device is represented as an integer vector. Where k=1,2. The dataset used in this embodiment is the N-MNIST and DVSGesture neuromorphic datasets. The first user signal and the second user signal are the classification tasks of the N-MNIST and DVSGesture datasets, respectively. That is, user 1 is responsible for the static digit classification task, and user 2 is responsible for the dynamic gesture recognition task. The corresponding network models are shown in Tables 1 and 2:

[0088] Table 1:

[0089]

[0090] Table 2:

[0091]

[0092] Among them, feature extractor It consists of convolutional layers, where k=1,2, Indicates feature extractor, The network parameters of the feature extractor are used to extract semantic features. Then, it is expanded and input into the feature encoder of the spiking neural network. In the context of spiking neural networks, the feature encoder... It is a fully connected semantic feature encoder. The network parameters of the feature encoder, and the final output feature dimension. The low-dimensional feature vectors are 5×6 and 20×24, respectively.

[0093] In some embodiments, please refer to Figure 1 and combined Figure 3 The step of allocating power to each low-dimensional feature vector and then superimposing them to obtain the superimposed signal includes:

[0094] S210. When superimposing user signals from various channels, the size of the low-dimensional feature vectors corresponding to each user signal is adjusted so that the user signals from various channels can be transmitted and superimposed simultaneously on the same channel.

[0095] Specifically, when an edge sensing device sends compressed semantic features to an edge server via a wireless channel, the size of each semantic feature needs to be adjusted to allow all user signals to be transmitted in the same channel, since the signals from each user need to be superimposed in the channel. Taking two user signals as an example, the first user signal and the second user signal need to be transmitted and superimposed in the same channel, requiring adjustment of the feature sizes of the two user signals. In this way, user 1 can transmit 16 data points while user 2 can transmit 1 data point.

[0096] In this embodiment, the signal between the edge sensing device and the edge server is a Rayleigh channel. Therefore, the edge server receives two user signal input signals (i.e., superimposed signals) at the receiving end, which can be represented as:

[0097] ;

[0098] Where Y represents the superimposed signal, Represents the channel coefficient. This represents the transmission power of the k-th user. , Indicates the maximum transmission power. N represents Additive White Gaussian Noise (AWGN), where, This represents a Gaussian distribution.

[0099] In some embodiments, please refer to Figure 1 and combined Figure 3 The steps of detecting the received signal power of each user signal in the superimposed signal, and separating and decoding each user signal from the superimposed signal according to the magnitude of the received signal power of each user signal to complete the corresponding inference task include:

[0100] S310. Detect the received signal power of each user signal in the superimposed signal, and decode the user signal with the highest received signal power among all user signals.

[0101] S320. After reconstructing the user signal with the highest received signal power in the superimposed signal, subtract it from the superimposed signal.

[0102] S330. Re-detect the user signal with the highest received signal power in the current superimposed signal, and after decoding and task inference, further reconstruct the signal by subtracting the reconstructed features from the superimposed signal until all user signals are decoded.

[0103] Specifically, when the edge server receives superimposed signals, it detects the inference tasks of near-end users with higher received signal power and classifies user signals according to the received signal power of each user signal.

[0104] In classifying, decoding, and reasoning user signals, this embodiment employs Serial Interference Cancellation (SIC) to address co-channel interference and optimize inference performance in order to recover the semantic features of users from mixed signals superimposed with noise from different users and perform inference tasks. The edge processor first processes the user signal with the highest received signal power, i.e., it decodes the user signal with the highest received signal power. Decoding and task inference are performed by the classifier corresponding to the user signal. Simultaneously, the user signal with the highest received signal power in the current superimposed signal is reconstructed so that the user signal with the highest received signal power can be subtracted from the input superimposed signal to obtain the signals of the remaining users, thus achieving signal separation and reducing the interference level of the remaining signals. After decoding and reasoning for one user signal, the above process is repeated: the user signal with the highest relative received signal power in the current superimposed signal is re-detected, decoded, and reasoned, and then reconstructed again, subtracting the reconstructed features from the superimposed signal. This process is iterated until all user signals have been decoded and reasoned. Figure 3 As shown, User 1 outputs the number 3, and User 2 outputs a waving gesture.

[0105] Further, please refer to Figure 1 and combined Figure 3 The step of reconstructing the user signal with the highest received signal power in the superimposed signal and then overlaying it onto the signal subtracted from the superimposed signal includes:

[0106] S321. A feature decoder is used to reconstruct semantic features to obtain a decoded signal; the decoded signal includes semantic information encoded from the corresponding user signal.

[0107] S322. Encode the decoded signal to simulate the reconstructed binary characteristics;

[0108] S323. Subtract the reconstructed features from the superimposed signal.

[0109] Specifically, different user signals have corresponding classifiers to complete the decoding and task inference. This embodiment uses two user signals to illustrate the signal reconstruction principle. In this embodiment, the first user signal is used... The received signal power is greater than that of the second user signal. The received signal power is explained below. For the first user signal, the second user signal is noise of the first user signal. Therefore, the edge server will first detect the task of the first user signal. The first step is to decode and perform task inference operations on the first user signal. During this process, [the following is omitted as it is not relevant to the translation]. Parameterized classifier The output of the first user signal is determined as follows:

[0110] ;

[0111] in, Predicted labels for task inference of the first user signal. For classifier The network parameters. Then, the edge server uses the network parameters provided by... A parameterized feature decoder is used to reconstruct the original feature vector. The original feature vector is the semantic feature extracted at the edge-sensing device:

[0112] ;

[0113] in, The decoded signal contains the encoded semantic information sent by the first user. Indicates feature extractor, For feature extractor The network parameters are then determined. Then, a feature encoder with the same structure as the edge-aware device is used. The signal is encoded to simulate the binary signal transmitted by the transmitter. This binary signal is the low-dimensional eigenvector:

[0114] ;

[0115] in, To reconstruct features, Represents the feature encoder The network parameters. After signal reconstruction, the edge server subtracts the reconstructed features from the superimposed signal Y. , obtain the second user signal :

[0116] ;

[0117] Where P1 is the transmission power of the first user signal. Subsequently, the second user signal... Perform the following categorization tasks directly:

[0118] ;

[0119] in, Classifier representing the second user signal Network parameters, This represents the predicted label for the second user signal.

[0120] In this way, interference from the first user signal can be eliminated, leaving only the second user signal affected by channel noise.

[0121] In some embodiments, the mean squared error is used as the loss function during training to calculate the difference between the output and the one-hot encoding of the corresponding label. The edge multi-task semantic communication method further includes:

[0122] S400: The feature extractor and feature encoder of each user signal are trained based on the mean square error loss function. The mean square error between the predicted label and the real label is calculated by the output of each user signal and the one-hot vector, so as to update the network parameters of the feature extractor and feature encoder.

[0123] Specifically, in this embodiment, for the neuromorphic dataset, this embodiment defines... For the simulation time step of the k-th task, define Let be the number of categories for this task, where For tasks that have already been marked, omit the index and output the result. Expressed as A tensor, where each time step corresponds to an output, then... Sum and average all outputs within a time period to obtain a result of length . Output Its corresponding structure is Finally, the edge server selects the output. The maximum value in The corresponding index serves as the inference label for user k. At the same time, by creating a length of The zero vector is obtained by setting the position corresponding to label y to 1 and the other positions to 0, in order to obtain the one-hot vector corresponding to user k. The training result is expected to show the strongest excitatory activity in the neuron corresponding to label y, while the activity of other neurons remains inactive.

[0124] This embodiment uses Mean Squared Error (MSE) as the loss function to calculate the difference between the final output and the one-hot encoding of the corresponding label. The definition of the Mean Squared Error loss function is as follows:

[0125] ;

[0126] in, This is the mean squared error loss function.

[0127] To extract effective semantic features, edge-aware devices require pre-training of the model. This embodiment uses the gradient descent algorithm for training. Taking two users as an example, the feature extractor and feature encoder for the first user's signal are first trained. Based on the mean squared error loss function, the feature extractor and feature encoder are trained by combining the output of the first user's signal with the one-hot vector. Calculate predicted labels With real labels Mean square error between By training mean square error Update ,in, The network parameters for the feature extractor of the first user signal. These are the network parameters of the feature encoder for the first user signal. The training method for the feature extractor and feature encoder for the second user signal is the same as that for the first user signal, using mean squared error (MSE) as the loss function, and applying the output of the second user signal to the one-hot vector. Calculate the corresponding mean square error By training mean square error Update ,in, The network parameters for the feature extractor of the second user signal. The network parameters of the feature encoder for the second user signal.

[0128] S500. Train the feature decoder to obtain the loss function of the feature decoder; wherein, the loss function of the decoder is composed of the decoded signal and the semantic features extracted from the neuromorphic data;

[0129] Specifically, when training the edge server, it is necessary to train the feature decoder and the classifier corresponding to each user. First, the feature decoder is trained. The loss function of the feature decoder can be composed of the decoded signal (reconstructed features) and semantic features (original feature vectors):

[0130] ;

[0131] in, This represents the loss function of the feature decoder. The semantic features representing the first user signal, This represents the decoded signal of the first user signal. It should be noted that when training the feature decoder, the feature extractors and feature encoders of the trained models need to be loaded into the system, and the corresponding model gradients should be disabled to prevent training. This ensures that the edge sensing device can extract effective semantic features for transmission. Specifically, the feature encoder of the first user signal in the edge server directly uses the original feature encoder of the first user signal, ensuring that the edge server can effectively recover the original features of the first user signal.

[0132] S600: Train the classifier for each user signal, and calculate the mean square error between the predicted label and the true label by using the output of each user and the one-hot vector.

[0133] Specifically, in the edge server, the training method for the classifiers corresponding to each user signal is similar to the training steps for the feature extractor and feature encoder. Both involve calculating the mean square error of the inference tasks of the two users. Therefore, the mean square errors of the first user signal and the second user signal can be expressed as follows: , Therefore, the total training loss function for the feature decoder and classifier in the edge server can be expressed as:

[0134] ;

[0135] in, This is the total training loss function for the feature decoder and classifier.

[0136] This application utilizes the N-MNIST and DVSGesture datasets for performance verification. The first user signal and the second user signal are responsible for performing inference tasks on the N-MNIST and DVSGesture datasets, respectively. To demonstrate the effectiveness and fairness of this application, the signal-to-noise ratio (SNR) is set to follow an exponential distribution and is randomly sampled. The average SNR of the first user signal and the second user signal are set as follows: and and in The range varies, and the output feature corresponding to user 1 is fixed at 5×6. The performance of the two datasets under different feature output dimensions is as follows: Figure 4 As shown.

[0137] To compare the performance differences between ANN and SNN in this system, the input data corresponding to ANN needs to be processed. The neuromorphic dataset is summed over time to generate a static image, where pixel intensity is proportional to the pixel pulse rate. The specific transformation process is as follows: Each polarity channel... It can be represented as a sequence of pulses in the channel. In the pulse sequence, located The pixels are labeled as:

[0138] ;

[0139] Each element represents a pulse in that frame, and then the pulses within that frame are summed and counted:

[0140] ;

[0141] in, This represents the integrated frame data.

[0142] To ensure fairness, the network structure in ANN remains the same as that in SNN, except that the activation function in ANN is replaced by LeakyReLU. This can be seen from... Figure 5 It was observed that in SNN, the edge server was able to recover the signals of User 1 and User 2 well and classify them correctly, with no significant decrease in classification accuracy compared to single-user classification accuracy. However, in ANN, the edge server could not effectively separate the signals of the two users from the superimposed signals, and the final classification results showed that both users interfered with each other, causing the classification accuracy of User 1 to be affected when the feature dimension of User 2 was increased.

[0143] To verify the effectiveness of SIC technology, a comparative experiment was conducted between systems using SIC and those not using SIC technology. In the system without SIC technology, the edge server directly classifies the received superimposed signals. Figure 6 The experimental results show that in the system without SIC technology, although the accuracy of User 2 is similar to that of the system using SIC technology, the accuracy of User 1 is far inferior to that of the system using SIC technology, which proves the effectiveness of SIC technology.

[0144] The table below shows the performance of this application under different signal-to-noise ratios, demonstrating that this application has good anti-interference capabilities.

[0145] .

[0146] The simulation experiments above demonstrate that this application exhibits strong robustness against channel fading and communication noise. The edge server, using SIC technology, can still effectively decode the transmitter features sequentially and maintain a high classification accuracy. System simulation results show that in static image recognition tasks, the inference accuracy of the spiking neural network (SNN) is improved by 13% to 35% compared to the artificial neural network (ANN); in dynamic gesture recognition tasks, the accuracy improvement reaches 19% to 52%. Furthermore, the system maintains relatively good classification accuracy even under low signal-to-noise ratio (SNR) conditions. When the average SNR for static digit recognition users and dynamic gesture recognition users is set to -5dB and -10dB, respectively, the corresponding accuracies are 96.40% and 94.44%. This verifies the effectiveness of the proposed method in adapting to channel conditions and enhancing noise resistance.

[0147] In some embodiments, such as Figure 7 As shown, this application also provides an edge multi-task semantic communication device, which includes:

[0148] The feature extraction and encoding unit is used to acquire the neuromorphic data of each user signal, and to extract semantic features from the neuromorphic data using a spiking neural network and encode them to obtain the corresponding low-dimensional feature vectors; as described in an embodiment of an edge multi-task semantic communication method, it will not be repeated here.

[0149] The signal superposition unit is used to allocate power to each low-dimensional feature vector and superimpose them to obtain a superimposed signal; as described in an embodiment of an edge multi-task semantic communication method, it will not be repeated here.

[0150] The signal separation and decoding unit is used to detect the received signal power of each user signal in the superimposed signal, and sequentially separate and decode each user signal from the superimposed signal according to the magnitude of the received signal power of each user signal to complete the corresponding inference task. This is specifically described in an embodiment of an edge multi-task semantic communication method, and will not be repeated here.

[0151] In some embodiments, this application also provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, is used to perform the following steps:

[0152] The neuromorphic data of various user signals are acquired, and semantic features are extracted and encoded based on the spiking neural network to obtain the corresponding low-dimensional feature vectors; as described in an embodiment of an edge multi-task semantic communication method, which will not be repeated here.

[0153] The superimposed signal is obtained by power allocation and superposition of each low-dimensional feature vector; as described in an embodiment of an edge multi-task semantic communication method, it will not be repeated here.

[0154] The received signal power of each user signal in the superimposed signal is detected. Based on the magnitude of the received signal power of each user signal, each user signal is sequentially separated from the superimposed signal and decoded to complete the corresponding inference task. This is specifically described in an embodiment of an edge multi-task semantic communication method, and will not be repeated here.

[0155] In some embodiments, this application also provides a storage medium storing a computer program thereon, which, when executed by a processor, is used to perform the following steps:

[0156] The neuromorphic data of various user signals are acquired, and semantic features are extracted and encoded based on the spiking neural network to obtain the corresponding low-dimensional feature vectors; as described in an embodiment of an edge multi-task semantic communication method, which will not be repeated here.

[0157] The superimposed signal is obtained by power allocation and superposition of each low-dimensional feature vector; as described in an embodiment of an edge multi-task semantic communication method, it will not be repeated here.

[0158] The received signal power of each user signal in the superimposed signal is detected. Based on the magnitude of the received signal power of each user signal, each user signal is sequentially separated from the superimposed signal and decoded to complete the corresponding inference task. This is specifically described in an embodiment of an edge multi-task semantic communication method, and will not be repeated here.

[0159] In summary, the edge multi-task semantic communication method, apparatus, computer device, and storage medium provided in this application have the following beneficial effects:

[0160] By collecting different reasoning tasks and extracting semantic features through multiple edge-aware devices, and performing classification tasks on the edge server, the task accuracy of multi-task semantic edge reasoning is improved.

[0161] Using spiking neural networks instead of traditional artificial neural networks enables edge sensing devices to directly distribute the power of signals without normalization and quantization steps. This effectively avoids the loss of feature information during the quantization stage. Compared with traditional artificial neural networks, it can better handle datasets containing time information without increasing network complexity, improve the accuracy of classification for inference tasks, and reduce computational complexity, thereby reducing energy consumption.

[0162] The receiver uses serial interference cancellation technology, which can effectively separate and recover signals that are directly superimposed. This effectively solves the problem of limited resources allocated to users with poor channel information in traditional multiple access technology, and balances efficiency and fairness.

[0163] It should be understood that the application of this application is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. An edge multi-task semantic communication method, characterized in that, include: The process involves acquiring neuromorphic data from various user signals, extracting semantic features from the neuromorphic data using a spiking neural network, and encoding these features to obtain corresponding low-dimensional feature vectors. Specifically, this includes: The acquired neuromorphic data is represented as an integer vector; wherein, the neuromorphic data is acquired through multiple edge sensing devices; the integer vector refers to a vector whose input vector is limited to integers; Semantic features are extracted using a feature extractor from a spiking neural network, and the extracted semantic features are then expanded. The expanded semantic features are input into the feature encoder of a spiking neural network for encoding, and then output a low-dimensional feature vector. The superimposed signal is obtained by power allocation and superposition of each low-dimensional feature vector; the step of obtaining the superimposed signal by power allocation and superposition of each low-dimensional feature vector includes: When superimposing user signals from various channels, the magnitude of the low-dimensional feature vectors corresponding to each user signal is adjusted so that the user signals from various channels are transmitted and superimposed simultaneously on the same channel. The received signal power of each user signal in the superimposed signal is detected, and each user signal is separated from the superimposed signal and decoded in sequence according to the magnitude of the received signal power of each user signal to complete the corresponding inference task.

2. The edge multi-task semantic communication method according to claim 1, characterized in that, The steps of detecting the received signal power of each user signal in the superimposed signal, and sequentially separating and decoding each user signal from the superimposed signal according to the magnitude of the received signal power of each user signal to complete the corresponding inference task include: The received signal power of each user signal in the superimposed signal is detected, and the user signal with the highest received signal power among all user signals is decoded; The user signal with the highest received signal power in the superimposed signal is reconstructed and then subtracted from the superimposed signal. The user signal with the highest received signal power in the current superimposed signal is re-detected, and after decoding and task inference, further reconstruction processing is performed. The reconstruction features are subtracted from the superimposed signal until all user signals are decoded.

3. The edge multi-task semantic communication method according to claim 2, characterized in that, The step of reconstructing the user signal with the highest received signal power in the superimposed signal and then subtracting it from the superimposed signal includes: A feature decoder is used to reconstruct semantic features to obtain a decoded signal; the decoded signal includes semantic information encoded from the corresponding user signal. The decoded signal is encoded to simulate the reconstructed binary characteristics; The reconstructed features are subtracted from the superimposed signal.

4. The edge multi-task semantic communication method according to claim 1, characterized in that, During training, mean squared error is used as the loss function to calculate the difference between the output and the one-hot encoding of the corresponding label. Edge multi-task semantic communication methods also include: The feature extractor and feature encoder of each user signal are trained based on the mean square error loss function. The mean square error between the predicted label and the real label is calculated by the output of each user signal and the one-hot vector, so as to update the network parameters of the feature extractor and feature encoder. The feature decoder is trained to obtain its loss function; the loss function of the feature decoder consists of the decoded signal and the semantic features extracted from the neuromorphic data. The classifiers for each user signal are trained, and the mean square error between the predicted label and the true label is calculated by using the output of each user signal and the one-hot vector.

5. The edge multi-task semantic communication method according to claim 1, characterized in that, In spiking neural networks, the activation function used in forward propagation is a step function, while in backpropagation a continuous function with an approximate shape is used as an alternative gradient function.

6. An edge multi-task semantic communication device based on the edge multi-task semantic communication method according to any one of claims 1-5, characterized in that, include: The feature extraction and encoding unit is used to acquire the neuromorphic data of each user signal, and to extract semantic features from the neuromorphic data using a spiking neural network and then encode them to obtain the corresponding low-dimensional feature vectors. The signal superposition unit is used to allocate power to each low-dimensional feature vector and then superimpose them to obtain a superimposed signal. The signal separation and decoding unit is used to detect the received signal power of each user signal in the superimposed signal, and to separate and decode each user signal from the superimposed signal in sequence according to the magnitude of the received signal power of each user signal to complete the corresponding inference task.

7. A computer device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, which, when executed by the processor, is used to implement the steps in the edge multitasking semantic communication method as described in any one of claims 1-5.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it is used to implement the steps in the edge multitasking semantic communication method as described in any one of claims 1-5.

Images