Multi-task semantic communication method, system and device and storage medium

By extracting and encoding semantic feature vectors in multi-user semantic communication and forwarding signals by using the same base station, the problems of large number and high cost in multi-user semantic broadcast communication are solved, and more efficient channel utilization and transmission accuracy are achieved.

CN118890243APending Publication Date: 2024-11-01PENG CHENG LAB
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202410854755.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-27
Publication Date
2024-11-01

AI Technical Summary

Technical Problem

The existing semantic communication technology requires a large number of base stations in multi-user semantic broadcast communication, resulting in underutilization of channel transmission capabilities and high network complexity and cost.

Method used

By extracting semantic eigenvectors at the task sending end, quantizing and modulating, forwarding signals from the same base station, encoding in separate subspaces, the task receiving end performs channel equalization and semantic inference to reduce interference from multiple users.

Benefits of technology

It significantly reduces multi-user interference, reduces transmission costs, improves transmission accuracy, and eliminates the need to design complex communication systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118890243A_ABST
    Figure CN118890243A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a multi-task semantic communication method, system and device and a storage medium, and relates to the technical field of communication. According to the method, a semantic feature vector of a transmitting signal is extracted by using a feature extraction unit, quantization and modulation are performed to obtain a semantic feature signal, the semantic feature signal is transmitted to a base station, and the base station generates a transmission signal corresponding to each task receiving end; and the task receiving end performs channel equalization on the transmission signal and inputs the transmission signal into a semantic reasoning model for semantic reasoning to obtain target processing data corresponding to the communication task. Semantic information is extracted into discrete semantic feature vectors in distinguishable semantic subspaces at a task sending end, then signals are forwarded by using a base station, semantic feature signals corresponding to different communication tasks are encoded in the separated subspaces, and the semantic feature signals are approximately orthogonal to each other. Since the semantic features of multiple users are in distinguishable feature domains, the interference is significantly reduced, a complex communication system does not need to be designed, the transmission cost is reduced, and the transmission accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a multi-task semantic communication method, system, device and storage medium. Background Art

[0002] As an emerging intelligent communication method, semantic communication deeply explores the semantic dimension of data to achieve highly abstract representation of information and intelligent and simple transmission. In related technologies, semantic communication is mainly based on single-user semantic communication. If multi-user semantic broadcast communication is to be carried out, two terminals that interact with each other are usually equipped with a base station each, and the two communicate separately. However, this type of communication network requires a large number of base stations, which cannot fully utilize the channel transmission capacity, resulting in a complex transmission network and high transmission costs. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a multi-task semantic communication method, system, device and storage medium, which can reduce the number of base stations and reduce transmission costs.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a multi-task semantic communication method, which is applied to a task sending end, the task sending end corresponds to a task receiving end, and is used to perform a communication task. The method includes:

[0005] Acquire a transmission signal corresponding to the communication task, and extract a semantic feature vector of the transmission signal using a feature extraction unit;

[0006] quantizing the semantic feature vector to obtain a discrete semantic feature vector, and modulating the discrete semantic feature vector to obtain a semantic feature signal;

[0007] The semantic feature signal is transmitted to a base station, so that the base station obtains a transmission signal corresponding to each of the task receiving ends according to the semantic feature signals of different task sending ends, and sends the transmission signal to multiple task receiving ends one by one. The task receiving end performs channel equalization on the transmission signal to obtain an estimated signal, and inputs the estimated signal into a pre-trained semantic reasoning model for semantic reasoning to obtain target processing data corresponding to the communication task.

[0008] In some embodiments, quantizing the semantic feature vector to obtain a discrete semantic feature vector includes:

[0009] Acquire a quantization codebook shared between the task sending end and the corresponding task receiving end, wherein the quantization codebook includes a preset number of quantization vectors;

[0010] Dividing the semantic feature vector into at least one linear vector, wherein the dimension of the linear vector is consistent with the dimension of the quantization vector;

[0011] The discrete semantic feature vector is obtained according to the inner product of the quantization vector and the linear vector.

[0012] In some embodiments, obtaining the discrete semantic feature vector according to the inner product of the quantization vector and the linear vector includes:

[0013] Perform inner products of the linear vector and the quantization vector respectively to obtain inner product values, and select the index position of the quantization vector in the quantization codebook corresponding to the smallest inner product value;

[0014] The discrete semantic feature vector is obtained based on the index position corresponding to each of the linear vectors.

[0015] In some embodiments, the transmission signal includes an image to be classified or an image to be reconstructed;

[0016] When the transmitted signal is an image to be classified, the semantic reasoning model is a neural network model, and the target processing data is a classification result of the image to be classified;

[0017] When the transmitted signal is an image to be reconstructed, the feature extraction unit is an encoder unit, the semantic reasoning model is a decoder unit, and the target processing data is a reconstruction result corresponding to the image to be reconstructed.

[0018] In some embodiments, when the feature extraction unit is an encoder unit, extracting the semantic feature vector of the transmission signal using the feature extraction unit includes:

[0019] After performing a first downsampling and self-attention processing on the transmission signal using a first encoding block, a first feature map is obtained, wherein a width and a height of the first feature map are both half of the transmission signal;

[0020] Using a second encoding block to perform a second downsampling and self-attention processing on the first feature map, thereby obtaining a second feature map, wherein the width and height of the second feature map are both half of the first feature map;

[0021] The second feature map is downsampled for the third time and self-attention processed by a third encoding block, and then encoded to obtain the semantic feature vector.

[0022] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes a multi-task semantic communication method, which is applied to a task receiving end, and the method includes:

[0023] Acquire a transmission signal from a base station, where the transmission signal is calculated by the semantic feature signal obtained by the base station according to the multi-task semantic communication method according to any one of the first aspects;

[0024] Performing channel equalization on the transmission signal to obtain an estimated signal;

[0025] The estimated signal is input into a pre-trained semantic reasoning model for semantic reasoning to obtain target processing data corresponding to the communication task.

[0026] In some embodiments, performing channel equalization on the transmission signal to obtain an estimated signal includes:

[0027] Acquire channel state information between the task receiving end and the base station, and calculate conjugate transposed data of the channel state information;

[0028] The transmission signal is divided by the conjugate transposed data to obtain the estimated signal.

[0029] In some embodiments, when the semantic reasoning model is a neural network model, before inputting the estimation signal into a pre-trained semantic reasoning model for semantic reasoning, the method further includes:

[0030] Acquire a training data set, wherein the training data set includes a transmission sample signal and a data processing label, wherein the transmission sample signal is obtained by the base station according to a semantic sample signal corresponding to each task sending end, and the semantic sample signal is obtained according to the transmission sample signal;

[0031] Obtaining an estimated sample signal according to the transmitted sample signal, and using the estimated sample signal to train the semantic reasoning model to obtain a data prediction result;

[0032] A first loss value is calculated according to the data prediction result and the data processing label, a second loss value is calculated according to the transmission sample signal and the total semantic sample signal, and a third loss value is calculated according to the transmission sample signal and the transmission sample signal;

[0033] A total loss value is obtained according to the first loss value, the second loss value and the third loss value, and the weight of the semantic reasoning model is adjusted based on the total loss value to obtain the trained semantic reasoning model.

[0034] In some embodiments, when the semantic reasoning model is a decoder unit, before inputting the estimation signal into a pre-trained semantic reasoning model for semantic reasoning, the method further includes:

[0035] Acquire a training data set, wherein the training data set includes a transmission sample signal and a data processing label, wherein the transmission sample signal is obtained by the base station according to a semantic sample signal corresponding to each task sending end, and the semantic sample signal is obtained according to the transmission sample signal;

[0036] Obtaining an estimated sample signal according to the transmitted sample signal, and using the estimated sample signal to train the semantic reasoning model to obtain a data prediction result;

[0037] Calculate a total loss value according to the data processing label and the data prediction result;

[0038] The weight of the semantic reasoning model is adjusted based on the total loss value to obtain the trained semantic reasoning model.

[0039] To achieve the above purpose, a third aspect of an embodiment of the present application proposes a multi-task semantic communication system, including:

[0040] A plurality of task sending ends, wherein the task sending ends are used to obtain semantic feature signals according to the multi-task semantic communication method according to any one of the first aspects, and send the semantic feature signals to a base station respectively, and each of the task sending ends corresponds to a task receiving end;

[0041] A base station, the base station is used to obtain the semantic feature vectors from a plurality of different task sending ends to obtain a transmission signal corresponding to each task receiving end, and send the transmission signal to the plurality of task receiving ends one by one;

[0042] A plurality of the task receiving ends are used to obtain a transmission signal, and utilize the multi-task semantic communication method described in any one of the second aspects to obtain target processing data corresponding to the communication task according to the transmission signal.

[0043] In some embodiments, the step of the base station acquiring the semantic feature vectors from a plurality of different task sending ends to obtain a transmission signal corresponding to each task receiving end includes:

[0044] Selecting target sending ends from the task sending ends one by one, and acquiring the task receiving ends corresponding to the target sending ends as target receiving ends;

[0045] Obtaining target transmission data according to the transmission parameters between the target transmitting end and the base station and the semantic feature vector of the target transmitting end;

[0046] Obtain interference transmission data according to the transmission parameters between the other task transmitting ends and the base station and the semantic feature vector of the task transmitting end, and accumulate the interference transmission data to obtain total interference data;

[0047] The transmission signal of the target receiving end is obtained according to the target transmission data and the total interference data, and the transmission signal of each task receiving end is obtained one by one.

[0048] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the method described in the first aspect or the second aspect when executing the computer program.

[0049] To achieve the above-mentioned purpose, the fifth aspect of an embodiment of the present application proposes a storage medium, which is a storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0050] The multi-task semantic communication method, system, device and storage medium proposed in the embodiment of the present application obtain the transmission signal corresponding to the communication task, and use the feature extraction unit to extract the semantic feature vector of the transmission signal, quantize the semantic feature vector to obtain a discrete semantic feature vector, modulate the discrete semantic feature vector to obtain a semantic feature signal, and transmit the semantic feature signal to the base station, so that the base station obtains the transmission signal corresponding to each task receiving end according to the semantic feature signals of different task sending ends, sends the transmission signal to multiple task receiving ends one by one, and the task receiving end performs channel equalization on the transmission signal to obtain an estimated signal, and inputs the estimated signal into a pre-trained semantic reasoning model for semantic reasoning to obtain target processing data corresponding to the communication task. The embodiment of the present application extracts semantic features of the transmission signal at different task sending ends, extracts the semantic information into discrete semantic feature vectors in a distinguishable semantic subspace, and then uses the same base station to forward the signal. During the transmission process, the semantic feature signals corresponding to different communication tasks are encoded in separate subspaces, and are approximately orthogonal to each other. Then, the task receiver of the corresponding communication task directly obtains the corresponding estimated signal through channel equalization and performs semantic reasoning on the estimated signal. Since the semantic features of multiple users are in a distinguishable feature domain, the interference of multiple users can be significantly reduced without the need to design a complex communication system, thereby reducing transmission costs and improving transmission accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of a multi-task semantic communication system provided in an embodiment of the present application.

[0052] Figure 2 It is an optional flowchart of the multi-task semantic communication method provided in an embodiment of the present application.

[0053] Figure 3It is a flowchart of using a feature extraction unit to extract a semantic feature vector of a transmission signal provided in an embodiment of the present application.

[0054] Figure 4 It is a schematic diagram of the structure of the encoder unit in the embodiment of the present application.

[0055] Figure 5 It is a flow chart of quantizing a semantic feature vector to obtain a discrete semantic feature vector provided in an embodiment of the present application.

[0056] Figure 6 It is a flowchart provided by an embodiment of the present application, in which a base station acquires semantic feature vectors from multiple different task sending ends to obtain a transmission signal corresponding to each task receiving end.

[0057] Figure 7 It is an optional flowchart of the multi-task semantic communication method provided in an embodiment of the present application.

[0058] Figure 8 A schematic diagram of the training process of the semantic reasoning model provided in an embodiment of the present application.

[0059] Fig. 9 It is a schematic diagram of the structure of the decoder unit provided in an embodiment of the present application.

[0060] Fig.10 Another schematic diagram of the training process of the semantic reasoning model provided in an embodiment of the present application.

[0061] Fig.11 A schematic diagram of the classification performance of an image to be classified provided in an embodiment of the present application.

[0062] Fig.12 Another schematic diagram of the classification performance of the image to be classified provided in an embodiment of the present application.

[0063] Fig.13 A schematic diagram of the reconstruction performance of an image to be reconstructed provided in an embodiment of the present application.

[0064] Fig.14 Another schematic diagram of the reconstruction performance of the image to be reconstructed provided in an embodiment of the present application.

[0065] Fig.15 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0067] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0069] First, some nouns involved in this application are analyzed:

[0070] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0071] As an emerging intelligent communication method, semantic communication deeply explores the semantic dimension of data to achieve highly abstract representation of information and intelligent and simple transmission. In related technologies, semantic communication is mainly based on single-user semantic communication. If multi-user semantic broadcast communication is to be carried out, two terminals that interact with each other are usually equipped with a base station each, and the two communicate separately. However, this type of communication network requires a large number of base stations, which cannot fully utilize the channel transmission capacity, resulting in a complex transmission network and high transmission costs.

[0072] Based on this, the embodiments of the present application provide a multi-task semantic communication method, system, device and storage medium, which extracts semantic features of the transmitted signal at different task sending ends, extracts the semantic information into discrete semantic feature vectors in distinguishable semantic subspaces, and then uses the same base station to forward the signal. During the transmission process, the semantic feature signals corresponding to different communication tasks are encoded in separate subspaces, and are approximately orthogonal to each other. Then, the task receiving end corresponding to the communication task directly obtains the corresponding estimated signal through channel equalization, and performs semantic reasoning on the estimated signal. Since the semantic features of multiple users are in distinguishable feature domains, the interference of multiple users can be significantly reduced, and there is no need to design a complex communication system, which reduces transmission costs and improves transmission accuracy.

[0073] The embodiments of the present application provide a multi-task semantic communication method, system, device and storage medium, which are specifically illustrated by the following embodiments. First, the multi-task semantic communication method in the embodiments of the present application is described.

[0074] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.

[0075] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0076] The multi-task semantic communication method provided in the embodiment of the present application relates to the field of communication technology. The multi-task semantic communication method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be a computer program running in a terminal or a server side. For example, a computer program can be a native program or a software module in an operating system; it can be a local (Native) application (APP, Application), that is, a program that needs to be installed in the operating system to run, such as a client that supports multi-task semantic communication, or it can be a small program, that is, a program that can be run only by downloading it to a browser environment; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plug-in. Among them, the terminal communicates with the server via a network. The multi-task semantic communication method can be executed by a terminal or a server, or by a terminal and a server in collaboration.

[0077] In some embodiments, the terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer or a smart watch. The server may be an independent server, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms; or a service node in a blockchain system, wherein each service node in the blockchain system forms a peer-to-peer (P2P, PeerTo Peer, P2P) network, and the P2P protocol is an application layer protocol running on the Transmission Control Protocol (TCP) protocol. The terminal and the server may be connected via Bluetooth, Universal Serial Bus (USB) or a network or other communication connection methods, which are not limited in this embodiment.

[0078] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0079] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0080] First, the multi-task semantic communication system provided by the embodiment of the present application is described.

[0081] Reference Figure 1 , Figure 1 It is a schematic diagram of a multi-task semantic communication system provided in an embodiment of the present application. Figure 1 The multi-task semantic communication system is a multi-user semantic multiple access network based on single-carrier frequency division multiple access (SFDMA). It includes multiple task transmitters, a base station and multiple task receivers. The task transmitter corresponds to one task receiver one by one to form a task group. Each task group is used to perform a communication task. It can be understood that the communication task can be image reconstruction, image classification, information transmission, etc. The communication tasks of different task groups can be the same or different, and this embodiment does not limit this. Figure 1 N task groups are taken as an example for explanation, and each task group includes a corresponding task sending end and a task receiving end.

[0082] In one embodiment, each task sending end corresponds to a semantic encoder. Taking the i-th task sending end as an example, the parameter of the semantic encoder is expressed as φ i , can be based on the transmission signal s of the task sender i Get the semantic feature signal x i Then the base station receiver simultaneously receives the semantic feature signals sent by multiple task transmitters. At this time, the different semantic feature signals are superimposed to obtain the transmission signal y corresponding to each task receiver. i , the base station transmits the signal y i Forwarding, the transmission signal y i Sent to the corresponding task receiving end.

[0083] Next, the task receiving end uses the semantic decoder to decode the transmission signal, specifically: i Perform channel equalization to obtain an estimated signal corresponding to the transmitted signal of the task transmitter Then the estimated signal Send it to the pre-trained semantic reasoning model for semantic reasoning to obtain the target processing data corresponding to the communication task

[0084] In one embodiment, Figure 1 The semantic encoder at the task sending end includes a feature extraction unit, a quantizer and a digital modulation unit, and the semantic decoder at the task receiving end includes an equalizer and a semantic reasoning model.

[0085] Among them, taking the i-th task sender as an example, its expected task processing data is u i , the task sender uses the feature extraction unit From the transmitted signal s i The semantic feature vector a is obtained from i , followed by the quantizer ψ i (·) for the semantic feature vector a i Quantize to get the discrete semantic feature vector z i , and then use the digital modulation unit Θ i (·) for the discrete semantic feature vector z i Modulate to get the semantic feature signal x i Correspondingly, the base station obtains the semantic feature signal x of each task sender i After that, the transmission signal y corresponding to each task receiving end is generated i , based on the channel state information g between the task receiving end and the base station i The transmission signal is sent to the corresponding task receiving end. The task receiving end uses an equalizer to adjust the transmission signal y i Perform channel equalization to obtain the estimated signal For example, the estimated signal at the receiving end of the i-th task is Then, the semantic reasoning model corresponding to the i-th task receiver is used Estimated signal Perform semantic reasoning to obtain target processing data Theoretically, the target processing data And the task processing data is u i Basically the same. Among them, θ i Representation semantic reasoning model Network parameters.

[0086] Combine the following Figure 1 A multi-task semantic communication method applied to a task sending end in an embodiment of the present application is described.

[0087] Figure 2 is an optional flowchart of the multi-task semantic communication method provided in an embodiment of the present application. Figure 2 The method may include but is not limited to steps 110 to 130. It can also be understood that this embodiment Figure 2 The order of step 110 to step 130 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0088] Step 110: Acquire a transmission signal corresponding to the communication task, and use a feature extraction unit to extract a semantic feature vector of the transmission signal.

[0089] In one embodiment, due to different communication tasks, the transmitted signal includes an image to be classified or an image to be reconstructed, wherein the image to be classified is used to perform an image classification task, and the image to be reconstructed is used to perform an image reconstruction task. When the transmitted signal is an image to be classified, the semantic reasoning model at the task receiving end is a neural network model, and the target processing data is the classification result of the image to be classified. When the transmitted signal is an image to be reconstructed, the feature extraction unit at the task sending end is an encoder unit, the semantic reasoning model at the task receiving end is a decoder unit, and the target processing data is the reconstruction result corresponding to the image to be reconstructed.

[0090] In one embodiment, when the transmitted signal is an image to be classified, for the i-th task transmitter, the feature extraction unit It can be a neural network model that can transmit the signal s i Extract features and encode them to get the semantic feature vector a i , this feature extraction process is expressed as:

[0091]

[0092] In one embodiment, when the transmitted signal is an image to be reconstructed, the feature extraction unit is an encoder unit, referring to Figure 3 , Figure 3 This is a flow chart of extracting a semantic feature vector of a transmission signal using a feature extraction unit provided in an embodiment of the present application, specifically including:

[0093] Step 310: Use the first encoding block to perform a first downsampling and self-attention processing on the transmitted signal to obtain a first feature map.

[0094] In one embodiment, when the transmitted signal is an image to be reconstructed, the transmitted signal is expressed as: Among them, H and W represent the height and width of the transmitted signal, and C represents the number of channels of the transmitted signal.

[0095] At this time, refer to Figure 4 , Figure 4 It is a schematic diagram of the structure of the encoder unit in the embodiment of the present application. Figure 4The encoder unit includes three encoding blocks, namely the first encoding block, the second encoding block and the third encoding block. The first encoding block, the second encoding block and the third encoding block are all Swin Transformer structures, including a coding patch block and a Swin Transformer coding block. The coding patch block is used to divide the input image into sizes to obtain multiple non-overlapping patches, which are then sent to the Swin Transformer coding block. For the patches obtained after division, self-attention calculation is performed inside each patch, and then the information is integrated through shift operations, thereby realizing the encoding process of the input image. The Swin Transformer structure uses the coding patch block and the Swin Transformer coding block to work together to extract and transform the semantic information in the image.

[0096] In one embodiment, the first encoding block first encodes the transmitted signal s i Perform the first downsampling and divide the transmitted signal into Non-overlapping patches are marked in order from top left to bottom right to obtain a concatenated sequence, which is embedded in Embedding and then processed by self-attention through the Swin Transformer block to obtain the first feature map. The first feature map is represented as: The width and height of the first feature map are both half of the transmitted signal, and the number of channels C1 is consistent with the number of channels of the transmitted signal.

[0097] Step 320: Use the second encoding block to perform a second downsampling and self-attention processing on the first feature map to obtain a second feature map.

[0098] In one embodiment, then in the second encoding block, the first feature map is first downsampled for the second time using the encoding patch block to reduce the size of the first feature map and the number of patches, thereby reducing the image resolution and adjusting the number of channels, which helps to reduce the amount of subsequent calculations. After the patch processing, the height and width of the feature map are reduced to half of the original, while the number of channels is expanded to twice the original. Subsequently, the adjusted patch is processed again by the Swin Transformer encoding block to output the second feature map, which is represented as: The width and height of the second feature map are both half of the first feature map, and the number of channels C2 is twice the number of channels C1 of the first feature map.

[0099] Step 330: Use the third encoding block to perform a third downsampling and self-attention processing on the second feature map, and then encode it to obtain a semantic feature vector.

[0100] In one embodiment, the second feature map is further downsampled by the coding patch block of the third coding block, and then processed by the Swin Transformer coding block to obtain a final feature map, whose width and height are half of the second feature map, and the number of channels C3 is twice the number of channels C2 of the second feature map. Finally, the feature map is encoded and the semantic feature vector is output. The semantic feature vector contains the key semantic information in the transmitted signal, and its size is significantly reduced compared to the transmitted signal.

[0101] Overall, the feature extraction unit That is, the encoder unit, the process of extracting the semantic feature vector of the transmitted signal is expressed as:

[0102] a i =Layer3(Layer2(Layer1(s i )))

[0103] Among them, Layer1 represents the first coding block, Layer2 represents the second coding block, and Layer3 represents the third coding block.

[0104] Step 120: quantizing the semantic feature vector to obtain a discrete semantic feature vector, and modulating the discrete semantic feature vector to obtain a semantic feature signal.

[0105] In one embodiment, in order to improve the transmission efficiency of the communication system and simplify the base station processing process, the embodiment of the present application performs a semantic feature vector a i Quantize the semantic feature vector using a quantizer to obtain a discrete semantic feature vector. Figure 5 , Figure 5 This is a flow chart of quantizing a semantic feature vector to obtain a discrete semantic feature vector provided by an embodiment of the present application, specifically including:

[0106] Step 510: Acquire a quantization codebook shared between a task sending end and a corresponding task receiving end.

[0107] The quantization codebook is a learnable codebook, that is, it can be adjusted according to the training process, which includes a preset number of quantization vectors, the quantization codebook M shared by the i-th task group i It is expressed as:

[0108]

[0109] Among them, K represents the preset number, d represents the dimension of each quantized vector, and m i,j represents the jth quantization vector in the i-th quantization codebook.

[0110] Step 520: Divide the semantic feature vector into at least one linear vector.

[0111] Among them, the dimension of the linear vector is consistent with the dimension d of the quantized vector, so the i-th semantic feature vector a i The linear vector a obtained by division i ′ is expressed as:

[0112]

[0113] Step 530: Obtain a discrete semantic feature vector according to the inner product of the quantization vector and the linear vector.

[0114] In one embodiment, first, inner products are performed on the linear vectors and the quantization vectors to obtain inner product values, and the index position of the quantization vector corresponding to the smallest inner product value in the quantization codebook is selected, and a discrete semantic feature vector is obtained based on the index position corresponding to each linear vector. In other words, each linear vector is vector inner producted with the quantization vector, and then the corresponding inner product value is obtained. It can be understood that each linear vector corresponds to K inner product values, from which the quantization vector corresponding to the smallest inner product value is selected as the index vector, and the position of the index vector in the quantization codebook is used as the index position z. i,j In this way, we get the index position of each linear vector, and then get the discrete semantic feature vector according to the index position. Assuming there are n linear vectors, the discrete semantic feature vector z is i It is expressed as:

[0115]

[0116] Among them, i (·) represents the quantizer at the sending end of the i-th task.

[0117] In one embodiment, the discrete semantic feature vector z is obtained i After that, the digital modulator For the discrete semantic feature vector z i Modulate to get the semantic feature signal x i , convert the bit stream into a signal form suitable for transmission on the channel, for example, 4PSK modulation can be used, expressed as:

[0118] x i =Θ i (z i ),i∈{1,..,N}

[0119] Step 130: Transmit the semantic feature signal to a base station.

[0120] In one embodiment, the base station receives semantic feature signals x transmitted by different task sending ends. iAfterwards, the transmission signal corresponding to each task receiving end is obtained according to the semantic feature signals of different task sending ends, and the transmission signal is sent to multiple task receiving ends one by one. Then the task receiving end performs channel equalization on the transmission signal to obtain an estimated signal, and the estimated signal is input into a pre-trained semantic reasoning model for semantic reasoning to obtain the target processing data corresponding to the communication task.

[0121] In one embodiment, referring to Figure 6 , Figure 6 The flowchart of the base station provided in the embodiment of the present application acquiring semantic feature vectors from multiple different task sending ends to obtain a transmission signal corresponding to each task receiving end specifically includes:

[0122] Step 610: Select target sending ends from the task sending ends one by one, and obtain the task receiving end corresponding to the target sending end as the target receiving end.

[0123] For example, if the i-th task sending end is selected as the target sending end, then the task receiving end in the corresponding i-th task group is selected as the target receiving end.

[0124] Step 620: Obtain target transmission data according to the transmission parameters between the target transmitting end and the base station and the semantic feature vector of the target transmitting end.

[0125] In one embodiment, the transmission parameters are calculated based on the channel fading coefficient and transmission power between the target transmitting end and the base station. For the i-th task transmitting end, g i represents the channel fading coefficient from it to the base station, that is, the channel state information, p i is its transmission power, and the transmission parameters are expressed as:

[0126]

[0127] Therefore, the target launch data is expressed as:

[0128]

[0129] Step 630: Obtain interference transmission data according to the transmission parameters between other task transmitting ends and the base station and the semantic feature vector of the task transmitting end, and accumulate the interference transmission data to obtain total interference data.

[0130] In one embodiment, the interference transmission data of the j-th task transmitting end relative to the i-th task transmitting end is expressed as: The total interference data is therefore expressed as:

[0131]

[0132] Step 640: Obtain the transmission signal of the target receiving end according to the target transmission data and the total interference data, and obtain the transmission signal of each task receiving end one by one.

[0133] Among them, the transmission signal y corresponding to the i-th task receiving end i It is expressed as:

[0134]

[0135] Among them, n i The mean is 0 and the variance is δ 2 The complex noise vector satisfies

[0136] The embodiment of the present application extracts semantic features of the transmitted signal at the transmitting end of different tasks, extracts the semantic information into discrete semantic feature vectors in a distinguishable semantic subspace, and then uses the same base station to forward the signal. During the transmission process, the semantic feature signals corresponding to different communication tasks are encoded in separate subspaces, which are approximately orthogonal to each other. Then, the task receiving end corresponding to the communication task directly obtains the corresponding estimated signal through channel equalization, and performs semantic reasoning on the estimated signal. Since the semantic features of multiple users are in a distinguishable feature domain, the interference of multiple users can be significantly reduced, and there is no need to design a complex communication system, which reduces the transmission cost and improves the transmission accuracy.

[0137] Combine the following Figure 1 A multi-task semantic communication method applied to a task receiving end in an embodiment of the present application is described.

[0138] Figure 7 is an optional flowchart of the multi-task semantic communication method provided in an embodiment of the present application. Figure 7 The method in the embodiment may include but is not limited to steps 710 to 740. Figure 7 The order of step 710 to step 730 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0139] Step 710: Acquire a transmission signal from a base station.

[0140] In one embodiment, the transmission signal is calculated by the base station using a semantic feature signal obtained by the multi-task semantic communication method described in any one of the above embodiments.

[0141] Step 720: Perform channel equalization on the transmission signal to obtain an estimated signal.

[0142] In one embodiment, channel equalization is performed in an ideal state, that is, it is assumed that the channel state information between each task receiving end and the base station is known information. It is understandable that if the channel state information is unknown, channel estimation is required at this time, and common channel estimation methods such as LS channel estimation, MMSE, LMMSE, etc. can be used, which is not limited in this embodiment.

[0143] Among them, the process of performing channel equalization on the transmission signal to obtain the estimated signal specifically includes: obtaining the channel state information between the task receiving end and the base station, calculating the conjugate transposed data of the channel state information, and dividing the transmission signal by the conjugate transposed data to obtain the estimated signal.

[0144] The estimated signal at the receiver of the i-th task It is expressed as:

[0145]

[0146] In the above formula, due to the channel state information g i It is known that, therefore, in the channel equalization process, the embodiment of the present application can divide the transmission signal by This converts the channel effect from a multiplication operation to an addition operation, thereby reducing the learning burden of the subsequent semantic reasoning model.

[0147] Step 730: Input the estimated signal into a pre-trained semantic reasoning model for semantic reasoning to obtain target processing data corresponding to the communication task.

[0148] In one embodiment, when the transmitted signal is an image to be classified, the semantic reasoning model is a neural network model, and the target processing data is the classification result of the image to be classified. When the transmitted signal is an image to be reconstructed, the feature extraction unit is an encoder unit, the semantic reasoning model is a decoder unit, and the target processing data is the reconstruction result corresponding to the image to be reconstructed.

[0149] In one embodiment, after the signal is equalized, the estimated signal Feed into semantic reasoning network The semantic reasoning network obtains semantic features from it, performs classification processing or image reconstruction processing on it, and outputs the final result to obtain the target processing data The reasoning process is expressed as:

[0150]

[0151] Among them, θ i Represents the parameters of the semantic reasoning network at the receiver end of the i-th task.

[0152] The semantic reasoning models of different types of transmitted signals and their corresponding training processes are described below.

[0153] In one embodiment, when the transmitted signal is an image to be classified, the semantic reasoning model is a neural network model. Figure 8 , Figure 8 A schematic diagram of the training process of the semantic reasoning model provided in the embodiment of the present application specifically includes:

[0154] Step 810: Obtain a training data set.

[0155] First, the transmission sample signal generated by each task transmitter and the corresponding data processing label are extracted, quantized and modulated to obtain the corresponding semantic sample signal. Then, the base station obtains the transmission sample signal of each task receiver according to the semantic sample signals of different task transmitters. The data processing label is the processing data expected to be obtained by the transmission sample signal.

[0156] In one embodiment, there are L samples in the training data set, corresponding to L transmission sample signals, and the number of sampling times of the semantic feature signal by the semantic encoder of each task sending end is M, that is, for each semantic sample signal corresponding to the transmission sample signal, the semantic encoder divides the semantic sample signal into M discrete data points, and correspondingly, the transmission sample signal also corresponds to M discrete transmission data, and the data processing label also corresponds to M discrete label data. The number of sampling here will affect the complexity and accuracy of data processing at the task sending end and the task receiving end. This embodiment sets the number of sampling according to actual needs.

[0157] Step 820: Obtain an estimated sample signal based on the transmitted sample signal, and use the estimated sample signal to train a semantic reasoning model to obtain a data prediction result.

[0158] In one embodiment, according to the above-mentioned method of calculating the estimated signal, an estimated sample signal is obtained according to the transmitted sample signal, and the estimated sample signal is sent to the semantic reasoning model for prediction to obtain a data prediction result.

[0159] Step 830: A first loss value is calculated based on the data prediction result and the data prediction result, a second loss value is calculated based on the transmission sample signal and the semantic sample signal, and a third loss value is calculated based on the transmission sample signal and the transmission sample signal.

[0160] Among them, the training process of the semantic reasoning model in the embodiment of the present application can be regarded as an optimization problem, and its objective function is expressed as:

[0161]

[0162] Among them, ω i >0 is an adjustable weight factor, I(U i ; Yi ′) represents the first loss value, which is the data prediction result Y i ′ and data processing label U i The correlation between them, I(X; Y i ) represents the second loss value, which is the transmission sample signal Y i The uncertainty between the total semantic sample signal X of multiple task senders, I(S i ; Y i ) represents the third loss value, which is the transmission sample signal Y i With the transmitted sample signal S i The correlation between them, I(X; Y i )-I(S i ; Y i ) can ensure the balance between downstream task performance and communication overhead during the communication process.

[0163] In order to develop a highly robust semantic encoder, assume that the i-th transmitted sample signal is s i ′, the data processing label is u i ′, the semantic sample signal is X i ′, the total semantic sample signal is x′, and the transmission sample signal is Y i ′, the data prediction result is y i ′, the above formula can be equivalent to:

[0164]

[0165] Among them, φ i represents the weight of the i-th feature extraction unit, Indicates in s i ′ and u i ′ is the expectation of the probability distribution under the condition, Indicates in s i ′ and y i ′ is the expectation of the probability distribution under the condition, Indicates that between x′ and y i ′, represents the probability distribution between and , H(Y i ′|x′) represents x′ and Y i The information entropy between ′, Indicates i ′ and Y i The information entropy between ′.

[0166] Since the posterior distribution function The calculation of involves high-dimensional integrals, so the present embodiment introduces two approximate distributions to overcome this problem. and To approximate high-dimensional distributions and θ i is a parameter that can be learned by the semantic reasoning model. Therefore, the first term in the above equation can be simplified as:

[0167]

[0168] Among them, D KL ≥0, indicating DL divergence.

[0169] Next, due to the transmission sample signal Y i ′ is the transmission representation corrupted by additional Gaussian noise, so the entropy function can be derived The lower bound of is:

[0170]

[0171] Therefore, the objective function in the embodiment of the present application can be converted into:

[0172]

[0173] in, and H(Y i ′|x′) in the conditional entropy term for the parameter φ i Analytical calculations were performed.

[0174] When the semantic encoder is a DNN-based encoder, the channel model can be obtained according to the definition of factorial distribution, where the factorial distribution is defined as: Therefore, the channel model is composed of the conditional distribution p(s i ′,u i ′) is derived.

[0175] And the entropy function can be decomposed as follows:

[0176]

[0177] Therefore, if M batches of data For each small batch of data Channel Noise Perform L-point sampling, and the objective function in the embodiment of the present application can be written as:

[0178]

[0179] Among them, x j (m,l) represents the semantic sample signal in the mth mini-batch data, y i (m,l) Represents the transmitted sample signal in the mth small batch of data, both of which are discrete feature vectors.

[0180] Step 840: Obtain a total loss value according to the first loss value, the second loss value, and the third loss value, and adjust the weight of the semantic reasoning model based on the total loss value to obtain a trained semantic reasoning model corresponding to each terminal.

[0181] In one embodiment, the total loss value, i.e., the above objective function, is expressed as:

[0182]

[0183] The design of the total loss value in the embodiment of the present application achieves a trade-off between the amount of semantic information transmitted, interference between users and inference accuracy. Therefore, the weight of the semantic reasoning model is adjusted according to the total loss value until the preset training termination condition is reached, and a trained semantic reasoning model suitable for different communication tasks can be obtained.

[0184] In one embodiment, when the transmitted signal is an image to be reconstructed, the semantic reasoning model is a decoder unit. Fig. 9 , Fig. 9 is a schematic diagram of the structure of a decoder unit provided in an embodiment of the present application. Figure 4 ,from Fig. 9 It can be seen that the decoder unit and the encoder unit have a symmetrical structure, including three decoding blocks, namely the first decoding block, the second decoding block and the third decoding block, wherein the first decoding block, the second decoding block and the third decoding block are all Swin Transformer structures, including a Swin Transformer decoding block and a decoding patch block. The Swin Transformer decoding block corresponds to the Swin Transformer encoding block and is used to decode and reconstruct the signal. The decoding patch block is used to upsample the input image. Therefore, in the embodiment of the present application, the estimated signal First, enter the first decoding block, after reconstruction and upsampling, by becomes Then, the second decoding block is entered, and after reconstruction and upsampling, becomes Finally, entering the third decoding block, after reconstruction and upsampling, the becomes H×W×3. At this time, the target processing data is the reconstruction result corresponding to the image to be reconstructed. The whole process is expressed as:

[0185]

[0186] Among them, Layer1' represents the first decoding block, Layer2' represents the second decoding block, and Layer3' represents the third decoding block.

[0187] The following describes the training process of the decoder unit. Fig.10 , Fig.10 Another schematic diagram of the training process of the semantic reasoning model provided in the embodiment of the present application specifically includes:

[0188] Step 1010: Obtain a training data set.

[0189] The training data set is generated in a similar manner to the training data set in step 810 .

[0190] Step 1020: Obtain an estimated sample signal based on the transmitted sample signal, and use the estimated sample signal to train a semantic reasoning model to obtain a data prediction result.

[0191] Step 1030: Calculate the total loss value based on the data processing label and the data prediction result.

[0192] Among them, the total loss value is expressed as:

[0193]

[0194] in, Represents the total loss value corresponding to the image to be reconstructed, n represents the number of samples, s i ′ represents the data processing label in image reconstruction, Represents the data prediction results in image reconstruction.

[0195] Step 1040: Adjust the weight of the semantic reasoning model based on the total loss value to obtain a trained semantic reasoning model corresponding to each terminal.

[0196] In one embodiment, the weight of the semantic reasoning model is adjusted according to the total loss value until a preset training termination condition is reached, and a trained semantic reasoning model can be obtained. The training termination condition here can be that the trained semantic reasoning model satisfies the evaluation index MS-SSIM, which is a multi-scale structural similarity and is an index used to measure the quality of image reconstruction. It evaluates the similarity between images by considering the degree of structural similarity at different scales.

[0197] In one embodiment, referring to Fig.11 , Fig.11 A schematic diagram of the classification performance of an image to be classified provided in an embodiment of the present application. Fig.11In the figure, the images of two users U1 and U2 are classified respectively, where the number of quantization bits is d=16 and the number of quantization bits is d=64, and the training signal-to-noise ratio is 5dB. In the figure, the vertical axis represents the classification accuracy (percentage), the horizontal axis represents the signal-to-noise ratio (dB), Upperbound represents the upper bound of performance, DeepJSCC represents the deep joint source channel coding scheme in the related technology, and SFDMA represents the multi-task semantic communication method provided in the embodiment of the present application. Fig.11 It can be seen that the classification accuracy of the SFDMA scheme provided in the embodiment of the present application is always better than the Deep JSCC scheme in the related technology, and is close to the upper bound of the performance corresponding to different users, which proves the effectiveness of the multi-task semantic communication method of the embodiment of the present application in image classification.

[0198] In one embodiment, referring to Fig.12 , Fig.12 Another schematic diagram of the classification performance of the image to be classified provided by the embodiment of the present application. In the figure, handwritten digits of different users need to be recognized. According to the recognition results, the semantic feature signals of users U1, U2 and U3 are separated in the semantic feature subspace, and the classification accuracy of each user is above 94%.

[0199] In one embodiment, referring to Fig.13 , Fig.13 A schematic diagram of the reconstruction performance of an image to be reconstructed provided in an embodiment of the present application. Fig.13 For the visualization of semantic feature subspace, Fig.13 As can be seen from the figure, the images of the two users U1 and U2 can be separated in the semantic feature subspace, and the reconstruction process will not be interfered by other users.

[0200] In one embodiment, referring to Fig.14 , Fig.14 Another schematic diagram of the reconstruction performance of the image to be reconstructed provided in an embodiment of the present application. Fig.14 The comparison of reconstruction performance using peak signal-to-noise ratio (PSNR) as the evaluation indicator is shown in . Fig.14 It can be seen that the reconstruction performance of the SFDMA scheme provided in the embodiment of the present application is always better than the DeepJSCC scheme in the related art, and is close to the upper bound of the performance, which proves the effectiveness of the multi-task semantic communication method of the embodiment of the present application in image reconstruction.

[0201] The embodiments of the present application improve the communication performance in the multi-user broadcast communication scenario. The semantic feature signals of different users are approximately orthogonal, and the semantic decoder of the corresponding task receiving end will only decode the relevant information of the corresponding task sending end to obtain the target processing data, and will not decode the remaining semantic feature signals of the broadcast channel, thereby improving the communication performance of multi-user broadcast communication in task-oriented communication. And each task receiving end only decodes the corresponding semantic feature signal, which can enhance the confidentiality of multi-user semantic broadcast communication transmission and improve transmission security performance.

[0202] The technical solution provided in the embodiment of the present application obtains the transmission signal corresponding to the communication task, and uses the feature extraction unit to extract the semantic feature vector of the transmission signal, quantizes the semantic feature vector to obtain a discrete semantic feature vector, modulates the discrete semantic feature vector to obtain a semantic feature signal, and transmits the semantic feature signal to the base station, so that the base station obtains the transmission signal corresponding to each task receiving end according to the semantic feature signals of different task sending ends, sends the transmission signal to multiple task receiving ends one by one, and the task receiving end performs channel equalization on the transmission signal to obtain an estimated signal, and inputs the estimated signal into a pre-trained semantic reasoning model for semantic reasoning to obtain target processing data corresponding to the communication task. The embodiment of the present application extracts semantic features of the transmission signal at different task sending ends, extracts the semantic information into discrete semantic feature vectors in a distinguishable semantic subspace, and then uses the same base station to forward the signal. During the transmission process, the semantic feature signals corresponding to different communication tasks are encoded in separate subspaces, and are approximately orthogonal to each other. Then, the task receiver of the corresponding communication task directly obtains the corresponding estimated signal through channel equalization and performs semantic reasoning on the estimated signal. Since the semantic features of multiple users are in a distinguishable feature domain, the interference of multiple users can be significantly reduced without the need to design a complex communication system, thereby reducing transmission costs and improving transmission accuracy.

[0203] The present application also provides an electronic device, including:

[0204] at least one memory;

[0205] at least one processor;

[0206] at least one program;

[0207] The program is stored in the memory, and the processor executes the at least one program to implement the multi-task semantic communication method implemented in the present application. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.

[0208] See also Fig.15 , Fig.15The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0209] The processor 1501 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0210] The memory 1502 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1502 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1502, and the processor 1501 calls and executes the multi-task semantic communication method of the embodiment of this application;

[0211] Input / output interface 1503, used to implement information input and output;

[0212] Communication interface 1504, used to realize communication interaction between the device and other devices, which can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.); and

[0213] A bus 1505 that transmits information between the various components of the device (e.g., the processor 1501, the memory 1502, the input / output interface 1503, and the communication interface 1504);

[0214] The processor 1501 , the memory 1502 , the input / output interface 1503 and the communication interface 1504 are connected to each other in communication within the device via the bus 1505 .

[0215] An embodiment of the present application also provides a storage medium, which is a storage medium that stores a computer program. When the computer program is executed by a processor, the multi-task semantic communication method is implemented.

[0216] As a non-transitory storage medium, the memory can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0217] The multi-task semantic communication method, system, device and storage medium proposed in the embodiment of the present application obtain the transmission signal corresponding to the communication task, and use the feature extraction unit to extract the semantic feature vector of the transmission signal, quantize the semantic feature vector to obtain a discrete semantic feature vector, modulate the discrete semantic feature vector to obtain a semantic feature signal, and transmit the semantic feature signal to the base station, so that the base station obtains the transmission signal corresponding to each task receiving end according to the semantic feature signals of different task sending ends, sends the transmission signal to multiple task receiving ends one by one, and the task receiving end performs channel equalization on the transmission signal to obtain an estimated signal, and inputs the estimated signal into a pre-trained semantic reasoning model for semantic reasoning to obtain target processing data corresponding to the communication task. The embodiment of the present application extracts semantic features of the transmission signal at different task sending ends, extracts the semantic information into discrete semantic feature vectors in a distinguishable semantic subspace, and then uses the same base station to forward the signal. During the transmission process, the semantic feature signals corresponding to different communication tasks are encoded in separate subspaces, and are approximately orthogonal to each other. Then, the task receiver of the corresponding communication task directly obtains the corresponding estimated signal through channel equalization and performs semantic reasoning on the estimated signal. Since the semantic features of multiple users are in a distinguishable feature domain, the interference of multiple users can be significantly reduced without the need to design a complex communication system, thereby reducing transmission costs and improving transmission accuracy.

[0218] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0219] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0220] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0221] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0222] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0223] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0224] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0225] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0226] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0227] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store programs.

[0228] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A multi-task semantic communication method, characterized in that: Applied to a task sending end, the task sending end corresponds to a task receiving end and is used to perform a communication task. The method includes: Acquire a transmission signal corresponding to the communication task, and extract a semantic feature vector of the transmission signal using a feature extraction unit; quantizing the semantic feature vector to obtain a discrete semantic feature vector, and modulating the discrete semantic feature vector to obtain a semantic feature signal; The semantic feature signal is transmitted to a base station, so that the base station obtains a transmission signal corresponding to each of the task receiving ends according to the semantic feature signals of different task sending ends, and sends the transmission signal to multiple task receiving ends one by one. The task receiving end performs channel equalization on the transmission signal to obtain an estimated signal, and inputs the estimated signal into a pre-trained semantic reasoning model for semantic reasoning to obtain target processing data corresponding to the communication task.

2. The multi-task semantic communication method according to claim 1, characterized in that: The step of quantizing the semantic feature vector to obtain a discrete semantic feature vector includes: Acquire a quantization codebook shared between the task sending end and the corresponding task receiving end, wherein the quantization codebook includes a preset number of quantization vectors; Dividing the semantic feature vector into at least one linear vector, wherein the dimension of the linear vector is consistent with the dimension of the quantization vector; The discrete semantic feature vector is obtained according to the inner product of the quantization vector and the linear vector.

3. The multi-task semantic communication method according to claim 2, characterized in that: The obtaining the discrete semantic feature vector according to the inner product of the quantization vector and the linear vector includes: Perform inner products of the linear vector and the quantization vector respectively to obtain inner product values, and select the index position of the quantization vector in the quantization codebook corresponding to the smallest inner product value; The discrete semantic feature vector is obtained based on the index position corresponding to each of the linear vectors.

4. The multi-task semantic communication method according to claim 1, characterized in that: The transmission signal includes an image to be classified or an image to be reconstructed; When the transmitted signal is an image to be classified, the semantic reasoning model is a neural network model, and the target processing data is a classification result of the image to be classified; When the transmitted signal is an image to be reconstructed, the feature extraction unit is an encoder unit, the semantic reasoning model is a decoder unit, and the target processing data is a reconstruction result corresponding to the image to be reconstructed.

5. The multi-task semantic communication method according to claim 4, characterized in that: When the feature extraction unit is an encoder unit, the step of extracting the semantic feature vector of the transmission signal by using the feature extraction unit includes: After performing a first downsampling and self-attention processing on the transmission signal using a first encoding block, a first feature map is obtained, wherein a width and a height of the first feature map are both half of the transmission signal; Using a second encoding block to perform a second downsampling and self-attention processing on the first feature map, thereby obtaining a second feature map, wherein the width and height of the second feature map are both half of the first feature map; The second feature map is downsampled for the third time and self-attention processed by a third encoding block, and then encoded to obtain the semantic feature vector.

6. A multi-task semantic communication method, characterized in that: Applied to the task receiving end, the method includes: Acquire a transmission signal from a base station, wherein the transmission signal is calculated by the base station according to the semantic feature signal obtained by the multi-task semantic communication method according to any one of claims 1 to 5; Performing channel equalization on the transmission signal to obtain an estimated signal; The estimated signal is input into a pre-trained semantic reasoning model for semantic reasoning to obtain target processing data corresponding to the communication task.

7. The multi-task semantic communication method according to claim 6, characterized in that: The performing channel equalization on the transmission signal to obtain an estimated signal comprises: Acquire channel state information between the task receiving end and the base station, and calculate conjugate transposed data of the channel state information; The transmission signal is divided by the conjugate transposed data to obtain the estimated signal.

8. The multi-task semantic communication method according to claim 6, characterized in that: When the semantic reasoning model is a neural network model, before inputting the estimation signal into a pre-trained semantic reasoning model for semantic reasoning, the method further includes: Acquire a training data set, wherein the training data set includes a transmission sample signal and a data processing label, wherein the transmission sample signal is obtained by the base station according to a semantic sample signal corresponding to each task sending end, and the semantic sample signal is obtained according to the transmission sample signal; Obtaining an estimated sample signal according to the transmitted sample signal, and using the estimated sample signal to train the semantic reasoning model to obtain a data prediction result; A first loss value is calculated according to the data prediction result and the data processing label, a second loss value is calculated according to the transmission sample signal and the total semantic sample signal, and a third loss value is calculated according to the transmission sample signal and the transmission sample signal; A total loss value is obtained according to the first loss value, the second loss value and the third loss value, and the weight of the semantic reasoning model is adjusted based on the total loss value to obtain the trained semantic reasoning model.

9. The multi-task semantic communication method according to claim 6, characterized in that: When the semantic reasoning model is a decoder unit, before inputting the estimation signal into a pre-trained semantic reasoning model for semantic reasoning, the method further includes: Acquire a training data set, wherein the training data set includes a transmission sample signal and a data processing label, wherein the transmission sample signal is obtained by the base station according to a semantic sample signal corresponding to each task sending end, and the semantic sample signal is obtained according to the transmission sample signal; Obtaining an estimated sample signal according to the transmitted sample signal, and using the estimated sample signal to train the semantic reasoning model to obtain a data prediction result; Calculate a total loss value according to the data processing label and the data prediction result; The weight of the semantic reasoning model is adjusted based on the total loss value to obtain the trained semantic reasoning model.

10. A multi-task semantic communication system, characterized in that: include: A plurality of task sending ends, wherein the task sending ends are used to obtain semantic feature signals according to the multi-task semantic communication method according to any one of claims 1 to 5, and send the semantic feature signals to a base station respectively, and each of the task sending ends corresponds to a task receiving end; A base station, the base station is used to obtain the semantic feature vectors from a plurality of different task sending ends to obtain a transmission signal corresponding to each task receiving end, and send the transmission signal to the plurality of task receiving ends one by one; A plurality of the task receiving ends are used to obtain a transmission signal, and utilize the multi-task semantic communication method described in any one of claims 6 to 9 to obtain target processing data corresponding to the communication task according to the transmission signal.

11. The multi-task semantic communication system according to claim 10, characterized in that: The step of the base station acquiring the semantic feature vectors from a plurality of different task sending ends to obtain a transmission signal corresponding to each task receiving end comprises: Selecting target sending ends from the task sending ends one by one, and acquiring the task receiving ends corresponding to the target sending ends as target receiving ends; Obtaining target transmission data according to the transmission parameters between the target transmitting end and the base station and the semantic feature vector of the target transmitting end; Obtain interference transmission data according to the transmission parameters between the other task transmitting ends and the base station and the semantic feature vector of the task transmitting end, and accumulate the interference transmission data to obtain total interference data; The transmission signal of the target receiving end is obtained according to the target transmission data and the total interference data, and the transmission signal of each task receiving end is obtained one by one.

12. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the multi-task semantic communication method according to any one of claims 1 to 9 when executing the computer program.

13. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the multi-task semantic communication method according to any one of claims 1 to 9 is implemented.

Citation Information

Cited By

  • Edge multi-task semantic communication method and device, computer equipment and storage medium

    CN119740099A

  • Edge multi-task semantic communication methods, devices, computer equipment and storage media

    CN119740099B

  • Multi-task semantic communication method and system, and device and storage medium

    WO2026001063A1