Semantic model training method and device, semantic communication method and device and federal learning system

Abstract

The invention provides a semantic model training method and device based on federal learning, and relates to the technical field of artificial intelligence, in particular to the technical fields of image processing, natural language processing, deep learning and the like. According to the specific implementation scheme, the method comprises the steps of obtaining local semantic features sent by multiple terminal devices and task execution information of a semantic model; based on the task execution information, detecting whether a network training completion condition is satisfied; in response to detection that the network training completion condition is not met, performing aggregation processing on the local semantic features to obtain global semantic features; and sending the global semantic feature to the terminal device in the plurality of terminal devices, continuing to obtain the local semantic feature and task execution information of the terminal device, detecting whether a network training completion condition is met or not, and sending the global semantic feature until the network training completion condition is detected to be met.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to the fields of image processing, natural language processing, deep learning, etc., and particularly to a semantic model training method and apparatus based on federated learning, a semantic communication method and apparatus, a federated learning system, and a computer-readable storage medium. Background Technology

[0002] Future 6G communication technology will integrate artificial intelligence and communication technologies, transforming from serving people and things to supporting the "Internet of Everything." Semantic communication, as a new technology that can extract user needs and semantic information and integrate them into the communication process, is becoming one of the potential key technologies of 6G. A semantic model refers to an artificial intelligence model obtained through joint training between semantic transceivers to achieve specific semantic functions during semantic communication, and characterized by model parameters.

[0003] With the rapid growth in demand for intelligent wireless communication services, more and more smart devices are deploying semantic communication systems that include semantic models to perform intelligent tasks. However, existing semantic communication systems cannot well adapt to actual distributed semantic communication scenarios under conditions such as privacy sensitivity, limited communication resources, and heterogeneous semantic models. Summary of the Invention

[0004] This disclosure provides a semantic model training method and apparatus based on federated learning, a semantic communication method and apparatus, a federated learning system, and a computer-readable storage medium.

[0005] According to the first aspect, a semantic model training method based on federated learning is provided. The method includes: acquiring local semantic features and task execution information of the semantic model sent by multiple terminal devices; detecting whether the network training completion condition is met based on the task execution information; in response to detecting that the network training completion condition is not met, aggregating the local semantic features to obtain global semantic features; sending the global semantic features to a terminal device among the multiple terminal devices, and continuing to acquire the local semantic features and task execution information of the terminal device, detect whether the network training completion condition is met, and send the global semantic features until the network training completion condition is met.

[0006] According to the second aspect, another method for training semantic models based on federated learning is provided. This method includes: acquiring a local dataset and a pre-trained semantic model; obtaining local semantic features and task execution information based on the local dataset and the semantic model, and sending the local semantic features and task execution information to a server; receiving global semantic features sent by the server; updating the semantic model based on the local dataset and the global semantic features to obtain an updated semantic model; obtaining new local semantic features and new task execution information based on the local dataset and the updated semantic model, replacing the local semantic features with the new local semantic features, replacing the task execution information with the new task execution information, and continuing to send the local semantic features and task execution information to the server.

[0007] According to the third aspect, a semantic communication method is provided, the method comprising: acquiring task data to be processed; inputting the task data to be processed into a semantic model to obtain a prediction result of the task data to be processed, wherein the semantic model is an updated semantic model trained using a federated learning-based semantic model training method as described in any implementation of the second aspect.

[0008] According to the fourth aspect, a semantic model training device based on federated learning is provided. The device includes: an information acquisition unit configured to acquire local semantic features and task execution information of the semantic model sent by multiple terminal devices; a detection unit configured to detect whether the network training completion condition is met based on the task execution information; an aggregation unit configured to aggregate the local semantic features to obtain global semantic features in response to the detection that the network training completion condition is not met; and a sending unit configured to send the global semantic features to the terminal devices among the multiple terminal devices, and continue to control the information acquisition unit, detection unit, aggregation unit, and sending unit to work until the network training completion condition is met.

[0009] According to the fifth aspect, another semantic model training device based on federated learning is provided, the device comprising: a data acquisition unit configured to acquire a local dataset and a pre-trained semantic model; a first obtaining unit configured to obtain local semantic features and task execution information based on the local dataset and the semantic model, and send the local semantic features and task execution information to a server; a receiving unit configured to receive global semantic features sent by the server; an updating unit configured to update the semantic model based on the local dataset and the global semantic features, and obtain an updated semantic model; and a second obtaining unit configured to obtain new local semantic features and updated task execution information based on the local dataset and the updated semantic model, replace the local semantic features with the new local semantic features, replace the task execution information with the new task execution information, and continue to send the local semantic features and task execution information to the server.

[0010] According to the sixth aspect, a semantic communication apparatus is also provided, the apparatus comprising: a text acquisition unit configured to acquire a comment text to be processed; and an input unit configured to input the comment text to be processed into a semantic model to obtain an evaluation result of the comment text to be processed, wherein the semantic model is an updated semantic model obtained by employing a federated learning-based semantic model training apparatus as described in any implementation of the fifth aspect.

[0011] According to the sixth aspect, a federated learning system is provided, comprising: a server and multiple terminal devices; each terminal device acquires a local dataset and a pre-trained semantic model; based on the local dataset and the semantic model, it obtains local semantic features and task execution information, and sends the local semantic features and task execution information to the server; the server, based on the task execution information, detects whether the network training completion condition is met; in response to detecting that the network training completion condition is not met, it aggregates the local semantic features to obtain global semantic features; it sends the global semantic features to a terminal device among the multiple terminal devices; the terminal device receives the global semantic features sent by the server; based on the local dataset and the global semantic features, it updates the semantic model to obtain an updated semantic model; based on the local dataset and the updated semantic model, it obtains new local semantic features and new task execution information, replaces the local semantic features with the new local semantic features, replaces the task execution information with the new task execution information, and continues to send the local semantic features and task execution information to the server; the server continues to acquire the local semantic features and task execution information of the terminal device, detect whether the network training completion condition is met, and send global semantic features until the network training completion condition is met.

[0012] According to a seventh aspect, a non-transitory computer-readable storage medium is provided that stores computer instructions for causing a computer to perform the method described in any implementation of the first to third aspects.

[0013] The embodiments of this disclosure provide a federated learning-based semantic model training method and apparatus. First, it acquires local semantic features and semantic model task execution information sent by multiple terminal devices. Second, based on the task execution information, it detects whether the network training completion condition is met. Third, in response to detecting that the network training completion condition is not met, it aggregates the local semantic features to obtain global semantic features. Finally, it sends the global semantic features to a terminal device among the multiple terminal devices, and continues to acquire the local semantic features and task execution information of that terminal device, detect whether the network training completion condition is met, and send the global semantic features until the network training completion condition is met. Thus, by aggregating global semantic features and broadcasting them to multiple terminal devices, collaborative training of a distributed semantic communication system is achieved. By exchanging global semantic features with each terminal device, heterogeneous semantic models can be deployed on each terminal device, supporting flexible application under different hardware conditions and task requirements, enhancing the system's generalization ability and practicality. Through collaborative training of global semantic features, the amount of data required for transmission is reduced, decreasing the communication burden of the wireless channel, lowering data transmission latency and energy consumption, making it suitable for deployment in resource-constrained environments with high real-time requirements.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0016] Figure 1 This is a flowchart of an embodiment of a semantic model training method based on federated learning according to the present disclosure;

[0017] Figure 2 This is a flowchart of another embodiment of the semantic model training method based on federated learning according to this disclosure;

[0018] Figure 3 This is a schematic diagram of the training process of the publicly disclosed semantic model;

[0019] Figure 4 This is a flowchart of an embodiment of the semantic communication method according to the present disclosure;

[0020] Figure 5 This is a schematic diagram of the structure of an embodiment of a semantic model training apparatus based on federated learning according to the present disclosure;

[0021] Figure 6 This is a schematic diagram of another embodiment of a semantic model training apparatus based on federated learning according to the present disclosure;

[0022] Figure 7 This is a schematic diagram of a structure of an embodiment of a semantic communication device according to the present disclosure;

[0023] Figure 8 This is a schematic diagram of a structure according to an embodiment of the federated learning system disclosed herein;

[0024] Figure 9 This is a block diagram of an electronic device used to implement the federated learning-based semantic model training method or semantic communication method of the embodiments of this disclosure. Detailed Implementation

[0025] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0026] The first implementation of traditional federated learning involves a centralized training approach. Each terminal device uploads its local data to a central server, which then trains a semantic model using all the data and distributes the trained model to each user. A second implementation based on model parameter aggregation is proposed. Each device trains its local semantic model using its local data, updates its local semantic model parameters, and uploads these parameters to the central server. The central server aggregates the uploaded semantic model parameters to update the global semantic model and broadcasts the updated model to all devices. The terminal devices and the central server iterate through this process until the global semantic model converges.

[0027] The first implementation requires terminal devices to upload local data to a central server, which often results in latency that cannot meet the demands of real-time services. Furthermore, due to privacy and security concerns, data silos often exist between different devices, limiting or hindering their ability to share local data. The second implementation requires frequent exchange of semantic model parameters between devices and the server, leading to expensive communication overhead. This technology also requires all devices to have the same semantic model structure and dimensions; however, due to differences in hardware and intelligent tasks, each device typically needs to be equipped with a semantic model with different structures and dimensions, often failing to meet the homogeneity requirement.

[0028] Based on this, this disclosure proposes a semantic model training method based on federated learning, which can be executed by a server. Figure 1The flowchart 100 of an embodiment of a semantic model training method based on federated learning according to the present disclosure is shown. The above-described semantic model training method based on federated learning includes the following steps:

[0029] Step 101: Obtain task execution information of local semantic features and semantic models sent by multiple terminal devices.

[0030] In this embodiment, the local semantic features are the features obtained by the semantic model of the terminal device after extracting semantic information from the local dataset. The local semantic features are the average output of the sending module of the semantic model for all local data in the local dataset, which represents the distribution of the local dataset of the terminal device at the semantic level.

[0031] In this embodiment, the semantic model refers to an artificial intelligence model that is jointly trained by the semantic transceiver to achieve specific task functions during semantic communication and is characterized by model parameters. The semantic model may include a sending module and a receiving module. The sending module includes a semantic encoder and a channel encoder, and the local semantic features are information obtained after being encoded by the semantic encoder and channel encoder in the semantic model. The receiving module includes a semantic decoder and a channel decoder.

[0032] In this embodiment, the semantic model can perform different tasks for different task data. For example, for text data, the semantic model can perform tasks such as machine translation and text classification; for image data, the semantic model can perform tasks such as image classification and semantic segmentation. It should be noted that for task data of different modalities, the semantic model only needs to employ different semantic neural network architectures, including CNN, LSTM, and Transformer, while the federated learning-based semantic model training methods involved in each semantic model are universal.

[0033] In this embodiment, task execution information refers to the training and execution results of the semantic model for a specific task. The specific task can be related to the function implemented by the semantic model. For example, for a semantic model oriented towards image classification, the specific task is image classification; for a semantic model oriented towards machine translation, the specific task is machine translation.

[0034] In this embodiment, the task execution information may include: the accuracy value of the semantic model of each terminal device in executing the task and the number of training iterations of the semantic model of each terminal device.

[0035] Step 102: Based on the task execution information, check whether the network training completion conditions are met.

[0036] In this embodiment, the execution entity and multiple terminal devices on which the semantic model training method based on federated learning is run are considered as a network, namely a federated learning network. The network training completion condition refers to the training conditions that meet the above network objectives.

[0037] In this embodiment, based on the task execution information of the semantic models of each terminal device, it can be detected whether the semantic models of each terminal device meet the network training completion conditions. In response to the fact that the semantic models of at least one terminal device do not meet the network training completion conditions, it is determined that the semantic models of at least one terminal device do not meet the network training completion conditions.

[0038] In this embodiment, when the semantic model of the terminal device does not meet the network training completion conditions, it is necessary to update the parameters of the semantic model of the terminal device through global semantic features so that the semantic model of the terminal device meets the network training completion conditions.

[0039] Step 103: In response to the detection that the network training completion condition is not met, the local semantic features are aggregated to obtain the global semantic features.

[0040] In this embodiment, the global semantic feature is a global representation of the local semantic features of all terminal devices. It characterizes the global distribution of the overall dataset composed of local data from multiple terminal devices at the semantic level. Aggregation processing refers to performing a weighted average of the semantic features element by element, that is, generating a global semantic feature from the local semantic features of multiple terminal devices.

[0041] In this embodiment, when it is detected that the semantic model of at least one terminal device does not meet the network training completion conditions, the local semantic features of the at least one terminal device can be directly aggregated, or all local semantic features of multiple terminal devices can be aggregated to obtain global semantic features.

[0042] Optionally, during the training process, the executing entity may select some terminal devices from multiple terminal devices to participate in the aggregation process based on a selection strategy. The selection strategy may include, but is not limited to, fixed scheduling ratio, dynamically adjustable scheduling ratio, random scheduling, and round-robin scheduling.

[0043] Step 104: Send global semantic features to the terminal devices among multiple terminal devices, and continue to acquire the local semantic features and task execution information of the terminal device, detect whether the network training completion conditions are met, and send global semantic features until the network training completion conditions are met.

[0044] In this embodiment, the executing entity can send global semantic features to all terminal devices among multiple terminal devices, or send global semantic features to terminal devices that do not meet the conditions for completing network training.

[0045] The federated learning-based semantic model training method disclosed herein first acquires local semantic features and semantic model task execution information sent by multiple terminal devices; second, based on the task execution information, it detects whether the network training completion condition is met; third, in response to the detection that the network training completion condition is not met, it aggregates the local semantic features to obtain global semantic features; finally, it sends the global semantic features to the terminal devices among the multiple terminal devices, and continues to acquire the local semantic features and task execution information of the terminal device, detect whether the network training completion condition is met, and send the global semantic features until the network training completion condition is met. Thus, by aggregating global semantic features and broadcasting them to multiple terminal devices, collaborative training of a distributed semantic communication system is achieved; by exchanging global semantic features with each terminal device, heterogeneous semantic models can be deployed on each terminal device, supporting flexible application under different hardware conditions and task requirements, enhancing the system's generalization ability and practicality; through collaborative training of global semantic features, the amount of data required for transmission is reduced, decreasing the communication burden of the wireless channel, reducing data transmission latency and energy consumption, making it suitable for deployment in resource-constrained environments with high real-time requirements.

[0046] In some optional implementations of this disclosure, in response to detecting that the network training completion condition is not met, the local semantic features are aggregated to obtain global semantic features, including: in response to detecting that the network training completion condition is not met, the local semantic features of multiple terminal devices are weighted and averaged to obtain global semantic features.

[0047] In this optional implementation, corresponding weights can be pre-assigned to multiple terminal devices, and the weights of each terminal device can be multiplied and added together with their local semantic features to obtain a weighted average of the local semantic features of multiple terminal devices. This weighted average is the global semantic feature.

[0048] The optional implementation provides a method for obtaining global semantic features by weighted averaging of the local semantic features of multiple terminals, thus providing a reliable way to obtain global semantic features.

[0049] Optionally, in response to detecting that the network training completion condition is not met, the above-mentioned aggregation processing of local semantic features to obtain global semantic features includes: determining the weight of each terminal device based on the proportion of the local dataset of each terminal device to the total local dataset of all terminal devices, multiplying the weight of each terminal device with its local semantic features and adding them together to obtain the weighted average of the local semantic features of the multiple terminal devices, which is the global semantic feature.

[0050] Optionally, in response to detecting that the network training completion condition is not met, the above-mentioned aggregation processing of local semantic features to obtain global semantic features includes: determining the weight of each terminal device based on the importance of each terminal device in the semantic communication system, multiplying and adding the weight of each terminal device with its local semantic features to obtain a weighted average of the local semantic features of the multiple terminal devices, which is the global semantic feature.

[0051] In some optional implementations of this disclosure, the above-mentioned response to detecting that the network training completion condition is not met, and the weighted average of the local semantic features of multiple terminal devices to obtain global semantic features, includes: determining the number of local data for each terminal device among the multiple terminal devices; determining the weight of the local semantic features of each terminal device based on the number of local data for each terminal device; and performing a weighted average of the local semantic features of multiple terminal devices based on the weights to obtain global semantic features.

[0052] In this optional implementation, the aforementioned weighted average of the local semantic features of multiple terminal devices refers to: multiplying the local semantic features of each terminal device in the multiple terminal devices with its corresponding weight to obtain a weighted feature product, and then summing the weighted feature products of all terminal devices in the multiple terminal devices to obtain the global semantic features.

[0053] The optional implementation provides a method for obtaining global semantic features. Based on the amount of local data of each terminal device, the weight of each terminal device is determined. Based on the weight and the local semantic features of each terminal device, the global semantic features are obtained, providing another reliable implementation method for obtaining global semantic features.

[0054] In some optional implementations of this disclosure, detecting whether the network training completion condition is met based on task execution information includes: determining the accuracy of the semantic models in multiple terminal devices based on task execution information; and determining that the network training completion condition is met in response to detecting that the accuracy of all semantic models in multiple terminal devices reaches a preset accuracy threshold.

[0055] In this optional implementation, the task execution information is the information of the semantic model of the terminal device executing the task. For example, the task execution information includes: the total task value, where the number of failed tasks is a first number. The difference is obtained by subtracting the first number from the total task value, and the accuracy is obtained by dividing the difference by the total task value.

[0056] In this optional implementation, the accuracy threshold can be determined based on training requirements; for example, the accuracy threshold could be 85%. It should be noted that the network training completion condition can be either the condition of detecting the semantic models of all terminal devices across multiple terminal devices, or the condition of the sum of the accuracies of detecting the semantic models of multiple terminal devices after detecting the semantic models of each terminal device.

[0057] This optional implementation provides a method for detecting whether the network training completion condition is met. By using the accuracy of the semantic model, it detects whether the network training completion condition is met, thus providing a reliable implementation method for the network training completion condition.

[0058] In some optional implementations of this disclosure, sending global semantic features to a terminal device among multiple terminal devices includes: selecting a terminal from multiple terminal devices based on a predicted scheduling strategy; and sending the global semantic features to that terminal device.

[0059] In this optional implementation, the scheduling strategy can include fixed scheduling ratio, dynamically adjustable scheduling ratio, random scheduling, and round-robin scheduling. A fixed scheduling ratio refers to a pre-allocated proportion for each terminal device among multiple terminal devices; based on this proportion, the number of times each terminal device is invoked can be determined. A dynamically adjustable scheduling ratio refers to dynamically adjusting the proportion allocated to each terminal device; this proportion can be dynamically set based on different needs. Random scheduling refers to randomly selecting a terminal device from multiple terminal devices. Round-robin scheduling refers to sequentially invokeing each terminal device among multiple terminal devices according to a certain polling period.

[0060] The optional implementation provides a method for sending global semantic features, which uses different scheduling strategies to retrieve terminal devices and send the global semantic features to the retrieved terminal devices, thereby improving the flexibility of global semantic feature distribution.

[0061] This disclosure also proposes another method for training semantic models based on federated learning, which can be executed by a terminal device communicating with a server. Figure 2 The flowchart 200 illustrates another embodiment of the semantic model training method based on federated learning according to this disclosure, which includes the following steps:

[0062] Step 201: Obtain the local dataset and the pre-trained semantic model.

[0063] In this embodiment, step 201 includes: detecting whether a semantic model exists locally; in response to detecting that no semantic model exists locally, acquiring a local dataset and a semantic neural network; training the semantic neural network based on the local dataset; and obtaining a semantic model in response to the semantic neural network meeting the training completion conditions.

[0064] In this embodiment, the semantic neural network is a deep neural network for constructing a semantic model. The semantic neural network includes a sending module and a receiving module connected to the sending module through a physical channel. The semantic model can be obtained by iteratively adjusting the semantic neural network multiple times.

[0065] In this embodiment, the execution entity running on the federated learning-based semantic model training method selects a semantic model suitable for its local environment based on its own hardware performance and intelligent task requirements, and sets the initial parameters of the semantic model.

[0066] In this embodiment, the local dataset includes at least one local data, which is data related to training the semantic model or performing tasks related to the semantic model. For example, if the semantic model is an image classification task-oriented semantic model, then the local dataset consists of images labeled with image types.

[0067] In this embodiment, the local dataset can be data stored locally or collected in real time, and the local dataset can be data sent in advance by the server. It should be noted that the server sends the local dataset to each of the multiple terminal devices, and the local datasets between any two terminal devices are different.

[0068] The collection, storage, use, processing, transmission, provision, and disclosure of local datasets and semantic models involved in the technical solutions disclosed herein are performed after authorization and comply with relevant laws and regulations.

[0069] Step 202: Based on the local dataset and semantic model, obtain local semantic features and task execution information, and send the local semantic features and task execution information to the server.

[0070] Step 202 above includes: inputting local data selected from the local dataset into the semantic model to obtain the local semantic features output by the sending module of the semantic model, and obtaining the prediction results output by the receiving module of the semantic model; based on the prediction results and the labels in the selected local data, determining the accuracy of the prediction results, and using the accuracy as task execution information.

[0071] In this embodiment, the server can be a central server, an edge server, or other device with strong computing power that can communicate with terminal devices.

[0072] Step 203: Receive the global semantic features sent by the server.

[0073] In this embodiment, the global semantic feature is a global representation of the local semantic features of all terminal devices, which represents the global distribution of the overall dataset composed of local data of each terminal device in multiple terminal devices at the semantic level.

[0074] Step 204: Update the semantic model based on the local dataset and global semantic features to obtain the updated semantic model.

[0075] In this embodiment, the global semantic features sent by the receiving server are compared with the local semantic features to determine the degree of difference between them. Based on the degree of difference, the local task execution status, and the local dataset, the semantic model is iterated and trained a predetermined number of times to update the semantic model. The predetermined number of times can be set based on the update requirements of the semantic model; for example, the predetermined number of times is 5.

[0076] In this embodiment, the semantic model is a model for predicting a predetermined task. Inputting local data from the local dataset into the semantic model yields the prediction results output by the semantic model. These prediction results are the task analysis results obtained after analyzing the predetermined task. The executing entity can perform the training steps A to B to complete one iteration of semantic model training. The method and quantity of selecting local data from the local dataset are not limited in this application.

[0077] Step A. Calculate the semantic loss of the semantic model based on the selected local data input to the semantic model.

[0078] In this embodiment, during each iteration of training of the semantic model, local data is selected from the local dataset and input into the semantic model. Based on the local semantic features and global semantic features output by the semantic model's sending module, the semantic loss of the semantic model is calculated.

[0079] In this embodiment, the loss function of the semantic model used to calculate the loss value of the semantic model can be the mean squared error function. During the iterative training of the semantic model, the gradient descent algorithm can be used to minimize the loss value of the semantic model, thereby iteratively optimizing the network parameters of the semantic model.

[0080] The gradient is essentially a vector representing the directional derivative of a loss function at a given point, where the loss function reaches its maximum value along that direction. In other words, the loss function changes most rapidly and at its highest rate of change along that direction at that point. In deep learning, the main task of a neural network is to find the optimal network parameters (weights and biases) during learning; these optimal parameters are those that minimize the loss function.

[0081] Step B. Based on the loss value of the semantic model, train the semantic model to obtain the updated semantic model.

[0082] In this embodiment, the updated semantic model can be a trained semantic model obtained by inputting selected local data into the semantic model for iterative training and then tuning the parameters of the semantic model; or the updated semantic model can be a semantic model obtained by training through a predetermined number of iterations.

[0083] In this embodiment, the loss value of the semantic model can be used to detect whether the semantic model meets the training completion condition. After the semantic model meets the training completion condition, the updated semantic model is obtained.

[0084] In this embodiment, the training completion condition includes: the loss value of the semantic model is less than a first loss threshold. The first loss threshold can be determined based on specific training requirements; for example, the first loss threshold is 0.01.

[0085] In this embodiment, when the semantic model does not meet the training completion conditions, the relevant parameters in the semantic model are adjusted to make the loss value of the semantic model converge. Based on the adjusted semantic model, the above training steps A-B are continued.

[0086] In this embodiment, adjusting the relevant parameters of the semantic model when the semantic model does not meet the training completion conditions helps the loss value of the semantic model converge.

[0087] In this embodiment, the above-mentioned training of the semantic model based on the loss value of the semantic model to obtain the trained semantic model includes: in response to the loss value of the semantic model being less than the loss threshold, determining that the semantic model meets the training completion condition, and obtaining the updated semantic model.

[0088] Optionally, when there is no difference between global semantic features and local semantic features, there is no need to update the semantic model; the local semantic features and task execution information can be directly sent to the server.

[0089] Optionally, when the semantic model includes a sending module and a receiving module, the above-mentioned updating of the semantic model includes: controlling the sending module of the semantic model to operate at a first learning rate η in a certain round. TGuided by a weighted sum of the task loss function and the semantic loss function, gradient descent is performed a specified number of times, controlling the receiving module of the semantic model to operate at a second learning rate η in a certain round. R Perform gradient descent a specified number of times, guided by the task loss function. The first and second learning rates are different.

[0090] Optionally, the above-mentioned updating semantic model includes: controlling the sending module and receiving module of the semantic model to perform gradient descent with a specified learning rate; the sending end and receiving end of the same terminal can maintain the same or different learning rates during the same round of updates; and the sending and receiving ends of the same terminal device can maintain the same or different learning rates during different rounds of updates.

[0091] Optionally, the above-mentioned updated semantic model can also include: controlling the semantic model to perform a fixed number of gradient descent operations, allowing different numbers of gradient descent operations to be performed between different terminals during the same round of updates, and allowing different numbers of gradient descent operations to be performed between the same terminal device during different rounds of updates.

[0092] Step 205: Based on the local dataset and the updated semantic model, obtain new local semantic features and new task execution information, replace the local semantic features with the new local semantic features, replace the task execution information with the new task execution information, and send the local semantic features and task execution information to the server.

[0093] In this embodiment, step 205 includes: selecting local data from the local dataset, inputting the selected local data into the updated semantic model, and obtaining the local semantic features output by the sending module of the updated semantic model as new local semantic features; and performing an accuracy test on the updated semantic model based on the local dataset to obtain new task execution information.

[0094] The federated learning-based semantic model training method disclosed herein first acquires a local dataset and a pre-trained semantic model; second, based on the local dataset and the semantic model, it obtains local semantic features and task execution information, and sends these to a server; third, it receives global semantic features from the server; fourth, based on the local dataset and the global semantic features, it updates the semantic model to obtain an updated semantic model; finally, based on the local dataset and the updated semantic model, it obtains new local semantic features and new task execution information, replaces the original local semantic features with the new ones, replaces the original task execution information with the new task execution information, and sends these to the server. Thus, by sending local semantic features to the server to obtain global semantic features, and updating the semantic model using the global semantic features and the local dataset to obtain an updated semantic model, the communication burden of the wireless channel can be reduced, data transmission latency and energy consumption can be lowered, making it suitable for deployment in resource-constrained environments with high real-time requirements, and improving the collaborative training of distributed semantic communication systems in practical, limited scenarios.

[0095] In some optional implementations of this disclosure, the semantic model includes: a sending module, which is used to extract features from the original information to obtain local semantic features; obtaining local semantic features and task execution information based on the local dataset and the semantic model includes: dividing the local dataset into a training dataset and a test dataset; selecting training data from the training dataset; inputting the training data into the sending module of the semantic model to obtain the local semantic features output by the sending module; and testing the semantic model based on the test dataset to obtain task execution information.

[0096] In this optional implementation, the local dataset can be divided into training and testing datasets according to a predetermined ratio. For example, the predetermined ratio is 7:3, meaning that local data accounts for 7 / 10 of the local data in the training dataset and 3 / 10 of the local data in the testing dataset, and the local data in the training dataset and the local data in the testing dataset are completely different. The method and quantity of selecting training data from the training set are not limited in this disclosure.

[0097] In this optional implementation, the local semantic feature is the average output of the semantic features of the semantic model sending module for all data samples in the local dataset, which represents the distribution of the device's local dataset at the semantic level.

[0098] In this optional implementation, the semantic model can be tested by sequentially inputting local data from the test dataset into it. After inputting the local data into the semantic model, task execution information can be calculated based on the prediction results of each local data point in the test dataset output by the semantic model. Specifically, the task execution information includes: the total number of times the test dataset was tested in the semantic model, the number of local data points in the test dataset that failed the test, or the number of local data points in the test dataset that succeeded in the test.

[0099] This optional implementation provides a reliable method for obtaining local semantic features and task execution information by dividing the local dataset into a training dataset and a test dataset. Based on the training dataset, local semantic features are obtained, and based on the test dataset, task execution information is obtained.

[0100] Optionally, the above-mentioned method of obtaining local semantic features and task execution information based on local datasets and semantic models further includes: training the semantic model based on training data to obtain a trained semantic model; inputting test data into the semantic model to obtain local semantic features output by the sending module; and testing the trained semantic model based on the test dataset to obtain task execution information.

[0101] In some embodiments of this disclosure, the semantic model is trained using the following training steps: acquiring a training sample set and a semantic neural network, the semantic neural network including: a sending module and a receiving module connected to the sending module via a physical channel; the sending module is used to extract features from the original information to obtain local semantic features; the receiving module is used to decode the local semantic features of the physical information transmission to obtain a prediction result; selecting training samples from the training sample set; inputting the training samples into the sending module of the semantic neural network to obtain the prediction result output by the receiving module of the semantic neural network; calculating the task loss based on the prediction result, the training samples, and the task loss function; and training the semantic neural network based on the task loss to obtain a pre-trained semantic model.

[0102] In this embodiment, the training sample set of the semantic neural network can be a local dataset or a dataset outside the local dataset. By training the semantic neural network, the pre-trained semantic model can perform a specific task. The specific task can be determined based on the requirements. For example, the specific task is an image classification task or a machine translation task.

[0103] In this embodiment, the task loss function for calculating the task loss value of the semantic model can be the cross-entropy loss function. The cross-entropy loss function measures the degree of difference between two different probability distributions in the same random variable, which in machine learning is represented as the difference between the true probability distribution and the predicted probability distribution in the prediction result. The smaller the value of the cross-entropy loss function, the better the prediction effect of the semantic model.

[0104] The semantic model training method provided in this embodiment trains the semantic neural network using a training sample set to obtain a pre-trained semantic model, providing a reliable way to obtain the semantic model.

[0105] In some optional implementations of this disclosure, the semantic model further includes: a receiving module connected to the transmitting module via a physical channel, the receiving module being used to decode the local semantic features of the physical information transmission to obtain a prediction result; updating the semantic model based on the local dataset and global semantic features to obtain the updated semantic model includes: selecting training data from the training dataset; inputting the training data into the semantic model to obtain the local semantic features output by the transmitting module of the semantic model and the prediction result output by the receiving module of the semantic model; calculating the semantic loss based on the local semantic features, global semantic features, and semantic loss function; calculating the task loss based on the local data, prediction result, and task loss function; and updating the parameters of the semantic model based on the semantic loss and task loss to obtain the updated semantic model.

[0106] In this optional implementation, the task loss function is a loss function that quantifies the difference between the probability distribution of the true local data value and the model's predicted value. The task loss function for calculating the semantic model's task loss value can be the cross-entropy loss function. For example... Figure 3 As shown, the task loss is obtained by calculating the cross-entropy loss function using the prediction results of the semantic model and the labels in the training samples.

[0107] In this optional implementation, the semantic loss function is a loss function that quantifies the difference between local semantic features and global semantic features. The semantic loss function used to calculate the semantic loss of the semantic model can be the mean squared error function, which is used to detect the deviation between the model's predicted values ​​and the true values. For example... Figure 3 As shown, the semantic loss is obtained by calculating the mean squared error function using the local semantic features and global semantic features output by the semantic model's sending module.

[0108] In this optional implementation, updating the parameters of the semantic model based on semantic loss and task loss to obtain the updated semantic model includes: adding the semantic loss and the task loss to obtain the total loss value; detecting whether the total loss value is less than or equal to the loss threshold; if it is greater than the loss threshold, adjusting the parameters of the semantic model, continuing to select training data from the training dataset, and inputting the training data into the semantic model to obtain the local semantic features output by the sending module of the semantic model and the prediction results output by the receiving module of the semantic model; calculating the semantic loss based on the local semantic features, global semantic features, and semantic loss function; calculating the task loss based on the local data, prediction results, and task loss function; detecting whether the total loss value is less than or equal to the loss threshold; and so on until the total loss value is detected to be less than or equal to the loss threshold to obtain the updated semantic model.

[0109] This optional implementation provides a reliable way to obtain the updated semantic model by calculating task loss and semantic loss.

[0110] In some optional implementations of this disclosure, obtaining new local semantic features and new task execution information based on the local dataset and the updated semantic model includes: inputting training data into the sending module of the updated semantic model to obtain new local semantic features output by the sending module; and testing the updated semantic model based on the test dataset to obtain new task execution information.

[0111] The method for obtaining new local semantic features and new task execution information provided in this embodiment inputs training data into the sending module of the updated semantic model to obtain new local semantic features output by the sending module; based on the test dataset, the updated semantic model is tested to obtain new task execution information, providing a reliable implementation method for obtaining new local semantic features and new task execution information.

[0112] Furthermore, based on the federated learning-based semantic model training method provided in the above embodiments, this disclosure also provides an embodiment of a semantic communication method, which combines artificial intelligence fields such as image processing and deep learning.

[0113] See Figure 4 The diagram illustrates a flow 400 of an embodiment of the semantic communication method according to this disclosure. The semantic communication method provided in this embodiment includes the following steps:

[0114] Step 401: Obtain the data of the task to be processed.

[0115] In this embodiment, the task data to be processed is source data for task processing. This source data can be image source data or text source data. The task data to be processed is input into a semantic model, which can provide a prediction result for the task. Specifically, when the semantic model is a model for implementing an image classification task, the task data to be processed is an image. The semantic model processes the image to obtain a prediction result for the image, which includes the classification type of the image. Specifically, when the semantic model is a model for implementing a text classification task, the task data to be processed is text. The semantic model processes the text to obtain a prediction result for the text, which includes the classification type of the text.

[0116] In this embodiment, the executing entity of the semantic communication method can obtain the task data to be processed in various ways. For example, the executing entity can obtain the task data to be processed stored in the database server through a wired or wireless connection. Alternatively, the executing entity can receive task data to be processed in real time from a terminal or other device.

[0117] Step 402: Input the task data to be processed into the semantic model to obtain the prediction results of the task data to be processed.

[0118] In this embodiment, the executing entity can input the task data to be processed obtained from step 401 into the semantic model to obtain the prediction result output by the semantic model.

[0119] In this embodiment, the semantic model can be adopted as described above. Figure 2 The updated semantic model trained using the method described in the embodiments can be found in the detailed training process. Figure 2 The relevant descriptions of the embodiments shown will not be repeated here.

[0120] The semantic communication method provided in the embodiments of this disclosure acquires task data to be processed, inputs the task data into a semantic model generated by a federated learning-based semantic model training method, and obtains a prediction result for the task data. Therefore, by using a semantic model to generate prediction results, the reliability and accuracy of the prediction results are improved.

[0121] Further reference Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a semantic model training device based on federated learning, which is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0122] like Figure 5As shown, the federated learning-based semantic model training device 500 provided in this embodiment includes: an information acquisition unit 501, a detection unit 502, an aggregation unit 503, and a sending unit 504. The information acquisition unit 501 can be configured to acquire local semantic features and semantic model task execution information sent by multiple terminal devices. The detection unit 502 can be configured to detect whether the network training completion condition is met based on the task execution information. The aggregation unit 503 can be configured to aggregate the local semantic features to obtain global semantic features in response to detecting that the network training completion condition is not met. The sending unit 504 can be configured to send the global semantic features to the terminal devices among the multiple terminal devices, and continue to control the information acquisition unit, detection unit, aggregation unit, and sending unit to work until the network training completion condition is detected.

[0123] In this embodiment, the specific processing of the information acquisition unit 501, detection unit 502, aggregation unit 503, and sending unit 504 in the federated learning-based semantic model training device 500, and the resulting technical effects, can be found in the following references: Figure 1 The relevant descriptions of steps 101, 102, 103, and 104 in the corresponding embodiments will not be repeated here.

[0124] In some optional implementations of this embodiment, the aggregation unit 503 is further configured as a retraining unit (not shown in the figure), which is configured to perform a weighted average of the local semantic features of multiple terminal devices to obtain global semantic features in response to detecting that the network training completion conditions are not met.

[0125] In some optional implementations of this embodiment, the aggregation unit 503 is further configured to: determine the number of local data for each of the multiple terminal devices; determine the weight of the local semantic features of each terminal device based on the number of local data for each terminal device; and perform a weighted average of the local semantic features of the multiple terminal devices based on the weights to obtain global semantic features.

[0126] In some optional implementations of this embodiment, the detection unit 502 is further configured to: determine the accuracy of the semantic models in multiple terminal devices based on task execution information; and determine that the network training completion condition is met in response to detecting that the accuracy of all semantic models in multiple terminal devices reaches a preset accuracy threshold.

[0127] In some optional implementations of this embodiment, the sending unit 504 is further configured to: select a terminal from multiple terminal devices based on a predicted scheduling strategy; and send global semantic features to the terminal device.

[0128] The federated learning-based semantic model training apparatus provided in the embodiments of this disclosure firstly acquires local semantic features and semantic model task execution information sent by multiple terminal devices; secondly, the detection unit 502 detects whether the network training completion condition is met based on the task execution information; thirdly, the aggregation unit 503 aggregates the local semantic features to obtain global semantic features in response to the detection that the network training completion condition is not met; finally, the sending unit 504 sends the global semantic features to the terminal devices among the multiple terminal devices, and continues to acquire the local semantic features and task execution information of the terminal device, detect whether the network training completion condition is met, and send the global semantic features until the network training completion condition is met. Therefore, by aggregating global semantic features and broadcasting them to multiple terminal devices, collaborative training of the distributed semantic communication system is achieved. By exchanging global semantic features with various terminal devices, heterogeneous semantic models can be deployed on each terminal device, supporting flexible application under different hardware conditions and task requirements, and enhancing the system's generalization ability and practicality. Through collaborative training of global semantic features, less data needs to be transmitted, reducing the communication burden of the wireless channel and lowering data transmission latency and energy consumption, making it suitable for deployment in resource-constrained environments with high real-time requirements.

[0129] Further reference Figure 6 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a semantic model training device based on federated learning, which is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0130] like Figure 6As shown, the semantic model training device 600 based on federated learning provided in this embodiment includes: a data acquisition unit 601, a first obtaining unit 602, a receiving unit 603, an updating unit 604, and a second obtaining unit 605. The data acquisition unit 601 can be configured to acquire a local dataset and a pre-trained semantic model. The first obtaining unit 602 can be configured to obtain local semantic features and task execution information based on the local dataset and the semantic model, and send the local semantic features and task execution information to a server. The receiving unit 603 can be configured to receive global semantic features sent by the server. The updating unit 604 can be configured to update the semantic model based on the local dataset and global semantic features to obtain an updated semantic model. The second obtaining unit 605 can be configured to obtain new local semantic features and updated task execution information based on the local dataset and the updated semantic model, replace the local semantic features with the new local semantic features, replace the task execution information with the new task execution information, and continue to send the local semantic features and task execution information to the server.

[0131] In this embodiment, the specific processing and technical effects of the data acquisition unit 601, the first obtaining unit 602, the receiving unit 603, the updating unit 604, and the second obtaining unit 605 in the federated learning-based semantic model training device 600 can be found in the following references. Figure 2 The relevant descriptions of steps 201, 202, 203, 204, and 205 in the corresponding embodiments will not be repeated here.

[0132] In some optional implementations of this embodiment, the semantic model includes: a sending module, which is used to extract features from the original information to obtain local semantic features; the first obtaining unit 602 is further configured to: divide the local dataset into a training dataset and a test dataset; select training data from the training dataset; input the training data into the sending module of the semantic model to obtain the local semantic features output by the sending module; and test the semantic model based on the test dataset to obtain task execution information.

[0133] In some optional implementations of this embodiment, the semantic model is trained using a training unit (not shown in the figure). The training unit is configured to: acquire a training sample set and a semantic neural network. The semantic neural network includes: a sending module and a receiving module connected to the sending module via a physical channel. The sending module is used to extract features from the original information to obtain local semantic features; the receiving module is used to decode the local semantic features transmitted by the physical information to obtain a prediction result; select training samples from the training sample set; input the training samples into the sending module of the semantic neural network to obtain the prediction result output by the receiving module of the semantic neural network; calculate the task loss based on the prediction result, the training samples, and the task loss function; and train the semantic neural network based on the task loss to obtain a pre-trained semantic model.

[0134] In some optional implementations of this embodiment, the semantic model further includes: a receiving module connected to the transmitting module via a physical channel, the receiving module being used to decode the local semantic features of the physical information transmission to obtain a prediction result; the updating unit 604 is further configured to: select training data from the training dataset; input the training data into the semantic model to obtain the local semantic features output by the transmitting module of the semantic model and the prediction result output by the receiving module of the semantic model; calculate the semantic loss based on the local semantic features, the global semantic features, and the semantic loss function; calculate the task loss based on the local data, the prediction result, and the task loss function; and update the parameters of the semantic model based on the semantic loss and the task loss to obtain the updated semantic model.

[0135] In some optional implementations of this embodiment, the second obtaining unit 605 is further configured to: input training data into the sending module of the updated semantic model to obtain new local semantic features output by the sending module; and test the updated semantic model based on the test dataset to obtain new task execution information.

[0136] The federated learning-based semantic model training apparatus provided in this disclosure comprises the following steps: First, a data acquisition unit 601 acquires a local dataset and a pre-trained semantic model. Second, a first obtaining unit 602 obtains local semantic features and task execution information based on the local dataset and the semantic model, and sends these to a server. Third, a receiving unit 603 receives global semantic features sent by the server. Fourth, an updating unit 604 updates the semantic model based on the local dataset and the global semantic features, obtaining an updated semantic model. Finally, a second obtaining unit 605 obtains new local semantic features and new task execution information based on the local dataset and the updated semantic model, replaces the original local semantic features with the new ones, replaces the original task execution information with the new task execution information, and sends these to the server. Thus, by sending local semantic features to the server to obtain global semantic features, and updating the semantic model using the global semantic features and the local dataset to obtain an updated semantic model, the communication burden of the wireless channel can be reduced, data transmission latency and energy consumption can be lowered, making it suitable for deployment in resource-constrained environments with high real-time requirements, and improving the collaborative training of distributed semantic communication systems in practical, constrained scenarios.

[0137] Further reference Figure 7 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a semantic communication device, which is similar to... Figure 4 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0138] like Figure 7 As shown, the semantic communication device 700 provided in this embodiment includes: a task acquisition unit 701 and an input unit 702. The task acquisition unit 701 can be configured to acquire task data to be processed. The input unit 702 can be configured to input the task data to be processed into a semantic model to obtain a prediction result for the task data. The semantic model is the updated semantic model obtained by the federated learning-based semantic model training device of the above embodiment.

[0139] In this embodiment, the specific processing of the task acquisition unit 701 and the input unit 702 in the semantic communication device 700 and the resulting technical effects can be referred to respectively. Figure 4 The relevant descriptions of steps 401 and 402 in the corresponding embodiments will not be repeated here.

[0140] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0141] Further reference Figure 8 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a federated learning system, which is similar to... Figure 1 , Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0142] like Figure 8 As shown, the federated learning system 800 provided in this embodiment includes: a server 801 and multiple terminal devices 802. Each terminal device 802 acquires a local dataset and a pre-trained semantic model; based on the local dataset and semantic model, it obtains local semantic features and task execution information, and sends the local semantic features and task execution information to server 801; server 801, based on the task execution information, detects whether the network training completion conditions are met; in response to detecting that the network training completion conditions are not met, it aggregates the local semantic features to obtain global semantic features Z; it sends the global semantic features Z to terminal device 802 among the multiple terminal devices; terminal device 802 receives the global semantic features Z sent by the server; based on the local dataset and global semantic features Z, it updates the semantic model to obtain the updated semantic model; based on the local dataset and the updated semantic model, it obtains new local semantic features and new task execution information, replaces the local semantic features with the new local semantic features, replaces the task execution information with the new task execution information, and continues to send the local semantic features and task execution information to server 801; server 801 continues to acquire the local semantic features and task execution information of the terminal device, detect whether the network training completion conditions are met, and send global semantic features until the network training completion conditions are met.

[0143] In this embodiment, the local semantic features of each terminal device are different, such as Figure 8 As shown, the local semantic features of the terminal devices include: z1, z2…zk, where k>1, and the value of k is determined by the number of terminal devices. The local dataset used by each terminal device when updating the semantic model can be a dataset sent by server 801. Different terminal devices have different local datasets, such as… Figure 8 The local dataset of the terminal device includes: D1, D2...Dk, where k>1, and the value of k is determined by the number of terminal devices.

[0144] In this embodiment, server 801 can be a central server, edge server, or other device with strong computing power and the ability to communicate with terminal devices.

[0145] Consider a distributed semantic communication scenario in which 100 terminal devices and 1 central server perform image classification pragmatic tasks on the MNIST and CIFAR-10 datasets, and the local datasets of each terminal device are non-independently identically distributed (non-IID), with no overlap between the local datasets of each terminal device.

[0146] Assume that each terminal device corresponds to a data distribution of at most m classes. First, the training data samples are classified according to labels. Then, each class of data samples is divided into (m×100) / 10 subclasses. Finally, the data samples are randomly assigned to each device in m subclasses. The parameter m describes the degree of non-IID in the local dataset distribution of the terminal device. The smaller m is, the greater the difference in data distribution between devices. In this embodiment, m=5 is set for the MNIST dataset and m=2 is set for the CIFAR-10 dataset.

[0147] For the MNIST dataset, a semantic model based on a Multi-Layer Perceptron (MLP) model was selected, while for the CIFAR-10 dataset, a semantic model based on a Convolutional Neural Network (CNN) model was selected. The semantic feature dimension was uniformly set to 10×64. The heterogeneity of the semantic models was characterized by the number and dimension of their hidden layers.

[0148] For semantic models based on MLP models, the transmitter uses multiple dense layers to extract semantic features, and then normalizes the extracted semantic features to ensure that the transmitted signal meets power constraints. The receiver uses multiple dense layers and a Softmax layer to implement a tag predictor. The received semantic features are processed by the tag predictor module to obtain the predicted tag.

[0149] For semantic models based on CNNs, the transmitter uses multiple convolutional layers to extract semantic features and pooling layers to further compress the semantic features. Similarly, the extracted semantic features are normalized. The receiver uses multiple dense layers and Softmax layers to implement a label predictor.

[0150] For image classification tasks, the task score of each terminal device is represented by classification accuracy. The global task score is defined as a weighted average of the task scores of all terminal devices. The weighting ratio for each device is determined by the proportion of its dataset samples to the total number of samples. The global task score is used as the performance metric.

[0151] For the MNIST dataset, the global task score of this publicly available federated learning system is 95.99 when the homogeneous semantic model is deployed on each terminal device and 95.98 when the heterogeneous semantic model is deployed on each terminal device.

[0152] For the CIFAR-10 dataset, the global task score of this publicly available federated learning system is 81.46 when the homogeneous semantic model is deployed on each terminal device, and 81.39 when the heterogeneous semantic model is deployed on each terminal device.

[0153] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0154] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0155] like Figure 9 As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded into random access memory (RAM) 903 from storage unit 908. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0156] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0157] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as federated learning-based semantic model training methods or semantic communication methods. For example, in some embodiments, the federated learning-based semantic model training methods or semantic communication methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the federated learning-based semantic model training methods or semantic communication methods described above can be performed. Alternatively, in other embodiments, computing unit 901 may be configured by any other suitable means (e.g., by means of firmware) to perform a federated learning-based semantic model training method or a semantic communication method.

[0158] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0159] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable federated learning-based semantic model training or semantic communication apparatus, such that when executed by the processor or controller, the program code enables the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0160] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0161] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0162] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0163] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0164] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0165] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A semantic model training method based on federated learning, the method comprising: obtaining local semantic features and task execution information of a semantic model sent by a plurality of terminal devices; based on the task execution information, detecting whether a network training completion condition is met; in response to detecting that the network training completion condition is not met, performing aggregation processing on the local semantic features to obtain global semantic features; sending the global semantic features to a terminal device in the plurality of terminal devices, and continuing to obtain local semantic features and task execution information of the terminal device, detecting whether the network training completion condition is met, and sending the global semantic features until it is detected that the network training completion condition is met.

2. The method of claim 1, wherein, The response to detecting that the network training completion condition is not met includes: in response to detecting that the network training completion condition is not met, performing weighted averaging on the local semantic features of the plurality of terminal devices to obtain the global semantic features.

3. The method of claim 2, wherein, The response to detecting that the network training completion condition is not met includes: determining the number of local data of each terminal device in the plurality of terminal devices; based on the number of local data of each terminal device, determining the weight of the local semantic features of each terminal device; based on the weight, performing weighted averaging on the local semantic features of the plurality of terminal devices to obtain the global semantic features.

4. The method of claim 1, wherein, The detection of whether the network training completion condition is met based on the task execution information includes: based on the task execution information, determining the accuracy of the semantic model in the plurality of terminal devices; in response to detecting that the accuracy of all semantic models of the plurality of terminal devices reaches a preset accuracy threshold, determining that the network training completion condition is met.

5. The method of claim 1, wherein, The sending of the global semantic features to a terminal device in the plurality of terminal devices includes: based on a predicted scheduling strategy, selecting a terminal from the plurality of terminal devices; sending the global semantic features to the terminal device. 6.A semantic model training method based on federated learning, the method comprising: obtaining a local data set and a pre-trained semantic model; based on the local data set and the semantic model, obtaining local semantic features and task execution information, and sending the local semantic features and the task execution information to a server; receiving global semantic features sent by the server; based on the local data set and the global semantic features, updating the semantic model to obtain an updated semantic model; based on the local data set and the updated semantic model, obtaining new local semantic features and new task execution information, using the new local semantic features instead of the local semantic features, using the new task execution information instead of the task execution information, and sending the local semantic features and the task execution information to the server.

7. The method of claim 6, wherein, The semantic model comprises a sending module for performing feature extraction on original information to obtain local semantic features; and the obtaining of local semantic features and task execution information based on the local data set and the semantic model comprises: dividing the local dataset into a training dataset and a test dataset; selecting training data from the training dataset; inputting the training data into a sending module of the semantic model to obtain local semantic features output by the sending module; testing the semantic model based on the test dataset to obtain task execution information.

8. The method of claim 6, wherein, The semantic model is trained by the following training steps: obtain a training sample set and a semantic neural network, the semantic neural network comprising: a sending module and a receiving module connected to the sending module through a physical channel, the sending module being configured to extract features from original information to obtain local semantic features, and the receiving module being configured to decode the local semantic features sent by the physical information to obtain a prediction result; selecting training samples from the training sample set; inputting the training samples into the sending module of the semantic neural network to obtain the prediction result output by the receiving module of the semantic neural network; calculating a task loss based on the prediction result, the training sample, and a task loss function; training the semantic neural network based on the task loss to obtain a pre-trained semantic model.

9. The method of claim 7, wherein, The semantic model further comprises: a receiving module connected to the sending module through a physical channel, the receiving module being configured to decode the local semantic features sent by the physical information to obtain a prediction result; and updating the semantic model based on the local dataset and the global semantic features to obtain an updated semantic model, comprising: selecting training data from the training dataset; inputting the training data into the semantic model to obtain local semantic features output by the sending module of the semantic model and prediction results output by the receiving module of the semantic model; calculating a semantic loss based on the local semantic features, the global semantic features, and a semantic loss function; calculating a task loss based on the local data, the prediction result, and a task loss function; updating parameters of the semantic model based on the semantic loss and the task loss to obtain an updated semantic model.

10. The method of claim 7, wherein, The new local semantic features and new task execution information obtained based on the local dataset and the updated semantic model comprise: inputting the training data into the sending module of the updated semantic model to obtain new local semantic features output by the sending module; testing the updated semantic model based on the test dataset to obtain new task execution information.

11. A semantic communication method, the method comprising: obtaining task data to be processed; inputting the task data to be processed into a semantic model to obtain a prediction result of the task data to be processed, the semantic model being an updated semantic model obtained by the semantic model training method based on federated learning of any one of claims 6-10.

12. A semantic model training device based on federated learning, the device comprising: an information acquisition unit configured to acquire local semantic features sent by a plurality of terminal devices and task execution information of a semantic model; a detection unit configured to detect whether a network training completion condition is met based on the task execution information; aggregating units configured to, in response to detecting that the network training completion condition is not satisfied, aggregate the local semantic features to obtain global semantic features; sending units configured to send the global semantic features to a terminal device in the plurality of terminal devices, and continue to control the information obtaining units, the detecting units, the aggregating units, and the sending units to work until detecting that the network training completion condition is satisfied.

13. A semantic model training apparatus based on federated learning, the apparatus comprising: data obtaining units configured to obtain a local data set and a pre-trained semantic model; first obtaining units configured to obtain, based on the local data set and the semantic model, local semantic features and task execution information, and send the local semantic features and the task execution information to a server; receiving units configured to receive global semantic features sent by the server; updating units configured to update, based on the local data set and the global semantic features, the semantic model to obtain an updated semantic model; second obtaining units configured to obtain, based on the local data set and the updated semantic model, new local semantic features and updated task execution information, use the new local semantic features to replace the local semantic features, use the new task execution information to replace the task execution information, and continue to send the local semantic features and the task execution information to the server.

14. A semantic communication apparatus, the apparatus comprising: task obtaining units configured to obtain to-be-processed task data; input units configured to input the to-be-processed task data into a semantic model to obtain a prediction result of the to-be-processed task data, the semantic model being an updated semantic model obtained by the semantic model training apparatus based on federated learning of claim 13.

15. A federated learning system, the system comprising: a server and a plurality of terminal devices; each terminal device obtains a local data set and a pre-trained semantic model; obtains, based on the local data set and the semantic model, local semantic features and task execution information, and sends the local semantic features and the task execution information to a server; the server detects, based on the task execution information, whether a network training completion condition is satisfied; in response to detecting that the network training completion condition is not satisfied, aggregates the local semantic features to obtain global semantic features; sends the global semantic features to a terminal device in the plurality of terminal devices; the terminal device receives global semantic features sent by the server; updates, based on the local data set and the global semantic features, the semantic model to obtain an updated semantic model; obtains, based on the local data set and the updated semantic model, new local semantic features and new task execution information, uses the new local semantic features to replace the local semantic features, uses the new task execution information to replace the task execution information, and continues to send the local semantic features and the task execution information to the server; The server continues to acquire the local semantic feature and the task execution information of the terminal device, detects whether the network training completion condition is met, and sends the global semantic feature until it is detected that the network training completion condition is met.

16. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are for causing the computer to perform the method of any one of claims 1-11.

Images