Semantic communication method and computer device

Abstract

This application proposes a semantic communication method and computer device. When a new modality category of a pragmatic task input source is obtained, a semantic model update request including the pragmatic task category and the new modality category can be sent. A first slice is received, which is a portion of the first semantic model corresponding to the task category and the new modality category. Using the first slice, a second semantic model can be directly obtained, so as to perform the above-mentioned pragmatic task on the input source of the new modality category through the second semantic model.

Description

Technical Field

[0001] This application mainly relates to the field of artificial intelligence applications, and more specifically to a semantic communication method and computer device. Background Technology

[0002] Semantic communication is a new architecture that integrates user needs and information meaning into the communication process. This architecture is expected to become the basic paradigm of the future Internet of Everything, fundamentally solving the problems of cross-system, cross-protocol, cross-network, and cross-human-machine incompatibility and interoperability in traditional data-based communication protocols, and truly realizing transparent intelligent communication for everything.

[0003] In practical applications of semantic communication, it is typically necessary to train corresponding semantic models for each multimodal category of a specific pragmatic task using its own dataset, and then deploy these models to nodes in the communication network. Only then can the nodes use these multiple semantic models to meet the semantic processing requirements of multimodal data. However, this undoubtedly increases the storage overhead and memory capacity requirements of the nodes, and also causes huge overhead in communication resources, making it impossible to widely adopt this technology. Summary of the Invention

[0004] To address the above problems, this application provides the following technical solution:

[0005] On the one hand, this application proposes a semantic communication method, the method comprising:

[0006] Obtain new modal categories from the input sources for pragmatic tasks;

[0007] Send a semantic model update request; the semantic model update request includes the task category of the pragmatic task and the new modality category;

[0008] Receive a first slice; the first slice is a partial first semantic model corresponding to the task category and the new modality category;

[0009] Using the first slice, a second semantic model is obtained, and the pragmatic task is performed on the input source through the second semantic model.

[0010] Optionally, before obtaining the new modality category of the pragmatic task input source, the method further includes:

[0011] Obtain the task category of any pragmatic task to be performed, and the modality category of the input source to be processed for that pragmatic task;

[0012] Send a semantic model retrieval request; the semantic model retrieval request includes the task category and the modality category;

[0013] Receive a third semantic model; the third semantic model is a trained semantic model corresponding to the task category and the modality category;

[0014] The third semantic model is executed to perform the pragmatic task on the input source to be processed through the third semantic model.

[0015] Optionally, obtaining the second semantic model using the first slice includes:

[0016] The first slice is used to replace the third slice in the third semantic model to obtain the second semantic model; the third semantic model refers to the semantic model that performs the pragmatic task on the input source of the original modality category.

[0017] Optionally, in the case of obtaining a new modality category of the pragmatic task input source, the method further includes:

[0018] Identify at least one stored candidate slice; the candidate slice is a partial semantic model trained for the pragmatic task;

[0019] Obtain the candidate modality category corresponding to the candidate slice; the candidate modality category is the modality category of the dataset used to train the semantic model to which the corresponding candidate slice belongs;

[0020] The new modality category is compared with the candidate modality category to obtain the corresponding comparison result;

[0021] If the comparison result is determined to be the same as any of the candidate modality categories, the candidate slice corresponding to the new modality category is determined as the first slice, and the step of obtaining the second semantic model using the first slice is executed;

[0022] If the comparison result indicates that the new modality category is different from all the candidate modality categories, or if it is determined that none of the candidate slices are stored, then the step of sending a semantic model update request is executed.

[0023] On the one hand, this application also proposes a semantic communication method, the method comprising:

[0024] Receive a semantic model update request; the semantic model update request includes a new modality category of the input information source for the semantic model to perform the pragmatic task, and the task category of the pragmatic task.

[0025] Obtain a first slice corresponding to the task category and the new modality category; the first slice is a partial first semantic model trained on the pragmatic task using a dataset belonging to the new modality category.

[0026] The first slice is sent so that the semantic model update requester can use the first slice to obtain a second semantic model capable of performing the pragmatic task on an input source with the new modality category.

[0027] Optionally, the method further includes:

[0028] Receive a semantic model acquisition request; the semantic model acquisition request includes the task category of the pragmatic task to be performed by the requested semantic model, and the modality category of the input information source to be processed for the pragmatic task;

[0029] Obtain a third semantic model corresponding to the task category and the modality category;

[0030] Send the third semantic model.

[0031] Optionally, the method further includes:

[0032] Receive multiple semantic models; the multiple semantic models are pragmatic tasks of the same task category and are trained on datasets of different modal categories;

[0033] Parameter difference analysis is performed on the multiple semantic models to obtain the slices for the same pragmatic function contained in each of the multiple semantic models.

[0034] The slices of each of the multiple semantic models are associated with the task category and the corresponding modality category and then stored.

[0035] Optionally, upon receiving any of the semantic model acquisition requests, the method further includes:

[0036] Send the cross-modal slices associated with the task category to enable the semantic model acquisition request end to identify the received cross-modal slices as candidate slices for the pragmatic task and store them.

[0037] The cross-modal slice refers to a slice contained in a semantic model other than the requested semantic model among the multiple semantic models associated with the task category.

[0038] Optionally, the method further includes:

[0039] Receive a semantic model training request; the semantic model training request includes multiple modal categories of different input sources for the pragmatic task to be performed by the requested semantic model, and the task category of the pragmatic task;

[0040] Obtain the datasets for each of the multiple modal categories labeled for the task category;

[0041] Based on the dataset, a semantic model corresponding to the task category and the corresponding modality category is trained;

[0042] Perform parameter difference analysis on multiple trained semantic models to obtain the slices for the same pragmatic function contained in each of the multiple semantic models.

[0043] Each slice of the multiple semantic models is associated with the task category and the corresponding modality category and then stored.

[0044] In another aspect, this application also proposes a computer device comprising a transceiver and a processor, wherein:

[0045] When the computer device is configured as any edge node in a communication network, the processor is configured to implement:

[0046] Obtain new modal categories from the input sources for pragmatic tasks;

[0047] Send a semantic model update request; the semantic model update request includes the task category of the pragmatic task and the new modality category;

[0048] Receive a first slice; the first slice is a partial first semantic model corresponding to the task category and the new modality category;

[0049] Using the first slice, a second semantic model is obtained, and the pragmatic task is performed on the input source through the second semantic model;

[0050] When the computer device is configured as the cloud of the communication network, the processor is used to implement:

[0051] Receive a semantic model update request; the semantic model update request includes a new modality category of the input information source for the semantic model to perform the pragmatic task, and the task category of the pragmatic task.

[0052] Obtain a first slice corresponding to the task category and the new modality category; the first slice is a partial first semantic model trained on the pragmatic task using a dataset belonging to the new modality category.

[0053] The first slice is sent so that the semantic model update requester can use the first slice to obtain a second semantic model capable of performing the pragmatic task on an input source with the new modality category. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0055] Figure 1 This is a flowchart illustrating an optional embodiment of the semantic communication method proposed in this application;

[0056] Figure 2 This is a schematic diagram illustrating the segmentation process of different semantic models that achieve the same pragmatic task in the semantic communication method proposed in this application.

[0057] Figure 3 This is a flowchart illustrating an optional embodiment two of the semantic communication method proposed in this application;

[0058] Figure 4 This is a flowchart illustrating an optional embodiment three of the semantic communication method proposed in this application;

[0059] Figure 5 This is a flowchart illustrating an optional embodiment four of the semantic communication method proposed in this application;

[0060] Figure 6 This is a flowchart illustrating an optional embodiment five of the semantic communication method proposed in this application;

[0061] Figure 7 This is a flowchart illustrating an optional embodiment six of the semantic communication method proposed in this application;

[0062] Figure 8 A schematic diagram of the hardware structure of an optional example of a computer device suitable for the semantic communication method proposed in this application. Detailed Implementation

[0063] Based on the content described in the background technology section, in the context of semantic communication of multimodal data, in order to reduce the storage overhead and memory capacity requirements of each edge node in the communication network, as well as the communication resource overhead caused by sending and receiving complete semantic models between different nodes, it is proposed to adjust the semantic model architecture for a single modality. By using a multimodal dataset, a semantic model suitable for multimodal data is trained, unifying multimodal data in a semantic space to achieve multimodal data processing. In this way, each edge node can deploy such a semantic model and perform the same pragmatic task on different modal data.

[0064] However, in this semantic communication process, the semantic model used for multimodal data has poor scalability, is limited to a limited number of data modalities, and has certain requirements for the storage and computing capabilities of edge nodes. As a result, it is difficult to guarantee the overall performance of edge node configuration in actual communication networks in terms of scalability and hardware cost, which also limits the applicability of this semantic communication method.

[0065] To further improve the above-mentioned problems, this application proposes to slice the semantic model trained on the dataset of each modality category to obtain cross-modal slices for the same pragmatic function. That is, to determine the slices (i.e., parts of the semantic model) contained in the semantic model of each modality category. In this way, after the modality of the input source of any pragmatic task is updated, the slices of the semantic model corresponding to the new modality category can be directly obtained to realize the semantic model update and obtain the semantic model for processing the input data of the new modality category. Compared with transmitting the complete semantic model corresponding to the new modality category, the slices transmitted in this application are partial semantic models, which greatly reduces the communication resource overhead and the storage overhead and memory capacity requirements of edge nodes for the semantic model.

[0066] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0067] Reference Figure 1 This is a flowchart illustrating an optional embodiment of the semantic communication method proposed in this application. This method can be applied to any edge node of a communication network. The edge node can support cloud computing services and can be a terminal device or server with certain data processing capabilities, such as a business platform built on the communication network side close to the local terminal, which can provide storage, computing and network resources. Some key business applications can be pushed down to the network edge to reduce bandwidth and latency losses caused by network transmission and multi-level forwarding. This application does not describe the system architecture and application principles of the communication network in detail. This application can determine the device type of the edge node according to the application scenario of the communication network.

[0068] Based on this, in the semantic communication scenario of communication networks, such as Figure 1 As shown, the semantic communication method performed by any edge node may include, but is not limited to, the following steps:

[0069] Step S11: Obtain the new modality category of the pragmatic task input source;

[0070] When performing any pragmatic task (such as image semantic segmentation or semantic recognition, etc., this application does not limit the semantic communication scenario) at the edge node, the semantic model structure for pragmatic tasks of different task categories often differs greatly. When performing a pragmatic task of a certain task category for the first time, the semantic model that can be in the input data of the input source to be processed can be obtained directly from at least one semantic model trained for the pragmatic task of that task category, based on its task category and the modality category of the current input source to be processed. The semantic model is then deployed in the edge node. After receiving input data from the input source, the edge node can execute the semantic model to perform the pragmatic task on the input data, thus meeting the semantic communication requirements.

[0071] The aforementioned input source can be the source of input data, such as at least one device, including a data acquisition device, a data transmission device, a data storage device, a data input device, or a communication server that supports a certain application service. This application does not limit the type of input source or the modal type of the input data it provides, and can be determined as appropriate.

[0072] In the above semantic communication process, if the modality of the input source of the above pragmatic task is updated, such as the modality category of the current input source of the pragmatic task being different from the modality category of the previous input source, the modality category of the current input source can be obtained. In order to distinguish it from the modality category of the previous input source, it can be described as the new modality category of the input source of the pragmatic task.

[0073] Optionally, in the above semantic communication process, it is also possible that, while the input source for the pragmatic task remains unchanged, the modal category of the current input data from that input source differs from the modal category of the previous input data. In this case, the modal category of the input source for the pragmatic task is determined to be updated, and the modal category of the current input data (i.e., the data to be processed in the pragmatic task) can be recorded as the new modal category of the input source for the pragmatic task. Therefore, this application does not limit the method for determining the new modal category of the pragmatic task input source in step S11 above; it can be determined as appropriate.

[0074] Step S12: Send a semantic model update request; the semantic model update request includes the task category of the pragmatic task and the new modality category;

[0075] Following the above analysis, when the modality update of the input data from the pragmatic task input source is determined and a corresponding new modality category is obtained, the semantic model of the original modality category of the input data used to process the pragmatic task input source is no longer applicable to processing the input data of the new modality category. It is necessary to obtain a semantic model applicable to processing the input data of the new modality category. Therefore, the edge node can send a semantic model update request containing the task category of the pragmatic task and the new modality category of its input source in order to request the semantic model corresponding to the new modality category.

[0076] Optionally, edge nodes can send semantic model update requests to the cloud of the communication network, so that the cloud can determine the corresponding first semantic model based on the task category and new modality category contained in the semantic model update request. The first semantic model can be obtained by training on the dataset corresponding to the new modality category labeled for the pragmatic task. This application does not describe in detail the training process of the semantic model corresponding to the dataset of each modality category.

[0077] In another possible implementation, if other edge nodes of the communication network store a first semantic model corresponding to a new modality category for the pragmatic task, the edge node can also send a semantic model update request to these other edge nodes to request an update of the semantic model in that edge node that is not suitable for processing the input data of the new modality category, thereby obtaining the required semantic model. Optionally, if the edge node locally stores fragments of the semantic model corresponding to different modality categories for the pragmatic task, the edge node can send a semantic model update request to the local storage device to request the fragment corresponding to the new modality category, thus constructing the required semantic model.

[0078] Therefore, in different application scenarios, edge nodes can send semantic model update requests to different objects to request updates to the semantic model of the input data in the edge node that is not suitable for processing the new modality category. It should be understood that the communication method (such as the type of communication protocol followed by the transmission of the semantic model update request) used by the edge node to send the semantic model update request to different objects can be different and can be determined according to actual communication needs. This application does not detail the specific implementation process of step S12.

[0079] Step S13: Receive the first slice; the first slice may be a partial first semantic model corresponding to the task category and the new modality category;

[0080] As described above regarding the technical solution of this application, if an edge node needs to process input data of a certain modality category for a pragmatic task, obtaining the corresponding complete semantic model will increase the storage overhead and memory capacity requirements of the edge node, and will also increase communication resource overhead when transmitting semantic models between different edge nodes. In order to solve this technical problem, when requesting an update of the semantic model according to the above method, it is possible to request the first slice in the first semantic model applicable to the input data of the new modality category for processing the input source of the pragmatic task, that is, the part of the first semantic model used to implement the pragmatic function of the input data of the new modality category, which is different from the slice in the semantic model used to process the input data of the original modality category, which implements the same pragmatic function of the input data of the original modality category.

[0081] In this way, the edge node can receive the first slice sent by the cloud or other edge nodes or local storage devices. The storage capacity of the first slice is much smaller than the storage capacity of the entire first semantic model. Compared with receiving the complete first semantic model, this greatly reduces the consumption of data transmission resources and the occupation of storage resources for the edge node.

[0082] In practical applications of this application, the process of segmenting and processing the semantic models obtained from training to obtain corresponding slices can be implemented by the cloud or the object that trained the semantic model. This application does not restrict the slice acquisition process of each semantic model and its execution object, and it can be determined as appropriate.

[0083] Step S14: Using the first slice, a second semantic model is obtained to perform pragmatic tasks on the input sources of the new modality category through the second semantic model.

[0084] After obtaining the first slice at the edge node using the method described above, the first slice can be used to replace the third slice in the third semantic model executed in the past for the same pragmatic task, thus forming a second semantic model for a new modality category of the input source for the pragmatic task. In this way, the second semantic model can be executed to perform pragmatic tasks on the input data of the new modality category from the input source, thereby meeting the semantic communication requirements.

[0085] For example, such as Figure 2 As shown, based on the above analysis, each semantic model can be divided into a base part and a slice after segmentation. Assuming that for datasets of different modal categories of the same pragmatic task, semantic models A and B are trained and obtained in a one-to-one correspondence, after parameter difference analysis of different semantic models (such as differences in data correlation, etc., this application does not limit the content of the difference analysis and can be determined as appropriate), semantic model A can be divided into base part A and slice A, and semantic model B can be divided into base part B and slice B.

[0086] In this application, the network structures of base part A and base part B can be the same or substantially the same, and can be used interchangeably. Slice A and slice B can be partial semantic models that achieve the same pragmatic function, but the model parameters of the two are significantly different, such as the corresponding network structures and their parameters. They cannot be directly substituted to achieve semantic processing of data of the same modality category. This application does not describe in detail the network structures and parameters of the base parts and slices obtained by semantic model segmentation, which can be determined as appropriate.

[0087] Based on this, when the edge node determines that the pragmatic task is being executed for the first time, after obtaining the corresponding semantic model A according to the modality category A of the current input source to be processed and the task category, the pragmatic task can be executed on the input data of modality category A through the semantic model A. After obtaining the modality category B of the input source for the pragmatic task, that is, the new modality category mentioned above is modality category B, if the pragmatic task is continued to be executed on the input data of modality category B through the semantic model A, the semantic communication requirements cannot be met. Therefore, this application can obtain the slice B (i.e., the first slice mentioned above) corresponding to the modality category B for the pragmatic task according to the method described above. Then, as... Figure 2 As shown, slice B can be combined with the basic part A in semantic model A to form semantic model C (i.e., the second semantic model mentioned above). That is, by replacing slice A in semantic model A with slice B, semantic model C is obtained.

[0088] As analyzed above, slice A and slice B can achieve the same pragmatic function. The difference lies in that they are more suitable for data processing in the corresponding modality category. The basic parts of the semantic models A and B are basically the same. Therefore, the performance of semantic model C, which is composed of the basic part A and slice B, is very similar to the performance of semantic model B, which contains the basic part B and slice B. If the difference between the output accuracy of semantic model C and semantic model B is less than a threshold (which can be a very small value, and the specific size is not limited), semantic model C can replace semantic model B to perform the same pragmatic task on the input data of modality category B and meet the semantic communication requirements.

[0089] As can be seen, for edge nodes performing pragmatic tasks, when the modal category of the input data from the pragmatic task input source is a new modal category, it is only necessary to obtain cross-modal slices, such as slice B above, according to the method described above, to construct a second semantic model for the input data in the new modal category, such as semantic model C above. Compared with directly obtaining the complete semantic model corresponding to the new modal category, such as semantic model B above, this greatly reduces the memory capacity requirements of the edge node and the overhead on communication resources while enhancing the edge node's ability to process multimodal data.

[0090] Reference Figure 3This is a flowchart illustrating an optional embodiment two of the semantic communication method proposed in this application. This embodiment can still be described from the perspective of the edge node side of the communication network, such as... Figure 3 As shown, the method may include:

[0091] Step S31: Obtain the task category of any pragmatic task to be performed, and the modality category of the input source to be processed for that pragmatic task;

[0092] Step S32: Send a semantic model acquisition request; the semantic model acquisition request may include the above-mentioned task categories and modality categories;

[0093] In practical applications, for any pragmatic task in semantic communication, a semantic model for the corresponding modality category can be trained in advance using datasets of different modality categories, i.e. datasets of different modality categories labeled for the same pragmatic task. After difference analysis, the slices in each semantic model for that pragmatic task can be determined.

[0094] As in the example above, using dataset A of modality category A and dataset B of modality category B, corresponding semantic models A and B that achieve the same pragmatic task are trained respectively. Then, semantic models A and B are segmented to obtain slice A of semantic model A and slice B of semantic model B, which can be associated with the corresponding modality category and stored.

[0095] It should be understood that for datasets of other modalities of the aforementioned pragmatic task, corresponding semantic models can also be trained, and these models can be segmented into slices corresponding to the modalities. The implementation process is similar and will not be detailed here. Therefore, a semantic model used to achieve a one-to-one correspondence between different modalities of the same pragmatic task can be segmented into slices using the same pragmatic function. These slices are then associated with their corresponding modalities and stored, and sent to each edge node in the communication network. Following the method described above, each edge node obtains a semantic model for processing the input data of the corresponding modality and performs the pragmatic task on the input data of that modality.

[0096] Furthermore, when it is necessary to perform the aforementioned pragmatic tasks on data of newly added modal categories, a new semantic model can be trained using the dataset of the newly added modal category labeled for the pragmatic task, as described above. The model can then be segmented to obtain slices that implement the aforementioned pragmatic functions. These slices can be sent to the corresponding edge nodes to quickly obtain the semantic model for the newly added modal category. This enables semantic processing of the input data for the newly added modal category, thus meeting the semantic communication requirements and increasing the scalability of the semantic model of the edge nodes.

[0097] Based on the above analysis, for any edge node in a communication network, when it first executes any pragmatic task, that is, when the edge node has not deployed any semantic model to implement the pragmatic task, it is necessary to first deploy a semantic model to implement the pragmatic task in the edge node. In order to obtain a semantic model that can accurately process the input data of the current input source to be processed for the pragmatic task, the task category of the pragmatic task and the modality category of the current input source to be processed can be obtained, and a semantic model acquisition request containing the task category and modality category can be sent, that is, a semantic model deployment request. For example, the semantic model acquisition request can be sent to the cloud of the communication network or to other edge nodes that have the requested semantic model. This application does not restrict the way the semantic model acquisition request is sent, and it can be determined as appropriate.

[0098] Step S33: Receive the third semantic model; the third semantic model may be a trained semantic model corresponding to the above task category and modality category;

[0099] Step S34: Execute the third semantic model to perform pragmatic tasks on the input source to be processed through the third semantic model;

[0100] Following the above analysis, for any edge node that has not deployed a semantic model to implement the above pragmatic task, a complete semantic model of the input data of the input source to be processed for accurately processing the pragmatic task can be obtained according to the method described above, namely the above-mentioned third semantic model. This model can be deployed in the edge node so that the edge node can execute the third semantic model and perform pragmatic tasks on the input data from the input source to be processed. The implementation process is not detailed in this application.

[0101] In some other embodiments proposed in this application, when an edge node first implements a certain pragmatic task, the sent semantic model acquisition request may also include the task category of the pragmatic task. After receiving any semantic model corresponding to the task category, and subsequently determining the modality category of the input source of the pragmatic task, if it is different from the modality category of the dataset used to train the semantic model, it is equivalent to obtaining a new modality category of the input source of the pragmatic task. Cross-modal slices can be obtained according to the described method, thereby obtaining the semantic model of the input data of the input source of the pragmatic task.

[0102] Step S35: Obtain the new modality category of the input source for this pragmatic task;

[0103] Step S36: Send a semantic model update request; the semantic model update request includes the task category of the pragmatic task and the new modality category;

[0104] Step S37: Receive the first slice; the first slice may be a partial first semantic model corresponding to the above-mentioned task category and the new modality category;

[0105] The implementation process of steps S35-S37 can be referred to the description of the corresponding parts of the above embodiments, and will not be described in detail in this embodiment.

[0106] For example, taking image semantic segmentation as a pragmatic task, the semantic model for implementing image semantic segmentation can adopt, but is not limited to, the Deeplab network structure, in conjunction with the above. Figure 2 As described in the example, dataset A could be the Cityscapes dataset (i.e., a city landscape dataset), and dataset B could be the GTA5 dataset (i.e., a game scene dataset). This application does not elaborate on the acquisition methods of these two types of datasets or their data content. This embodiment can extend different datasets from the same source domain to obtain semantic models for image semantic segmentation processing of data of corresponding modality categories.

[0107] Specifically, the Deeplab network (i.e., a general initial semantic model) can be trained using the Cityscapes dataset to obtain the corresponding semantic model A; the Deeplab network can be trained using the GTA5 dataset to obtain the corresponding semantic model B. Then, the differences in network parameters between the two datasets can be measured using, but not limited to, CCA (Canonical Correlation Analysis). The model parts with large differences are segmented to obtain slices in the corresponding semantic model. This application does not limit the segmentation implementation method of different semantic models and can be determined as appropriate.

[0108] Next, following the method described above, slice A of semantic model A trained on the GTA5 dataset can be replaced with slice B of semantic model B trained on the Cityscapes dataset to obtain semantic model C. Then, MIoU (Mean Intersection over Union) can be used as the semantic segmentation evaluation metric to measure the performance of the semantic model. The performance of semantic model C in image semantic segmentation is then tested using a test dataset from the GTA5 dataset.

[0109] Semantic segmentation of the GTA5 test dataset using semantic model C yields a MIoU score of 0.507. Furthermore, semantic segmentation of the Cityscapes test dataset using the same semantic model A yields a MIoU score of 0.693, and semantic segmentation of the GTA5 test dataset using the same model A yields a MIoU score of 0.256. The MIoU score can be the ratio between the intersection and union of the ground truth set and the predicted value set output by the model, or a score obtained according to certain rules. Generally, a higher MIoU score indicates better semantic segmentation and higher performance of the corresponding semantic model. Therefore, if the MIoU score is 1, the semantic segmentation information is fully recovered during the semantic segmentation process using the corresponding semantic model; if the MIoU score is 0, the semantic segmentation of the corresponding semantic model fails.

[0110] The semantic model performance test results show that using the semantic model corresponding to a certain modality category to perform the same pragmatic task on data of another modality category yields poor semantic communication results. In other words, the semantic model has poor semantic processing performance for data of that other modality category. Therefore, when updating the modality of the input source for the same pragmatic task, it is necessary to obtain the semantic model of the input source for the new modality category. In order to reduce the storage overhead and memory capacity requirements of edge nodes, it is not necessary to directly obtain the complete semantic model of the input source for the new modality category. Instead, a partial semantic model that implements the same pragmatic function corresponding to the new modality category can be obtained, that is, the part with the largest difference from the semantic models of other modality categories that implement the same pragmatic task, which is denoted as a slice.

[0111] As can be seen from the test results above, the performance of the second semantic model, which is based on the first slice and the third semantic model, is similar to that of the first semantic model trained on the dataset of the new modality category to perform the same pragmatic task. For example, the difference between the MIoU scores is less than the score threshold (the value is small, and the specific size can be determined according to the situation). Therefore, this application can meet the requirements of semantic communication by performing pragmatic tasks on the input sources of the new modality category through the second semantic model.

[0112] Step S38: Use the first slice to replace the third slice in the third semantic model to obtain the second semantic model;

[0113] Step S39: Execute the second semantic model to perform pragmatic tasks on the input sources of the new modality category through the second semantic model.

[0114] As analyzed above, the third semantic model refers to a semantic model that performs pragmatic tasks on input sources of the original modality category. When it is determined that the modality category of the input source of the pragmatic task changes and a new modality category of the input source of the pragmatic task is obtained, since the third semantic model is no longer applicable to performing pragmatic tasks on input data of the new modality category, this application can obtain the first slice corresponding to the new modality category from the slices (i.e., different parts of the semantic model) of different modality categories corresponding to the task category of the pragmatic task. The second semantic model is directly constructed from this cross-modality slice and the basic part in the third semantic model. That is, the semantic model applicable to performing pragmatic tasks on input data of the new modality category, thus meeting the semantic communication needs of input data of the new modality category.

[0115] Therefore, in scenarios where edge nodes perform the same pragmatic task on different modal data, it is not necessary to obtain the complete semantic model corresponding to each modal category for the pragmatic task. Instead, it is only necessary to obtain the slice corresponding to the new modal category, i.e., the cross-modal slice, and replace the slice in the original semantic model corresponding to the modal category. This yields a complete semantic model for performing pragmatic tasks on the input data of the new modal category. This reduces the overhead of storage capacity and communication resources on the edge nodes, as well as the memory capacity requirements of the edge nodes. It also facilitates the expansion of the semantic model of the edge nodes and meets the semantic communication needs of efficient processing of multimodal data on the edge nodes.

[0116] Reference Figure 4 This is a flowchart illustrating an optional embodiment three of the semantic communication method proposed in this application. This embodiment provides a more detailed description of how edge nodes obtain the first slice in the semantic communication method proposed above, but it is not limited to the detailed implementation method described in this embodiment. Figure 4 As shown, the method may include:

[0117] Step S41: Obtain the new modality category of the input source for this pragmatic task;

[0118] Step S42: Determine whether candidate slices for the pragmatic task are stored. If yes, proceed to step S43; otherwise, proceed to step S49.

[0119] In this embodiment of the application, the above-mentioned candidate slices may be partial semantic models trained for the pragmatic task. In conjunction with the description of the corresponding part of the above embodiment, for the same pragmatic task, a semantic model corresponding to the pragmatic task can be trained through datasets of different modal categories. After analyzing the differences between different semantic models that achieve the same pragmatic task, slices that achieve the same pragmatic function in each semantic model are determined.

[0120] Optionally, after obtaining multiple semantic models that implement the same pragmatic task and their contained slices in the cloud of the communication network, the cross-modal slices corresponding to each edge node of the communication network can be determined, that is, the slices contained in other semantic models besides the semantic model. Then, each cross-modal slice is sent to the corresponding edge node. The edge node can determine the received cross-modal slice as a candidate slice for the pragmatic task, associate it with the task category of the corresponding pragmatic task and the modality category of the input data that can perform the pragmatic task, and then store it.

[0121] Step S43: Obtain the candidate modality category corresponding to the candidate slice;

[0122] The candidate modality category is the modality category of the dataset used to train the semantic model to which the corresponding candidate slice belongs. Different modality identifiers can be configured to represent the one-to-one corresponding modality category. This application does not restrict the representation of different modality categories.

[0123] Step S44: Compare the new modality category with the candidate modality categories to obtain the corresponding comparison results;

[0124] Step S45: Based on the comparison results, determine whether there exists any candidate modality category that is the same as the new modality category. If there is, proceed to step S46; otherwise, proceed to step S47.

[0125] Step S46: Obtain the candidate slice corresponding to the new modality category and determine it as the first slice;

[0126] Following the above analysis, if an edge node needs to obtain a cross-modal slice (such as the first slice mentioned above) of a new modality category for the pragmatic task input source, it can first check whether a candidate slice is stored locally. If a candidate slice is stored, it can check whether the cross-modal slice has been stored this time. This can be implemented in the comparison method described above, but is not limited to. If the cross-modal slice is stored, it can be read directly from the edge node's storage device.

[0127] Step S47: Send a semantic model update request; the semantic model update request includes the task category of the pragmatic task and the new modality category;

[0128] Step S48: Receive the first slice; the first slice may be a portion of the first semantic model corresponding to the above-mentioned task category and the new modality category;

[0129] If the edge node does not store candidate slices, or if it is determined by comparing the candidate modality categories of the candidate slices stored by the edge node itself with the obtained new modality category, and it is found that the edge node does not store candidate slices corresponding to the new modality category, a semantic model update request can be sent to the cloud or other edge nodes in the communication network to request the first slice for implementing pragmatic tasks on the input data of the new modality category. The implementation process can be referred to the description in the corresponding part of the context, and will not be described in detail in this embodiment.

[0130] Step S49: Using the first slice, a second semantic model is obtained to perform pragmatic tasks on the input sources of the new modality category through the second semantic model.

[0131] The implementation process of step S49 can be referred to the description of the corresponding part of the above embodiment, and will not be described in detail in this embodiment.

[0132] Based on the semantic communication method described above from the edge node side, the implementation process of the semantic communication method will be described below from the cloud side of the communication network. The cloud can be a cloud server that supports cloud computing services. It is the control terminal of edge computing and realizes the communication management of different edge nodes in the communication network. This application will not describe the structure and basic functions of the cloud in detail.

[0133] Reference Figure 5 This is a flowchart illustrating an optional embodiment four of the semantic communication method proposed in this application, as shown below. Figure 5 As shown, semantic communication methods executed in the cloud can include:

[0134] Step S51: Receive a semantic model update request; the semantic model update request includes the new modality category of the input information source for the semantic model to perform the pragmatic task, and the task category of the pragmatic task.

[0135] In this embodiment, after a new modality category of the pragmatic task input source is obtained at any edge node (i.e., the semantic model update request end), in order to realize the pragmatic task of the input data of the new modality category, a corresponding semantic model update request can be sent to the cloud. The implementation process can refer to the description of the semantic communication method executed on the edge node side above, and will not be described in detail in this embodiment.

[0136] Step S52: Obtain the first slice corresponding to the task category and the new modality category; the first slice is a partial first semantic model trained on the pragmatic task using the dataset belonging to the new modality category.

[0137] Based on the difference analysis of multiple semantic models for different modal categories of the same pragmatic task in the above embodiments, the relevant description content of each slice of multiple semantic models is obtained. The cloud can obtain the slices of these multiple semantic models, associate them with the task category and modal category of the corresponding pragmatic task, and store them. In this way, when the cloud receives any semantic model update request, it can obtain the first slice associated with the task category and new modal category contained in the request according to the association relationship between different slices and task category and modal category.

[0138] It should be noted that the implementation process of training a one-to-one corresponding semantic model for different modal categories of the dataset for any pragmatic task, and then further segmenting it to obtain slices, as well as the execution object of this implementation process (such as the cloud and / or at least one edge node), can be referred to but not limited to the description of the corresponding part of the context. This embodiment will not be described in detail here.

[0139] Step S53: Send the first slice so that the semantic model update requesting end can use the first slice to obtain a second semantic model that can perform pragmatic tasks on input sources with new modality categories.

[0140] It is evident that when any edge node needs to process input data of a new modality category from the input source of the same pragmatic task, the cloud only needs to feed back the partial semantic model corresponding to the new modality category, i.e., the first slice, to the edge node. Compared to sending the entire semantic model, this greatly reduces the amount of data transmitted, reduces the consumption of communication resources, and also reduces the storage overhead and memory usage of the transmitted data at the receiving end. This reduces the storage overhead and memory capacity requirements of the edge node and increases the scope of application.

[0141] For edge nodes that receive the first slice, the third slice of the third semantic model for the original modality category can be directly replaced to obtain a second semantic model for performing pragmatic tasks on input data of the new modality category. The performance of the second semantic model is not much different from that of the first semantic model trained on the dataset of the new modality category. Using the second semantic model can meet the semantic communication requirements of input data of the new modality category, improve the scalability of the semantic model of the edge node, and meet the semantic communication of multimodal data.

[0142] Reference Figure 6 This is a flowchart illustrating an optional embodiment five of the semantic communication method proposed in this application. This embodiment can still be described from the cloud side, such as... Figure 6 As shown, the semantic communication method may include:

[0143] Step S61: Receive multiple semantic models; these multiple semantic models can be pragmatic tasks for the same task category, trained on datasets of different modal categories;

[0144] Step S62: Perform parameter difference analysis on the multiple semantic models to obtain the slices for the same pragmatic function contained in each of the multiple semantic models.

[0145] Step S63: After associating the slices of each of the multiple semantic models with the task category and the corresponding modality category, store them;

[0146] Referring to the description of the corresponding parts in the above embodiments, such as Figure 2 As shown, datasets labeled with different modal categories for the same pragmatic task can be used. Then, a semantic model that implements the pragmatic task can be trained using each dataset. By analyzing parameter correlation differences, the parts with large parameter differences can be identified, and the semantic model can be segmented accordingly to obtain slices of the same pragmatic function in different semantic models. In order to facilitate the subsequent acquisition of cross-modal slices by edge nodes, the slices of each semantic model for the same pragmatic task can be associated with the task type of the pragmatic task and the modal category of the dataset used for training and then stored. The specific storage method is not limited in this application and can be determined as appropriate.

[0147] In some embodiments, multiple semantic models implementing the same pragmatic task can be trained by the cloud according to the method described above, or by one or more edge nodes and / or terminal devices capable of accessing the edge nodes. The trained semantic models are then sent to the cloud, allowing the cloud to obtain slices of each semantic model. In the case where the cloud receives multiple trained semantic models from multiple edge nodes, different edge nodes can train semantic models corresponding to different modalities of the same pragmatic task, avoiding redundant training and resource waste. The implementation process is not detailed in this application.

[0148] Optionally, if a semantic model for each of multiple modal categories is trained for a pragmatic task at any edge node or terminal device, parameter difference analysis can be performed on the multiple semantic models in accordance with, but not limited to, the methods described above, to achieve segmentation of the multiple semantic models, obtain slices corresponding to each modal category, and then send these multiple semantic models and their contained slices to the cloud for storage.

[0149] Step S64: Receive a semantic model acquisition request; the semantic model acquisition request includes the task category of the pragmatic task to be performed by the requested semantic model, and the modality category of the input information source to be processed for the pragmatic task.

[0150] Step S65: Obtain the third semantic model corresponding to the task category and modality category;

[0151] Step S66: Send the third semantic model so that the semantic model acquisition requesting end executes the third semantic model and performs pragmatic tasks on the input source to be processed;

[0152] Based on the description of the corresponding part of the edge node side embodiment above, when any edge node needs to execute a certain pragmatic task for the first time, since no semantic model implementing the pragmatic task is deployed in the edge node, it needs to first obtain a semantic model that can implement the pragmatic task. After the cloud receives the semantic model acquisition request sent by the edge node, it can parse the semantic model acquisition request to obtain the task category of the pragmatic task to be implemented and the modality category of the input source to be processed for the pragmatic task. Then, it can select the semantic model corresponding to the modality category of the input source to be processed from the stored semantic models for different modality categories of the pragmatic task, and denot it as the third semantic model, and send it to the corresponding edge node (i.e. the semantic model acquisition request end) for deployment.

[0153] Optionally, in order to enable terminal devices to perform pragmatic tasks on the input sources to be processed through other edge nodes, the cloud can also send the obtained third semantic model to each edge node in the communication network for deployment. In this way, when the terminal device sends data of the same modality category for the pragmatic task to any edge node, the edge node can perform the pragmatic task on the data through the third semantic model.

[0154] In some other embodiments, after the cloud stores multiple slices of different modal categories for the same pragmatic task according to the above method, it can determine the modal category of the cross-modal slice of each edge node, that is, other modal categories besides the modal category corresponding to the semantic model deployed by the edge node. Then, it can send the cross-modal slices associated with the task category of the pragmatic task (that is, the slices contained in the semantic models other than the requested semantic model among the multiple semantic models associated with the task category of the pragmatic task), such as sending the cross-modal slices to the corresponding edge nodes as candidate slices for storage.

[0155] Step S67: Receive a semantic model update request;

[0156] Following the above analysis, for the semantic model acquisition request end or other edge nodes, after determining the modality update of the pragmatic task input source and obtaining the new modality category, a semantic model update request can be sent to the cloud. Alternatively, if it is determined that the candidate slice corresponding to the new modality category is not stored locally, the semantic model update request can be sent to the cloud to request the slice corresponding to the new modality category that implements the pragmatic task. This application does not restrict the sending object of the semantic model update request or its triggering conditions, and can be determined as appropriate.

[0157] Step S68: Obtain the first slice corresponding to the new modality category of the pragmatic task and its input information source;

[0158] Step S69: Send the first slice to the semantic model update request end to obtain a second semantic model that can perform pragmatic tasks on input sources with new modality categories using the first slice.

[0159] By querying the content contained in the semantic model update request, the cloud can find the first slice of the semantic model update request end (such as any edge node) that performs pragmatic tasks on the input source of the new modality category. Then, it only needs to feed back the first slice to the semantic model update request end so that it can obtain the second semantic model according to the method described above, and realize the semantic processing of the new modality data.

[0160] Since the first slice is part of the first pragmatic model, its data volume is much smaller than that of the entire first semantic model. Thus, sending the first slice from the cloud to the edge nodes greatly reduces the consumption of communication resources compared to sending the complete first semantic model. For the edge nodes receiving the first slice, it also reduces storage overhead and memory capacity requirements.

[0161] In some further embodiments of this application, in conjunction with the semantic communication method described in the above embodiments, before the first node in the communication network trains a semantic model to implement the pragmatic task, it can query whether other edge nodes and the cloud have already trained a semantic model to implement the pragmatic task, and the modality category to which the dataset used for training the trained semantic model belongs, etc. This can be achieved by sending a semantic model query request. It can be determined that the second node of the communication network (i.e., at least one edge node different from the first node) or the cloud server has trained a certain number of semantic models to implement the pragmatic task. The semantic model training information containing the modality category corresponding to the trained semantic model can be fed back to the first node so that the first node can train semantic models of other modality categories, and then send them to the cloud, whereby the cloud determines the slices contained in each semantic model according to the above method.

[0162] Once it is determined that the second node or the cloud has already trained multiple semantic models that implement all modal categories of the pragmatic task, that is, has trained semantic models for a preset number of modal categories for the pragmatic task, i.e., the number of received semantic models is equal to the preset number of modal categories, a semantic model training notification message can be sent to the first node to notify the first node that it no longer needs to train a semantic model to implement the pragmatic task, and can directly obtain the required semantic model according to the method described above.

[0163] For edge nodes or terminal devices that have trained semantic models, the semantic models can be directly reported to the cloud for segmentation processing, or the corresponding slices can be obtained and the semantic models and their slices can be sent to the cloud for storage. This application does not restrict the training process of multiple semantic models corresponding to different modal categories of the same pragmatic task, nor the execution object that performs the training or even semantic model segmentation processing. It can be determined according to the actual scenario requirements.

[0164] Based on the above analysis, referring to Figure 7 The diagram shown is a flowchart of an optional embodiment six of the semantic communication method proposed in this application. This embodiment describes the training and segmentation process of the semantic model. This method can be executed in the cloud, such as... Figure 7 As shown, the semantic communication method may include:

[0165] Step S71: Receive a semantic model training request;

[0166] As analyzed above, a semantic model training request may include multiple modalities of different input sources for the pragmatic task to be performed by the requested semantic model, as well as the task category of the pragmatic task. The semantic model training request can be sent by any edge node, and this application does not restrict the way it is generated or triggered.

[0167] Step S72: Obtain the datasets for each of the multiple modal categories labeled for the task category;

[0168] Step S73: Based on different datasets, train semantic models corresponding to task categories and corresponding modal categories;

[0169] Combination Figure 2 The training process shown can be used to train an initial semantic model on datasets A and B, or even more datasets, labeled with the same pragmatic task. This allows for the acquisition of a semantic model that can be used to implement the pragmatic task for data of the corresponding modality category. For example, semantic model A corresponds to dataset A, and semantic model B corresponds to dataset B. The training process is not described in detail.

[0170] Step S74: Perform parameter difference analysis on the multiple semantic models obtained from training to obtain the slices for the same pragmatic function contained in each of the multiple semantic models.

[0171] Step S75: Associate the slices of each of the multiple semantic models with the task category and the corresponding modality category, and then store them.

[0172] The implementation process of steps S74 and S75, as well as the implementation process of updating the existing semantic model in the edge node based on the query cross-modal slice to obtain the semantic model for processing the input data of the new modality category, can be referred to the description of the corresponding part of the above embodiment, and will not be described in detail in this embodiment.

[0173] Optionally, after obtaining the dataset for the new modality category of the pragmatic task, the corresponding semantic model can still be trained according to the scheme described above. The model can then be compared with other trained semantic models to obtain the slices contained in the newly trained semantic model. These slices can be added to the cloud storage space so that the edge nodes can support the performance of pragmatic tasks on the data of the new modality category, thereby improving the scalability of the semantic model of the edge nodes.

[0174] Optionally, this application also proposes a semantic communication device that can be applied to any edge node of a communication network. This device may include:

[0175] The modality category acquisition module is used to obtain new modality categories from the input sources of pragmatic tasks;

[0176] A semantic model update request sending module is used to send a semantic model update request; the semantic model update request includes the task category of the pragmatic task and the new modality category;

[0177] A first slice receiving module is configured to receive a first slice; the first slice is a partial first semantic model corresponding to the task category and the new modality category.

[0178] The second semantic model acquisition module is used to obtain a second semantic model using the first slice, so as to perform the pragmatic task on the input source through the second semantic model.

[0179] Regarding the functional units included in each functional module of the aforementioned edge node, please refer to the relevant description of the semantic communication method described above from the edge node side. This embodiment will not elaborate on them here.

[0180] Optionally, this application also proposes a semantic communication device that can be applied to the cloud of a communication network, the device including:

[0181] A semantic model update request receiving module is used to receive a semantic model update request; the semantic model update request includes a new modality category of the input information source for the semantic model to perform the pragmatic task, and the task category of the pragmatic task.

[0182] The first slice acquisition module is used to obtain a first slice stored corresponding to the task category and the new modality category; the first slice is a partial first semantic model trained for the pragmatic task using a dataset belonging to the new modality category.

[0183] The first slice sending module is used to send the first slice so that the semantic model update request end can use the first slice to obtain a second semantic model that can perform the pragmatic task on the input source with the new modality category.

[0184] Regarding the functional units included in each of the aforementioned cloud-based functional modules, please refer to the relevant description of the semantic communication method described above from the cloud side; this embodiment will not elaborate further here.

[0185] It should be noted that the various modules and units in the above-mentioned device embodiments can all be stored as program modules in the memory of the corresponding side device in the communication network. The processor in the side device executes the above-mentioned program modules stored in the memory to realize the corresponding functions. The functions realized by each program module and its combination, as well as the technical effects achieved, can be referred to the description of the corresponding part of the above-mentioned method embodiment executed by the corresponding side device. This embodiment will not repeat it.

[0186] This application also proposes a computer-readable storage medium storing at least one computer instruction set, which can be loaded and executed by a processor to implement the semantic communication method executed on the corresponding side of the aforementioned communication network.

[0187] Reference Figure 8 The above is a schematic diagram of the hardware structure of an optional example of a computer device suitable for the semantic communication method proposed in this application. The computer device may include a transceiver 81 and a processor 82, wherein:

[0188] The transceiver 81 can be used to receive and send information. It can be a communication module that supports wireless communication networks or wired communication networks, such as a WIFI module, a 5G / 6G (fifth generation mobile communication network / sixth generation mobile communication network) module, a wireless radio frequency module, a short-range communication module, a GPRS module, or a network data cable, etc. The composition structure of the transceiver 81 can be determined according to the communication method between different nodes in the communication network. This application does not limit the composition structure of the transceiver 81.

[0189] When a computer device is configured as a device with different identities in a communication network, the processor 82 can be used to implement the semantic communication method executed on the corresponding side. For example, if the computer device is configured as any edge node in the communication network, it can be used to implement the semantic communication method described above from the edge node side; if the computer device is configured as the cloud of the communication network, it can be used to implement the semantic communication method described above from the cloud side. The implementation process is described in the corresponding side embodiment above, and will not be repeated in this embodiment.

[0190] In practical applications of this application, the processor 82 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices. The type of processor 82 in different execution objects of the semantic communication method can be the same or different, and this application does not impose any restrictions.

[0191] Optionally, the semantic communication method described above can be implemented by program code. The processor 82 may include a storage device for storing program code that implements the semantic communication method executed on the corresponding side. The processor 82 can implement the semantic communication method executed on the corresponding side by executing the program code.

[0192] In some other embodiments, the computer device may also include a memory, which is a device independent of the processor 81, such as a high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device, as described above, which can be used to store program code implementing a semantic communication method executed on the corresponding side, and the processor 82 loads and executes the program code to implement the semantic communication method executed on the corresponding side.

[0193] It should be understood that, Figure 8 The structure of the computer device shown does not constitute a limitation on the computer device in the embodiments of this application. In practical applications, the computer device may include more than Figure 8 The additional components shown, or combinations of certain components, such as sensor modules composed of various sensors, alarm devices, power supply modules, etc., when the computer equipment is a terminal device, may also include at least one input component such as a touch sensing unit for sensing touch events on a touch display panel, a keyboard, a mouse, a camera, a microphone, etc., and at least one output component such as a monitor, a speaker, a vibration mechanism, a lamp, etc., which can be determined according to the type of computer equipment and its functional requirements, and will not be listed one by one in this application.

[0194] It should be noted that, regarding the above embodiments, relational terms such as "first," "second," etc., are merely used to distinguish one operation, unit, or module from another, and do not necessarily require or imply any such actual relationship or order between these units, operations, or modules. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, or system. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, or system that includes said element.

[0195] Furthermore, as shown in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may also include plural forms. Also, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone.

[0196] Furthermore, the flowcharts illustrating the operations performed by the system according to embodiments of this application do not necessarily follow a precise sequential order. Instead, the steps can be processed in reverse order or simultaneously. Additionally, other operations can be added to these processes, or one or more steps can be removed from them.

[0197] Finally, the various embodiments in this specification are described in a progressive or parallel manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus and computer devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0198] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A semantic communication method, the method comprising: Obtain new modal categories from the input sources for pragmatic tasks; Send a semantic model update request; The semantic model update request includes the task category of the pragmatic task and the new modality category; Receive the first slice; The first slice is a partial first semantic model corresponding to the task category and the new modality category; Using the first slice, the third slice in the third semantic model executed in the past for the same pragmatic task is replaced to form a second semantic model of a new modality category for the input source of the pragmatic task, so as to perform the pragmatic task on the input source through the second semantic model.

2. The method according to claim 1, further comprising, before obtaining the new modality category of the pragmatic task input source: Obtain the task category of any pragmatic task to be performed, and the modality category of the input source to be processed for that pragmatic task; Send a semantic model retrieval request; The semantic model acquisition request includes the task category and the modality category; Receive a third semantic model; the third semantic model is a trained semantic model corresponding to the task category and the modality category; The third semantic model is executed to perform the pragmatic task on the input source to be processed through the third semantic model.

3. The method according to claim 1 or 2, wherein the third semantic model refers to a semantic model that performs the pragmatic task on the input source of the original modality category.

4. The method according to claim 1 or 2, wherein, in the case of obtaining a new modality category of the pragmatic task input source, the method further comprises: Identify at least one candidate slice that has been stored; The candidate slice is a partial semantic model trained for the pragmatic task; Obtain the candidate modality category corresponding to the candidate slice; The candidate modality category is the modality category of the dataset used to train the semantic model to which the corresponding candidate slice belongs; The new modality category is compared with the candidate modality category to obtain the corresponding comparison result; If the comparison result is determined to be the same as any of the candidate modality categories, the candidate slice corresponding to the new modality category is determined as the first slice, and the step of obtaining the second semantic model using the first slice is executed; If the comparison result indicates that the new modality category is different from all the candidate modality categories, or if it is determined that none of the candidate slices are stored, then the step of sending a semantic model update request is executed.

5. A semantic communication method, the method comprising: Receive semantic model update requests; The semantic model update request includes a new modality category of the input information source for the semantic model performing the pragmatic task, and the task category of the pragmatic task. Obtain the first slice corresponding to the task category and the new modality category; The first slice is a partial first semantic model trained on the pragmatic task using a dataset belonging to the new modality category; The first slice is sent so that the semantic model update requester can use the first slice to replace the third slice in the previously executed third semantic model for the same pragmatic task, thereby obtaining a second semantic model capable of performing the pragmatic task on an input source with the new modality category.

6. The method according to claim 5, further comprising: Receive semantic model retrieval request; The semantic model acquisition request includes the task category of the pragmatic task to be performed by the semantic model to be acquired, and the modality category of the input information source to be processed for the pragmatic task. Obtain a third semantic model corresponding to the task category and the modality category; Send the third semantic model.

7. The method according to claim 5 or 6, further comprising: Receive multiple semantic models; the multiple semantic models are pragmatic tasks of the same task category and are trained on datasets of different modal categories; Parameter difference analysis is performed on the multiple semantic models to obtain the slices for the same pragmatic function contained in each of the multiple semantic models. The slices of each of the multiple semantic models are associated with the task category and the corresponding modality category and then stored.

8. The method according to claim 7, wherein upon receiving any of the semantic model acquisition requests, the method further comprises: Send the cross-modal slices associated with the task category to enable the semantic model acquisition request end to identify the received cross-modal slices as candidate slices for the pragmatic task and store them. The cross-modal slice refers to a slice contained in a semantic model other than the requested semantic model among the multiple semantic models associated with the task category.

9. The method according to claim 5 or 6, further comprising: Receive semantic model training requests; The semantic model training request includes multiple modal categories of different input sources for the pragmatic task to be performed by the requested semantic model, as well as the task category of the pragmatic task. Obtain the datasets for each of the multiple modal categories labeled for the task category; Based on the dataset, a semantic model corresponding to the task category and the corresponding modality category is trained; Perform parameter difference analysis on multiple trained semantic models to obtain the slices for the same pragmatic function contained in each of the multiple semantic models. Each slice of the multiple semantic models is associated with the task category and the corresponding modality category and then stored.

10. A computer device, the computer device comprising a transceiver and a processor, wherein: When the computer device is configured as any edge node in a communication network, the processor is configured to implement: Obtain new modal categories from the input sources for pragmatic tasks; Send a semantic model update request; the semantic model update request includes the task category of the pragmatic task and the new modality category; Receive the first slice; The first slice is a partial first semantic model corresponding to the task category and the new modality category; Using the first slice, the third slice in the third semantic model that was previously executed for the same pragmatic task is replaced to form a second semantic model of a new modality category for the input source of the pragmatic task, so as to perform the pragmatic task on the input source through the second semantic model; When the computer device is configured as the cloud of the communication network, the processor is used to implement: Receive a semantic model update request; the semantic model update request includes a new modality category of the input information source for the semantic model to perform the pragmatic task, and the task category of the pragmatic task. Obtain the first slice corresponding to the task category and the new modality category; The first slice is a partial first semantic model trained on the pragmatic task using a dataset belonging to the new modality category; The first slice is sent so that the semantic model update requester can use the first slice to replace the third slice in the previously executed third semantic model for the same pragmatic task, thereby obtaining a second semantic model capable of performing the pragmatic task on an input source with the new modality category.

Images