Semantic base-based multi-user information transmission method and related apparatus

Abstract

The present disclosure provides a semantic base-based multi-user information transmission method and a related apparatus. When applied to a base station, the method comprises: determining a plurality of pieces of source information to be sent to a plurality of user terminals; performing semantic feature extraction on each piece of source information on the basis of a plurality of semantic feature extraction modes to obtain a first semantic feature set corresponding to each piece of source information; on the basis of differences between a plurality of semantic features in the first semantic feature sets, rearranging the plurality of semantic features in the plurality of first semantic feature sets to obtain second semantic feature sets; and superposing the plurality of first semantic feature sets and the plurality of second semantic feature sets to obtain a superposed semantic feature set, and sending the superposed semantic feature set to the plurality of user terminals. A semantic base comprises the correspondence between the semantic feature extraction modes and the semantic features, and the rearranged semantic features and the non-rearranged semantic features are superposed for transmission, thereby suppressing semantics interference between different users.

Description

Multi-user information transmission method based on semantic basis and related device

[0001] This application claims priority to Chinese Patent Application No. 202410214574.2, filed with the Patent Office of China on February 27, 2024, entitled “Multi-user Information Transmission Method and Related Device Based on Semantic Basis”. This application claims priority to Chinese Patent Application No. 202410752194.4, filed with the Patent Office of China on June 12, 2024, entitled “Multi-user Information Transmission Method and Related Device Based on Semantic Basis”. The entire contents of the prior applications are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of communication technology, and in particular to a multi-user information transmission method based on semantic bases and related devices. Background Art

[0003] Semantic communication technology significantly improves communication efficiency compared to related grammatical communication technologies by extracting and transmitting semantic information related to communication intent. However, these related semantic communication solutions suffer from semantic interference between users when transmitting information non-orthogonally across multiple users.

[0004] Summary of the Invention

[0005] In view of this, the embodiments of the present disclosure propose a multi-user information transmission method and related devices based on semantic bases, which can effectively solve the problem of semantic interference between users in semantic communication solutions, thereby effectively improving the accuracy of semantic communication.

[0006] According to a first aspect of the exemplary embodiments of the present disclosure, a multi-user information transmission method based on a semantic basis is provided, which is applied to a base station. The method includes:

[0007] Determining a plurality of information sources to be sent to a plurality of user terminals;

[0008] Based on several semantic feature extraction methods, semantic feature extraction is performed on each of the source information to obtain a first semantic feature set corresponding to each of the source information, wherein the first semantic feature set includes several semantic features obtained based on the several semantic feature extraction methods;

[0009] Rearranging the plurality of semantic features in the first semantic feature set based on differences between the plurality of semantic features in the first semantic feature set to obtain a second semantic feature set;

[0010] The plurality of first semantic feature sets and the plurality of second semantic feature sets are superimposed to obtain a superimposed semantic feature set, and the superimposed semantic feature set is sent to the plurality of user terminals.

[0011] Based on the same inventive concept, a second aspect of the exemplary embodiments of the present disclosure provides a multi-user information transmission method based on a semantic basis, applied to a user terminal, the method comprising:

[0012] receiving a superimposed semantic feature set sent by a base station;

[0013] In response to determining that the decoding order corresponding to the user terminal ranks first, decoding the superimposed semantic feature set to obtain information source information corresponding to the user terminal; or

[0014] In response to determining that the decoding order corresponding to the user terminal is not ranked first, serial interference elimination is performed on the superimposed semantic feature set to obtain the semantic feature set corresponding to the user terminal, and the semantic feature set is decoded to obtain the source information corresponding to the user terminal.

[0015] Based on the same inventive concept, a third aspect of the exemplary embodiments of the present disclosure provides a multi-user information transmission method based on a semantic basis, applied to a user terminal, the method comprising:

[0016] Determining information source to be sent to a base station;

[0017] performing semantic feature extraction on the source information based on a plurality of semantic feature extraction methods to obtain a third semantic feature set corresponding to the source information, wherein the third semantic feature set includes a plurality of semantic features obtained based on the plurality of semantic feature extraction methods;

[0018] In response to determining that the user terminal does not participate in the reordering, sending the third semantic feature set to the base station; or,

[0019] In response to determining that the user terminal participates in the rearrangement, the several semantic features in the third semantic feature set are rearranged based on the differences between the several semantic features in the third semantic feature set to obtain a fourth semantic feature set, and the fourth semantic feature set is sent to the base station.

[0020] Based on the same inventive concept, a fourth aspect of the exemplary embodiments of the present disclosure provides a multi-user information transmission method based on a semantic basis, applied to a base station, the method comprising:

[0021] receiving a superimposed semantic feature set sent by a plurality of user terminals;

[0022] determining a decoding order of the plurality of user terminals according to channel parameters of the plurality of user terminals;

[0023] Based on the decoding order, serial interference cancellation is performed on the superimposed semantic feature sets to obtain a plurality of semantic feature sets, and the semantic feature sets are decoded to obtain source information corresponding to the semantic feature sets.

[0024] Based on the same inventive concept, a fifth aspect of the exemplary embodiments of the present disclosure provides a multi-user information transmission device based on a semantic basis, applied to a base station, the device comprising:

[0025] A first information source information determination module is configured to determine a plurality of information source information to be sent to a plurality of user terminals;

[0026] A first semantic feature extraction module is configured to perform semantic feature extraction on each of the information sources based on a plurality of semantic feature extraction methods to obtain a first semantic feature set corresponding to each of the information sources, wherein the first semantic feature set includes a plurality of semantic features obtained based on the plurality of semantic feature extraction methods;

[0027] a first semantic feature rearrangement module configured to rearrange the plurality of semantic features in the first semantic feature set based on differences between the plurality of semantic features in the first semantic feature set to obtain a second semantic feature set;

[0028] The first semantic feature sending module is configured to superimpose several first semantic feature sets and several second semantic feature sets to obtain superimposed semantic feature sets, and send the superimposed semantic feature sets to the several user terminals.

[0029] Based on the same inventive concept, a sixth aspect of the exemplary embodiments of the present disclosure provides a multi-user information transmission device based on a semantic basis, applied to a user terminal, the device comprising:

[0030] A first semantic feature receiving module is configured to receive a superimposed semantic feature set sent by a base station;

[0031] A first semantic feature decoding module is configured to, in response to determining that the decoding order corresponding to the user terminal is ranked first, decode the superimposed semantic feature set to obtain source information corresponding to the user terminal;

[0032] The first serial interference elimination module is configured to, in response to determining that the decoding order corresponding to the user terminal is not ranked first, perform serial interference elimination on the superimposed semantic feature set to obtain the semantic feature set corresponding to the user terminal, decode the semantic feature set, and obtain the source information corresponding to the user terminal.

[0033] Based on the same inventive concept, a seventh aspect of the exemplary embodiments of the present disclosure provides a multi-user information transmission device based on a semantic basis, applied to a user terminal, the device comprising:

[0034] A second information source information determination module is configured to determine information source information to be sent to the base station;

[0035] a second semantic feature extraction module configured to perform semantic feature extraction on the source information based on a plurality of semantic feature extraction methods to obtain a third semantic feature set corresponding to the source information, wherein the third semantic feature set includes a plurality of semantic features obtained based on the plurality of semantic feature extraction methods;

[0036] a second semantic feature sending module, configured to send the third semantic feature set to the base station in response to determining that the user terminal does not participate in the reordering;

[0037] The second semantic feature rearrangement module is configured to, in response to determining that the user terminal participates in the rearrangement, rearrange the several semantic features in the third semantic feature set based on the differences between the several semantic features in the third semantic feature set to obtain a fourth semantic feature set, and send the fourth semantic feature set to the base station.

[0038] Based on the same inventive concept, an eighth aspect of the exemplary embodiments of the present disclosure provides a multi-user information transmission device based on a semantic basis, applied to a base station, the device comprising:

[0039] A second semantic feature receiving module is configured to receive a superimposed semantic feature set sent by a plurality of user terminals;

[0040] a second semantic feature decoding module, configured to determine a decoding order of the plurality of user terminals according to channel parameters of the plurality of user terminals;

[0041] The second serial interference cancellation module is configured to perform serial interference cancellation on the superimposed semantic feature sets based on the decoding order to obtain several semantic feature sets, decode the semantic feature sets, and obtain source information corresponding to the semantic feature sets.

[0042] Based on the same inventive concept, a ninth aspect of the exemplary embodiments of the present disclosure provides a multi-user information transmission system based on a semantic basis, including: a base station and a user terminal;

[0043] When the base station device is configured to execute the method according to the first aspect, the user terminal is configured to execute the method according to the second aspect;

[0044] When the user terminal is configured to execute the method as described in the third aspect, the base station device is configured to execute the method as described in the fourth aspect.

[0045] Based on the same inventive concept, the tenth aspect of the exemplary embodiment of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and run on the processor, wherein when the processor executes the program, it implements the method as described in the first aspect, the method as described in the second aspect, the method as described in the third aspect, or the method as described in the fourth aspect.

[0046] Based on the same inventive concept, the eleventh aspect of the exemplary embodiment of the present disclosure provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the method as described in the first aspect, the method as described in the second aspect, the method as described in the third aspect, or the method as described in the fourth aspect.

[0047] Based on the same inventive concept, the twelfth aspect of the exemplary embodiment of the present disclosure provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the method as described in the first aspect, the method as described in the second aspect, the method as described in the third aspect, or the method as described in the fourth aspect.

[0048] From the above description, it can be seen that the multi-user information transmission method and related device based on semantic base provided by the embodiment of the present disclosure, when applied to a base station, includes: determining a plurality of source information to be sent to a plurality of user terminals; performing semantic feature extraction on each of the source information based on a plurality of semantic feature extraction methods to obtain a first semantic feature set corresponding to each of the source information, wherein the first semantic feature set includes a plurality of semantic features obtained based on the plurality of semantic feature extraction methods; rearranging the plurality of semantic features in the plurality of first semantic feature sets based on the differences between the plurality of semantic features in the first semantic feature set to obtain a second semantic feature set; superimposing the plurality of first semantic feature sets and the plurality of second semantic feature sets to obtain a superimposed semantic feature set, and sending the superimposed semantic feature set to the plurality of user terminals. Among them, the semantic base includes the correspondence between the semantic feature extraction method and the semantic feature, and the rearranged semantic features are superimposed and transmitted with the non-rearranged semantic features, thereby suppressing the mutual interference of semantics between different users. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] FIG1 is a schematic diagram of an application scenario of a semantic-based multi-user information transmission system provided by an exemplary embodiment of the present disclosure;

[0051] FIG2 is a flow chart of a multi-user information transmission method based on semantic bases provided by an exemplary embodiment of the present disclosure;

[0052] FIG3 is a schematic diagram of an application scenario of a semantic-based multi-user information transmission method provided by an exemplary embodiment of the present disclosure;

[0053] FIG4 is a schematic diagram of a flow chart of a feature rearrangement method provided by an exemplary embodiment of the present disclosure;

[0054] FIG5 is another flowchart of a multi-user information transmission method based on semantic bases provided by an exemplary embodiment of the present disclosure;

[0055] FIG6 is another flowchart of a multi-user information transmission method based on semantic bases provided by an exemplary embodiment of the present disclosure;

[0056] FIG7 is another flowchart of a multi-user information transmission method based on semantic bases provided by an exemplary embodiment of the present disclosure;

[0057] FIG8 is another flowchart of a multi-user information transmission method based on semantic bases provided by an exemplary embodiment of the present disclosure;

[0058] FIG9 is a schematic diagram of another application scenario of a semantic-based multi-user information transmission system provided by an exemplary embodiment of the present disclosure;

[0059] FIG10 is a schematic structural diagram of a multi-user information transmission device based on semantic bases provided by an exemplary embodiment of the present disclosure;

[0060] FIG11 is another structural diagram of a multi-user information transmission device based on semantic bases provided by an exemplary embodiment of the present disclosure;

[0061] FIG12 is another structural diagram of a multi-user information transmission device based on semantic bases provided by an exemplary embodiment of the present disclosure;

[0062] FIG13 is another structural diagram of a multi-user information transmission device based on semantic bases provided by an exemplary embodiment of the present disclosure;

[0063] FIG14 is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0064] It is understandable that before using the technical solutions disclosed in the embodiments of this application, the type, scope of use, usage scenarios, etc. of the personal information involved in this application should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0065] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. Thus, based on the prompt message, the user can independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operation of the technical solution of this application.

[0066] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0067] It is understandable that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this application. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0068] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0069] To make the objectives, technical solutions, and advantages of the present disclosure more clearly understood, the principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided solely to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0070] It should be understood herein that any number of elements in the drawings is for illustration only and not for limitation, and any naming is only for distinction and does not have any limiting meaning.

[0071] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should have the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. The article "one" or "an" before an element does not exclude the presence of multiple such elements.

[0072] The principles and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure.

[0073] As described in the background art, semantic communication technology significantly improves communication efficiency by extracting and transmitting semantic information related to communication intent compared to communication technology based on grammatical information in related technologies.

[0074] However, the inventors of the present disclosure have discovered that the semantic communication schemes in related arts have the problem of semantic interference between users when facing non-orthogonal transmission of information by multiple users.

[0075] In order to solve the above problems, the present disclosure provides a multi-user information transmission scheme based on a semantic base, which, when applied to a base station, specifically includes: determining a number of source information to be sent to a number of user terminals; based on a number of semantic feature extraction methods, performing semantic feature extraction on each of the source information to obtain a first semantic feature set corresponding to each of the source information, wherein the first semantic feature set includes a number of semantic features obtained based on the several semantic feature extraction methods; based on the differences between the several semantic features in the first semantic feature set, rearranging the several semantic features in the several first semantic feature sets to obtain a second semantic feature set; superimposing the several first semantic feature sets and the several second semantic feature sets to obtain a superimposed semantic feature set, and sending the superimposed semantic feature set to the several user terminals. Wherein, the semantic base includes the correspondence between the semantic feature extraction method and the semantic feature, and the rearranged semantic features are superimposed and transmitted with the non-rearranged semantic features, thereby suppressing the mutual interference of semantics between different users.

[0076] Specifically, the serial interference cancellation methods used in existing non-orthogonal communication technologies focused on syntactic information cannot be directly applied to detect the superimposed semantic features. This is because existing modulation and demodulation methods are designed for syntactic communication and are used to form and detect discrete constellation point symbols. Semantic features, however, are typically extracted by deep neural networks and are continuous floating-point numbers. Therefore, directly estimating continuous semantic features using traditional modulation and demodulation methods results in poor detection performance, which in turn reduces the performance of serial interference cancellation. Existing non-orthogonal transmission schemes focused on semantic features generally do not design serial interference cancellation methods for semantic feature superposition. Instead, they employ a deep learning-based multi-user joint decoder to simultaneously interpret the information of multiple users. However, once trained, these decoders are only applicable to scenarios with a fixed number of users and cannot be directly extended to more users. Furthermore, they require redesign and training for each additional user, resulting in poor scalability in real-world multi-user communication scenarios. Furthermore, these works only address multi-user interference at the syntactic level, that is, direct interference at the symbol level, while ignoring the interference at the semantic level of the superimposed symbols. Due to the continuity of semantic features, interference cannot be perfectly eliminated at the grammatical level. The residual semantic interference will affect the performance of semantic level decoding. The above works have not considered this problem, resulting in a decrease in the efficiency of non-orthogonal semantic transmission.

[0077] In this disclosure, a non-orthogonal transmission scheme for multi-user semantic information based on semantic bases is proposed. Based on the semantic bases, an efficient joint semantic channel encoding and decoding method and transmission strategy are implemented to carry multiple semantic features for simultaneous and co-frequency transmission, thereby improving the efficiency of semantic information transmission between the base station and multiple semantic users. The base station uses a joint semantic channel encoder to extract multiple semantic features and robustly encode them, transmitting them to multiple semantic users in a non-orthogonal manner. A joint semantic channel decoder is used to reconstruct source information based on the semantic features. To reduce semantic interference between users in non-orthogonal transmission, the present invention proposes a semantic feature rearrangement method based on semantic base differences. By calculating the distance between semantic bases of different users at the base station side, features with large semantic base differences are superimposed and transmitted. This allows the joint semantic channel decoder at each semantic user to only detect semantic features that match the decoded semantic base. Semantic features of other users that cause interference cannot be interpreted by the decoder that identifies the corresponding features due to the different semantics they carry, thereby suppressing semantic interference at the transmitting end. On the semantic user side of the receiving end, the present invention proposes a serial interference elimination method based on semantic basis differences, which uses semantic basis to decode and regenerate the semantic features that cause interference, and at the same time uses semantic basis differences to suppress the expression of other semantic user features, so that the interference can be more accurately estimated and subtracted from the received signal, further reducing semantic interference, improving the decoding accuracy of the target semantic features, and further improving the overall semantic feature transmission efficiency of the multi-user semantic communication system.

[0078] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

[0079] Refer to FIG1 , which is a schematic diagram of an application scenario of a semantic-based multi-user information transmission system provided by an exemplary embodiment of the present disclosure.

[0080] This application scenario includes a base station and several user terminals (semantic users). This application scenario is a downlink scenario, in which the base station is the transmitter and the user terminal is the receiver.

[0081] The base station and the user terminal can be connected through a wired or wireless communication network to achieve data interaction.

[0082] A user terminal may be an electronic device located near a user and having data transmission and multimedia input / output functions, including but not limited to a desktop computer, a mobile phone, a mobile computer, a tablet computer, a media player, a smart wearable device, a personal digital assistant (PDA), or other electronic devices capable of implementing the above functions. The electronic device may include a processor and a display screen with a touch input function, the display screen being used to present a graphical user interface (GUI), which may display an application interface, and the processor being used to process application data, generate the GUI, and control the display of the GUI on the display screen.

[0083] When the multi-user information transmission method based on semantic bases is run on a base station, the base station is used to provide a multi-user information transmission service based on semantic bases to users of user terminals. The base station determines a plurality of source information to be sent to a plurality of user terminals; based on a plurality of semantic feature extraction methods, performs semantic feature extraction on each of the source information to obtain a first semantic feature set corresponding to each of the source information, wherein the first semantic feature set includes a plurality of semantic features obtained based on the plurality of semantic feature extraction methods; based on the differences between the plurality of semantic features in the first semantic feature set, rearranges the plurality of semantic features in the plurality of first semantic feature sets to obtain a second semantic feature set; superimposes the plurality of first semantic feature sets and the plurality of second semantic feature sets to obtain a superimposed semantic feature set, and sends the superimposed semantic feature set to the plurality of user terminals.

[0084] In some exemplary embodiments, a user terminal receives a superimposed semantic feature set transmitted by a base station; in response to determining that the decoding order corresponding to the user terminal is ranked first, the superimposed semantic feature set is decoded to obtain information source information corresponding to the user terminal; or, in response to determining that the decoding order corresponding to the user terminal is not ranked first, serial interference cancellation is performed on the superimposed semantic feature set to obtain a semantic feature set corresponding to the user terminal, and the semantic feature set is decoded to obtain information source information corresponding to the user terminal. The user terminal is installed with a client for communicating with the base station, and the client displays the information source information to the user.

[0085] The following describes a semantic-based multi-user information transmission method according to an exemplary embodiment of the present disclosure, using the application scenario of Figure 1. It should be noted that the above application scenario is provided solely to facilitate understanding of the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. Rather, the embodiments of the present disclosure can be applied to any applicable scenario.

[0086] Next, a multi-user information transmission method based on semantic basis implemented in a downlink scenario will be described with reference to FIG1 , wherein the base station is a transmitting end and the user terminal is a receiving end.

[0087] 2 , when the base station is the transmitting end, the semantic-based multi-user information transmission method implemented by the base station includes the following steps:

[0088] Step S210: Determine a plurality of information sources to be sent to a plurality of user terminals.

[0089] In some exemplary embodiments, there are several user terminals, and based on the distances between the user terminals and the base station, the user terminals are sorted in ascending order of distance.

[0090] As a specific example, assume that there are N user terminals, and the user terminals are sorted in ascending order of distance, and user terminal 1, user terminal 2, ..., user terminal N are obtained. Then there is d1 <d2<…<d N , where d i represents the distance between the i-th user terminal and the base station.

[0091] In specific implementation, since the user terminal is used to implement the multi-user information transmission method based on the semantic basis, the user terminal is also referred to as a semantic user in this disclosure.

[0092] In some exemplary embodiments, determining the plurality of information sources to be sent to the plurality of user terminals includes:

[0093] The information source information to be sent to different user terminals is different.

[0094] In a specific implementation, there is no specific association between the source information to be sent to different user terminals, that is, the source information to be sent to different user terminals is independent of each other. In this case, the source information to be sent to different user terminals can be the same or different, but usually they are different.

[0095] In some exemplary embodiments, the information source information includes at least one of the following:

[0096] Text source information, voice source information, image source information, and digital source information.

[0097] In practical applications, a source is the entity that generates various types of information. The symbols provided by the source are uncertain and can be described by random variables and their statistical properties. Information is abstract, while the source is concrete. For example, when people converse, the human vocal system is the speech source; when people read books or newspapers, the illuminated books and newspapers themselves are the text source. Other common sources include image sources and digital sources.

[0098] Step S220: Based on several semantic feature extraction methods, semantic feature extraction is performed on each of the source information to obtain a first semantic feature set corresponding to each of the source information, wherein the first semantic feature set includes several semantic features obtained based on the several semantic feature extraction methods.

[0099] In some exemplary embodiments, the semantic feature extraction method includes an encoding function.

[0100] The method comprises: performing semantic feature extraction on each of the source information based on several semantic feature extraction methods to obtain a first semantic feature set corresponding to each of the source information, wherein the first semantic feature set includes several semantic features obtained based on the several semantic feature extraction methods, including: performing semantic feature extraction on each of the source information based on several encoding functions to obtain the first semantic feature set corresponding to each of the source information, wherein the first semantic feature set includes several semantic features obtained based on several encoding functions.

[0101] In some exemplary embodiments, the first semantic feature set includes several semantic features obtained based on the several semantic feature extraction methods, including: the semantic features obtained based on different semantic feature extraction methods are different.

[0102] As a specific example, the module that implements semantic feature extraction is called a joint semantic channel encoder. In practice, the joint semantic channel encoder is implemented using a deep neural network. The network structure can be flexibly adjusted based on the specific source modality and transmission requirements, such as a transformer structure for text transmission or a convolutional neural network structure for image transmission.

[0103] In the embodiment of the present disclosure, the semantic base includes the correspondence between the semantic feature extraction method and the semantic features. The above correspondence can specifically refer to the relationship between the feature extraction function and the extracted semantic features. The establishment method is to use a large amount of data to train the neural network offline so that it can learn which features to extract from the original data to complete data reconstruction, and what is the extraction function corresponding to each feature. Therefore, in the embodiment of the present disclosure, in addition to including the extracted semantic features, the semantic base also clarifies the correspondence between this feature extraction function and the features.

[0104] In specific implementation, the joint semantic channel encoder contains multiple encoding functions for extracting multiple semantic features related to the reconstruction of the original information from the information to be transmitted. The process is expressed as:

[0105] Among them, s i is the original information of user i, x i is the semantic feature of user i. represents the transfer function of the joint semantic channel encoder, α is the set of encoder neural network parameters. Corresponding to multiple encoding functions in the joint semantic channel encoder, x i =[x i,1 ,…,x i,C′ Each semantic feature x in i,c It also needs to be decoded by the corresponding decoding function at the receiving end. The semantic base contains the correspondence between this encoding and decoding function and the semantic features.

[0106] Step S230: Based on the differences between the plurality of semantic features in the first semantic feature set, rearrange the plurality of semantic features in the first semantic feature set to obtain a second semantic feature set.

[0107] In some exemplary embodiments, the rearranging the plurality of semantic features in the first semantic feature set based on the differences between the plurality of semantic features in the first semantic feature set to obtain the second semantic feature set includes:

[0108] determining difference parameters between the plurality of semantic features in the first semantic feature set, and obtaining a difference parameter matrix based on the difference parameters;

[0109] Based on the difference parameter matrix, pairing is performed with the purpose of maximizing the difference parameter between the two semantic features, thereby obtaining the serial numbers of the paired semantic features;

[0110] Obtaining a rearranged sequence of sequence numbers based on the sequence numbers of the paired semantic features;

[0111] Based on the rearranged sequence of sequence numbers, the order of the plurality of semantic features in the first semantic feature set to be rearranged is rearranged to obtain the second semantic feature set.

[0112] In some exemplary embodiments, the step of rearranging the order of the plurality of semantic features in the first semantic feature set to be rearranged based on the rearranged sequence of sequence numbers to obtain the second semantic feature set includes:

[0113] Determining sequence numbers of the plurality of semantic features in the first semantic feature set to be rearranged;

[0114] Based on the rearranged sequence of sequence numbers, the order of the plurality of semantic features is rearranged to obtain the second semantic feature set.

[0115] In some exemplary embodiments, the rearranging the plurality of semantic features in the first semantic feature set based on the differences between the plurality of semantic features in the first semantic feature set to obtain the second semantic feature set includes:

[0116] For the N first semantic feature sets:

[0117] Rearranging the plurality of semantic features in the N-1th first semantic feature set based on differences between the plurality of semantic features in the Nth first semantic feature set to obtain a first second semantic feature set;

[0118] Rearranging the plurality of semantic features in the N-3th first semantic feature set based on differences between the plurality of semantic features in the N-2th first semantic feature set to obtain a second second semantic feature set;

[0119] Repeat the above steps until the rearrangement is complete.

[0120] Referring to Figure 3, in a specific implementation, after extracting the semantic features of N user original information, the base station first evaluates the semantic basis differences. Since the N users share the same joint semantic channel encoder, the correspondence between the extracted semantic features and the encoding function is the same. That is, under the same type of source information, the semantic basis does not change with changes in the source information. Therefore, the evaluation of semantic basis differences can be completed based on the semantic features of a single user, and the resulting semantic basis differences can be used to reorder the semantic features of multiple users. For example, the semantic features of semantic user 1 can be used to evaluate the semantic basis differences. In actual use, the semantic features of other users can also be selected to complete this process.

[0121] Referring to FIG4 , as a specific example, the rearrangement process specifically includes the following steps:

[0122] Step 1: Calculation and The distance between c1, c2 = {1, ..., C'}, and form a semantic feature rearrangement matrix D of C' × C', expressed as:

[0123] DStep 2: Select a pair of features with the largest distance from D and record the corresponding channel numbers.

[0124] Step 3: Delete the rows and columns containing the selected feature pairs from D.

[0125] Step 4: Repeat steps 2 and 3 until all features are matched.

[0126] After completing the above steps, given the channel number of the semantic feature {1,…,C′}, the feature channel number of the pairing based on the largest possible semantic base difference is c r ={c1,…,c c For a user’s semantic feature x i =[x i,1 ,…,x i,C′ ], and the semantic features of other users are ranked by c r Rearrange it so that x i The feature of each channel is paired with a feature whose semantic basis is as different as possible.

[0127] For semantic users 1,…,N, the reordering of semantic features will be performed alternately among semantic users, i.e., given x N , rearrange x N-1 ; Given x N-2 , rearrange x N-3 ; until x1. The rearranged semantic features are expressed as Among them, r indicates whether rearrangement has occurred. If r = 0, it means that rearrangement has not occurred. Otherwise, it means that rearrangement has occurred. The rules followed are: for semantic features

[0128] r i =0, if N is an even number, i is an even number; if N is an odd number, i is an odd number;

[0129] r i =1, if N is odd, i is even; if N is even, i is odd.

[0130] Step S240: Superimpose the plurality of first semantic feature sets and the plurality of second semantic feature sets to obtain a superimposed semantic feature set, and send the superimposed semantic feature set to the plurality of user terminals.

[0131] In some exemplary embodiments, superimposing the plurality of first semantic feature sets and the plurality of second semantic feature sets to obtain a superimposed semantic feature set, and sending the superimposed semantic feature set to the plurality of user terminals includes:

[0132] determining a decoding order of the plurality of user terminals according to channel parameters of the plurality of user terminals;

[0133] Allocating transmission power to the first semantic feature set and the second semantic feature set based on the decoding order;

[0134] Superimposing a plurality of the first semantic feature sets and a plurality of the second semantic feature sets based on the transmission power to obtain the superimposed semantic feature sets;

[0135] Based on the transmission power, the superimposed semantic feature set is sent to the plurality of user terminals.

[0136] In practice, semantic interference can be suppressed by superimposing the rearranged features with the unrearranged semantic features. Whether to reorder and the original order before reordering are communicated to each semantic user via other links, and reliable transmission is assumed.

[0137] The method described in the embodiment of the present disclosure will be described below using a specific example. The semantic features of each user are first power-normalized, and the access point divides the total power into N parts and allocates them to N users respectively. In order to achieve efficient serial interference cancellation at the receiving end, the base station determines the decoding order of the receiving end according to the channel conditions of each user, and allocates different powers to the information of different users accordingly. The channel gain from semantic user i to the base station is expressed as This includes the path loss PL(d i ) and small-scale fading g iThe path loss is specifically modeled as: Wherein, ρ0 is the path loss when the reference distance d0 = 1m, and l is the path loss factor.

[0138] Small-scale fading is modeled as Rice distribution, assuming that the small-scale fading from all semantic users to the base station is the same. From the above, we can see that the greater the distance between the semantic user and the base station, the greater the path loss and the smaller the channel gain. <d2<…<d N Under the assumption that |h1| 2 >|h2| 2 >…>|h N | 2 . Define an auxiliary item to indicate the decoding order of the receiving end, expressed as Among them, σ 2 It is usually assumed that the noise power at each user is the same, so the relationship between the auxiliary items of the decoding order is G1>G2>…>G N .G i The smaller the number, the earlier the decoding order. Therefore, the decoding order of each user at the receiving end is π=[N,…,1]. The earlier the user number is, the earlier it will be decoded. This order will be synchronized to each user by the base station. Based on the decoding order list π, the power allocated to different semantic users satisfies P1 <P2<…<P N , so the signal transmitted by the base station is expressed as:

[0139] It should be noted that for the semantic feature x N For example, although x N-2 No feature re-ranking based on semantic base differences was performed, but due to the low power assigned to this feature, the feature from x N-2 The semantic interference of the unrearranged semantic features such as x1 is small, and the semantic interference from x N-1 The semantic interference of is still the main interference. The proposed feature rearrangement method based on semantic base difference can be used to suppress the main interference and improve x N The decoding performance of other semantic features can be improved.

[0140] In the downlink scenario, the base station is the transmitter and the user terminal is the receiver. In the above exemplary embodiment, a multi-user information transmission method based on a semantic basis implemented by the base station when the base station is the transmitter is introduced. Next, a multi-user information transmission method based on a semantic basis implemented by the user terminal when the user terminal is the receiver will be introduced.

[0141] 5 , when a user terminal is a receiving end, a semantic-based multi-user information transmission method implemented by the user terminal includes the following steps:

[0142] Step S510: Receive a superimposed semantic feature set sent by a base station.

[0143] In specific implementation: At the receiving end, the signal received by each semantic user is expressed as: y1 = h1x + z1, ..., y N =h N x+z N , where z i is Gaussian white noise.

[0144] Step S520 : In response to determining that the decoding order corresponding to the user terminal ranks first, decoding the superimposed semantic feature set to obtain information source information corresponding to the user terminal.

[0145] Step S530: In response to determining that the decoding order corresponding to the user terminal is not ranked first, serial interference elimination is performed on the superimposed semantic feature set to obtain the semantic feature set corresponding to the user terminal, and the semantic feature set is decoded to obtain the source information corresponding to the user terminal.

[0146] In some exemplary embodiments, decoding the semantic feature set to obtain information source information corresponding to the user terminal includes:

[0147] In response to determining that the semantic feature set has been rearranged, restoring the semantic feature set to obtain a restored semantic feature set;

[0148] The restored semantic feature set is decoded to obtain the information source information corresponding to the user terminal.

[0149] As a specific example:

[0150] N users adopt different signal detection strategies. Semantic user N has the worst channel conditions, but receives the highest transmission power. Therefore, the interference from the symbols of the other N-1 semantic users can be treated as noise, and the joint semantic channel decoder can be directly used to interpret its own information. The joint semantic channel decoder is also composed of a deep neural network. Similar to the joint semantic encoder, the specific neural network structure is determined by the source modality and transmission requirements. The decoding process is expressed as:

[0151] in, It represents the transfer function of the joint semantic channel decoder, and β is the set of decoder neural network parameters. The channel condition of semantic user N-1 is better than that of semantic user N, and it is allocated the second highest power, so it is strongly interfered by semantic user N. First, the serial interference cancellation method needs to be adopted to eliminate the interference of semantic user N, and then the decoding of its own symbols is completed. The symbols of the remaining N-2 semantic users are regarded as noise. Similarly, the remaining semantic users sequentially eliminate the semantic interference of semantic users with higher power and then perform the decoding of their own symbols. However, due to the continuity of semantic features, the demodulation-re-modulation-cancellation idea adopted by the existing serial interference cancellation method cannot be implemented. Therefore, this scheme realizes the serial interference cancellation for the semantic features of the source information based on the semantic basis, uses the semantic basis difference to interpret and estimate the interference and the target semantic features, and suppresses the expression of other non-target semantic features during this process, so as to improve the decoding accuracy of the semantic features to be decoded and further achieve the elimination of interference. At semantic user i, 1≤i<N, the serial interference cancellation process for the semantic features of the source information is shown in Figure 6, specifically as follows:

[0152] Step 1: Detect the received signal y i , and obtain where j is the serial number of the user with the earliest position in the current decoding order list;

[0153] Step 2: Determine whether user i is the first user to be decoded. If so, directly decode; otherwise, the decoding of user j needs to be completed first;

[0154] Step 3: Judge whether it has undergone feature rearrangement based on semantic basis difference at the transmitting end. If r j = 1, first restore to the original order, that is and then use the joint semantic channel decoder to reconstruct the source information sent to user j, that is

[0155] Step 4: Use the joint semantic channel encoder to map the reconstructed information of user j into semantic features to estimate the interference from user j; if r j = 1, rearrangement also needs to be performed in this step, expressed as

[0156] Step 5: Subtract from the received signal y i to obtain:

[0157] Step 6: Delete user j from the decoding order list and update it to the first user number in the current list. If user i is not the first user to be decoded, continue with steps 3 to 5. Otherwise, decode the symbol of user i.

[0158] Step 7: If r i =1, then first Restore to the original order, i.e. based on The information sent to user i is reconstructed using the joint semantic channel decoder, i.e.

[0159] In summary, the specific transmission process in the downlink scenario is as follows:

[0160] Step 1: The base station uses a joint semantic channel encoder to extract semantic features from the source information of each semantic user;

[0161] Step 2: The base station uses the semantic feature rearrangement module based on semantic base differences to evaluate the semantic base differences based on the semantic features of individual semantic users, rearranges the features of some semantic users accordingly, and records the semantic feature rearrangement information;

[0162] Step 3: The decoding order at the receiving end is determined based on the channel state of the semantic user. Different powers are allocated to the semantic features of each user, and the superimposed signals are transmitted to each user. Simultaneously, the semantic feature reordering information and the decoding order are reliably transmitted to the user using other links.

[0163] Step 4: Semantic users rearrange information and decoding order based on semantic features, perform serial interference elimination based on semantic base differences, and obtain their own semantic features;

[0164] Step 5: The semantic user uses the joint semantic channel decoder to recover the source information based on the separated semantic features.

[0165] In the above exemplary embodiment, a semantic-based multi-user information transmission method was described for a downlink scenario, where the base station was the transmitter and the user terminal was the receiver. Next, a semantic-based multi-user information transmission method for an uplink scenario will be described, where the user terminal was the transmitter and the base station was the receiver.

[0166] 7 , when a user terminal is a transmitting end, a semantic-based multi-user information transmission method implemented by the user terminal includes the following steps:

[0167] Step S710: Determine the information source to be sent to the base station.

[0168] Step S720: Based on several semantic feature extraction methods, semantic feature extraction is performed on the source information to obtain a third semantic feature set corresponding to the source information, wherein the third semantic feature set includes several semantic features obtained based on the several semantic feature extraction methods.

[0169] Step S730: In response to determining that the user terminal does not participate in the re-arrangement, send the third semantic feature set to the base station.

[0170] Step S740: In response to determining that the user terminal participates in the rearrangement, based on the differences between the several semantic features in the third semantic feature set, the several semantic features in the third semantic feature set are rearranged to obtain a fourth semantic feature set, and the fourth semantic feature set is sent to the base station.

[0171] 8 , when the base station is a receiving end, the semantic-based multi-user information transmission method implemented by the base station includes the following steps:

[0172] Step S810: Receive a superimposed semantic feature set sent by several user terminals.

[0173] Step S820: Determine a decoding order of the user terminals according to the channel parameters of the user terminals.

[0174] Step S830: Based on the decoding order, perform serial interference cancellation on the superimposed semantic feature sets to obtain a plurality of semantic feature sets, decode the semantic feature sets, and obtain source information corresponding to the semantic feature sets.

[0175] As a specific example, the present disclosure adopts the system framework shown in FIG9 to implement uplink information transmission based on semantic base, and the distance between the user and the base station satisfies, d1 <d2<…<d N , where a joint semantic channel encoder and a semantic feature reordering module based on semantic base differences are deployed at each semantic user on the transmitter side. The specific semantic information encoding process is the same as that in the downlink transmission scenario, and both use the joint semantic channel encoder to extract the semantic features of the source information x i The base station determines the sequence number of semantic users that need to participate in the rearrangement based on the number of users N. The specific rules are the same as those used in the downlink scenario, and the corresponding users are notified before the uplink communication. Since the semantic base does not change with the change of the source information under the same type of source, the evaluation of the semantic base difference can be completed based on the semantic features of a single user. Therefore, the semantic users participating in the rearrangement can evaluate the differences between the semantic bases based on their own semantic features and complete the rearrangement of their own semantic information accordingly. The specific method of rearrangement is similar to that of the downlink scenario, as shown in Figure 9. The semantic feature reordering information of the users participating in the reordering is represented as follows: During the uplink transmission process, each user uses its own power to process the rearranged semantic features and the un-rearranged semantic features through power constraints and transmits them. The semantic features of each user are superimposed into one information flow in the wireless channel. At the same time, the rearranged information It is also synchronized to the base station via other links, and assumes orthogonal and reliable transmission.

[0176] At the base station side of the receiving end, according to the channel conditions of each semantic user |h1| 2 >|h2| 2 >…>|h N | 2 The corresponding decoding order auxiliary items are G1>G2>…>G N Unlike the downlink scenario, in the uplink scenario, the receiver must first decode the users with better channel conditions and eliminate them from the received signal to ensure that the channels of users with poor channel quality can be correctly decoded. Therefore, the decoding order in the uplink scenario is π = [N,…,1], which is determined by the base station and does not need to be synchronized with each user. The signal received by the base station is represented as:

[0177] Semantic feature reordering information based on decoding order list π and user synchronization The base station can use the semantic basis difference-based serial interference cancellation method shown in Figure 6 to iteratively separate the semantic features of each semantic user, and use the joint semantic channel decoder to complete the reconstruction of the corresponding source information.

[0178] In summary, the specific transmission process in the uplink scenario is as follows:

[0179] Step 1: The base station determines the sequence number of the semantic user that needs to participate in the reordering and notifies the corresponding semantic user;

[0180] Step 2: Each semantic user uses a joint semantic channel encoder to extract semantic features from the source information;

[0181] Step 3: Semantic users who need to participate in the rearrangement use the semantic feature rearrangement module based on semantic base differences to evaluate the semantic base differences between their own semantic features, rearrange the features accordingly, and record the semantic feature rearrangement information;

[0182] Step 4: Each user transmits the semantic features using its own power. At the same time, the semantic feature rearrangement information is reliably transmitted to the base station using other links.

[0183] Step 5: The base station determines the decoding order based on the channel status of each user, reorders the information based on the semantic features, and performs a serial interference cancellation process based on semantic basis differences to obtain the semantic features of each user;

[0184] Step 6: The base station uses a joint semantic channel decoder to recover the source information based on the separated semantic features.

[0185] The proposed non-orthogonal semantic information transmission method based on semantic basis differences is implemented using a neural network and requires offline training based on a large amount of data. This allows the joint semantic channel codec to learn an efficient and robust semantic feature encoding and decoding strategy for non-orthogonal transmission. The loss function used in training is the weighted sum of the mean absolute error of the reconstruction of all semantic user source information, expressed as: in,

[0186] The joint semantic channel codec is trained end-to-end, incorporating semantic feature reordering based on semantic base differences and serial interference cancellation based on semantic base differences. This allows the joint semantic channel codec to more directly understand the differences between semantic features and the interference between users, thereby learning more efficient, comprehensive, and robust semantic feature extraction methods and decoding strategies. Once training is complete, the joint semantic channel codec is deployed at the base station and at each user. The semantic feature reordering module based on semantic base differences does not require training; it only needs to be deployed at the base station and semantic users.

[0187] In the present disclosure, a non-orthogonal communication method oriented to the semantic features of multi-user sources is proposed, which significantly improves the efficiency of multi-user semantic transmission.

[0188] In this disclosure, a semantic feature rearrangement method based on semantic base differences is proposed, which effectively suppresses semantic interference among multiple semantic users.

[0189] In the present disclosure, a serial interference elimination method based on semantic basis difference is proposed to further reduce the semantic interference between multiple semantic users.

[0190] Compared with other non-orthogonal semantic communication design schemes, the present disclosure designs a semantic feature rearrangement method based on semantic basis differences and a serial interference elimination method based on semantic basis differences.

[0191] First of all, the semantic-based non-orthogonal source semantic feature encoding and decoding method proposed in the present invention can achieve efficient and robust semantic feature extraction and interpretation under semantic interference, and only needs to train one set of encoders and decoders for multiple users. It is also suitable for point-to-point communication and multi-user non-orthogonal communication, and has stronger scalability.

[0192] Secondly, the semantic feature rearrangement method based on semantic base differences proposed in this disclosure effectively suppresses the semantic interference between multi-user semantic features by utilizing the differences between semantic bases to construct semantic decoding dislocations, thereby improving the reconstruction quality of source information and the semantic transmission efficiency of multi-user semantic communication systems.

[0193] Finally, the serial interference elimination method based on semantic base differences proposed in the present invention can realize serial elimination of semantic interference between semantic features based on the continuous characteristics of semantic features, utilizing the semantic encoding and decoding functions of the semantic bases and the differences between semantic bases, and provide effective support for multi-user detection in non-orthogonal semantic communication systems.

[0194] It should be noted that the method of the embodiments of the present disclosure can be performed by a single device, such as a computer or server. The method of the embodiments of the present disclosure can also be applied in a distributed scenario, where multiple devices cooperate to perform the method. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiments of the present disclosure, and the multiple devices will interact with each other to complete the method.

[0195] It should be noted that the above description is limited to some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0196] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides a multi-user information transmission device based on semantic bases.

[0197] Referring to FIG10 , a multi-user information transmission device based on semantic bases is applied to a base station. The device includes the following modules:

[0198] A first information source information determination module 1010 is configured to determine a plurality of information source information to be sent to a plurality of user terminals;

[0199] A first semantic feature extraction module 1020 is configured to perform semantic feature extraction on each source information based on a plurality of semantic feature extraction methods to obtain a first semantic feature set corresponding to each source information, wherein the first semantic feature set includes a plurality of semantic features obtained based on the plurality of semantic feature extraction methods;

[0200] A first semantic feature rearrangement module 1030 is configured to rearrange the plurality of semantic features in the first semantic feature set based on differences between the plurality of semantic features in the first semantic feature set to obtain a second semantic feature set;

[0201] The first semantic feature sending module 1040 is configured to superimpose several first semantic feature sets and several second semantic feature sets to obtain superimposed semantic feature sets, and send the superimposed semantic feature sets to the several user terminals.

[0202] In some exemplary embodiments, the first semantic feature rearrangement module 1030 is specifically configured to:

[0203] For the N first semantic feature sets, sort them in ascending order based on the distances between the user terminals and the base stations corresponding to the first semantic feature sets;

[0204] Rearranging the plurality of semantic features in the N-1th first semantic feature set based on differences between the plurality of semantic features in the Nth first semantic feature set to obtain a first second semantic feature set;

[0205] Rearranging the plurality of semantic features in the N-3th first semantic feature set based on differences between the plurality of semantic features in the N-2th first semantic feature set to obtain a second second semantic feature set;

[0206] The above steps are repeated until there is no more first semantic feature sets to be rearranged, thereby obtaining a plurality of second semantic feature sets.

[0207] In some exemplary embodiments, the first semantic feature rearrangement module 1030 is specifically configured to:

[0208] Determining a sequence number of each of the semantic features in the first semantic feature set;

[0209] determining a difference parameter between every two semantic features in the first semantic feature set;

[0210] Pairing the two semantic features with the goal of maximizing the difference parameter between the two semantic features, and determining the sequence numbers of the paired semantic features;

[0211] Based on the serial numbers of the paired semantic features, rearrange the serial number sequence composed of the serial numbers of each semantic feature in the first semantic feature set to obtain a rearranged serial number sequence;

[0212] Based on the rearranged sequence of sequence numbers, the order of the plurality of semantic features in the first semantic feature set to be rearranged is rearranged to obtain the second semantic feature set.

[0213] In some exemplary embodiments, the first semantic feature rearrangement module 1030 is specifically configured to:

[0214] Determining a sequence number of each of the plurality of semantic features in the first semantic feature set to be rearranged;

[0215] Based on the rearranged sequence of sequence numbers, the order of the plurality of semantic features in the first semantic feature set to be rearranged is rearranged to obtain the second semantic feature set.

[0216] In some exemplary embodiments, the first semantic feature sending module 1040 is specifically configured to:

[0217] determining a decoding order for the plurality of user terminals based on channel parameters of the plurality of user terminals;

[0218] Allocating transmission power to the first semantic feature set and the second semantic feature set based on the decoding order;

[0219] Superimposing a plurality of the first semantic feature sets and a plurality of the second semantic feature sets based on the transmission power to obtain the superimposed semantic feature sets;

[0220] Based on the transmission power, the superimposed semantic feature set is sent to the plurality of user terminals.

[0221] In some exemplary embodiments, the information source information to be sent to different user terminals is different.

[0222] In some exemplary embodiments, the semantic feature extraction method includes an encoding function, and the first semantic feature extraction module 1020 is specifically configured to:

[0223] Based on the plurality of encoding functions, semantic features are extracted for each of the information source information to obtain the first semantic feature set corresponding to each of the information source information, wherein the first semantic feature set includes the plurality of semantic features obtained based on the plurality of encoding functions.

[0224] In some exemplary embodiments, the semantic features obtained based on different semantic feature extraction methods are different.

[0225] Referring to FIG11 , a multi-user information transmission device based on semantic base is applied to a user terminal. The device includes the following modules:

[0226] The first semantic feature receiving module 1110 is configured to receive a superimposed semantic feature set sent by a base station;

[0227] A first semantic feature decoding module 1120 is configured to, in response to determining that the decoding order corresponding to the user terminal is ranked first, decode the superimposed semantic feature set to obtain information source information corresponding to the user terminal;

[0228] The first serial interference elimination module 1130 is configured to, in response to determining that the decoding order corresponding to the user terminal is not ranked first, perform serial interference elimination on the superimposed semantic feature set to obtain the semantic feature set corresponding to the user terminal, decode the semantic feature set, and obtain the source information corresponding to the user terminal.

[0229] In some exemplary embodiments, the first serial interference cancellation module 1130 is specifically configured to:

[0230] In response to determining that the semantic feature set has been rearranged, restoring the semantic feature set to obtain a restored semantic feature set;

[0231] The restored semantic feature set is decoded to obtain the information source information corresponding to the user terminal.

[0232] 12 , a multi-user information transmission device based on semantic base is applied to a user terminal. The device includes the following modules:

[0233] The second information source information determination module 1210 is configured to determine information source information to be sent to the base station;

[0234] The second semantic feature extraction module 1220 is configured to perform semantic feature extraction on the source information based on a plurality of semantic feature extraction methods to obtain a third semantic feature set corresponding to the source information, wherein the third semantic feature set includes a plurality of semantic features obtained based on the plurality of semantic feature extraction methods;

[0235] The second semantic feature sending module 1230 is configured to send the third semantic feature set to the base station in response to determining that the user terminal does not participate in the reordering;

[0236] The second semantic feature rearrangement module 1240 is configured to, in response to determining that the user terminal participates in the rearrangement, rearrange the several semantic features in the third semantic feature set based on the differences between the several semantic features in the third semantic feature set to obtain a fourth semantic feature set, and send the fourth semantic feature set to the base station.

[0237] Referring to FIG13 , a multi-user information transmission device based on semantic base is applied to a base station. The device includes the following modules:

[0238] The second semantic feature receiving module 1310 is configured to receive a superimposed semantic feature set sent by a plurality of user terminals;

[0239] A second semantic feature decoding module 1320 is configured to determine a decoding order of the plurality of user terminals according to the channel parameters of the plurality of user terminals;

[0240] The second serial interference cancellation module 1330 is configured to perform serial interference cancellation on the superimposed semantic feature sets based on the decoding order to obtain several semantic feature sets, decode the semantic feature sets, and obtain source information corresponding to the semantic feature sets.

[0241] For the convenience of description, the above devices are described as being functionally divided into various modules. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0242] The apparatus of the above embodiment is used to implement the corresponding semantic-based multi-user information transmission method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0243] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the semantic-based multi-user information transmission method described in any of the above embodiments is implemented.

[0244] FIG14 shows a more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment. The device may include: a processor 1410, a memory 1420, an input / output interface 1430, a communication interface 1440, and a bus 1450. The processor 1410, the memory 1420, the input / output interface 1430, and the communication interface 1440 are communicatively connected to each other within the device via the bus 1450.

[0245] The processor 1410 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0246] The memory 1420 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1420 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1420 and is called and executed by the processor 1410.

[0247] The input / output interface 1430 is used to connect an input / output module to implement information input and output. The input / output module can be configured as a component in the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0248] The communication interface 1440 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0249] The bus 1450 comprises a pathway for transmitting information between the various components of the device (eg, the processor 1410 , the memory 1420 , the input / output interface 1430 , and the communication interface 1440 ).

[0250] It should be noted that although the above device only shows the processor 1410, the memory 1420, the input / output interface 1430, the communication interface 1440, and the bus 1450, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0251] The electronic device of the above embodiment is used to implement the corresponding semantic-based multi-user information transmission method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0252] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the semantic-based multi-user information transmission method described in any of the above embodiments.

[0253] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0254] The above-mentioned non-transitory computer-readable storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (such as ROMs, EPROMs, EEPROMs, non-volatile memories (NAND FLASH), solid-state drives (SSDs)), etc.

[0255] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the semantic-based multi-user information transmission method described in any embodiment in the above exemplary method part, and have the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0256] Based on the same inventive concept, corresponding to the semantic-based multi-user information transmission method described in any of the above embodiments, the present disclosure further provides a computer program product comprising computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processors to perform the semantic-based multi-user information transmission method. For the execution entities corresponding to the steps in each embodiment of the semantic-based multi-user information transmission method, the processors executing the corresponding steps can belong to the corresponding execution entities.

[0257] The computer program product of the above embodiment is used to enable the computer and / or the processor to execute the semantic-based multi-user information transmission method as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0258] Those skilled in the art will appreciate that embodiments of the present disclosure may be implemented as a system, method, or computer program product. Therefore, the present disclosure may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the present disclosure may also be implemented in the form of a computer program product in one or more computer-readable media containing computer-readable program code.

[0259] Any combination of one or more computer-readable media can be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive examples) of computer-readable storage media can include, for example: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0260] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0261] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0262] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0263] It should be understood that each block in the flowchart and / or block diagram, and combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine. These computer program instructions are executed by the computer or other programmable data processing device to produce a device that implements the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0264] These computer program instructions can also be stored in a computer-readable medium that enables a computer or other programmable data processing device to operate in a specific manner. In this way, the instructions stored in the computer-readable medium produce a product that includes an instruction device that implements the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0265] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, such that the instructions executed on the computer or other programmable apparatus provide a process that implements the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0266] Furthermore, although the operations of the disclosed method are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in that particular order, or that all of the operations shown must be performed to achieve the desired results. Rather, the steps depicted in the flowcharts may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into a single step, and / or a single step may be broken down into multiple steps.

[0267] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0268] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the application, the features and functions of two or more modules or units described above can be concretized in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0269] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. Within the scope of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0270] In addition, for simplicity of description and discussion, and in order not to make the embodiment of the application difficult to understand, the known power supply / ground connection with integrated circuit (IC) chip and other components may or may not be shown in the accompanying drawings provided. In addition, the device can be shown in the form of a block diagram to avoid making the embodiment of the application difficult to understand, and this also takes into account the following fact, that is, the details of the embodiment of these block diagram devices are highly dependent on the platform to be implemented in the embodiment of the application (that is, these details should be fully within the scope of understanding of those skilled in the art). When specific details (for example, circuit) are set forth to describe exemplary embodiments of the application, it will be apparent to those skilled in the art that the embodiment of the application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0271] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the embodiments discussed.

[0272] The embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

[0273] Although the spirit and principles of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the specific embodiments disclosed, and the division into various aspects does not mean that the features of these aspects cannot be combined to benefit. Such division is merely for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the appended claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. A multi-user information transmission method based on semantic base, applied to a base station, comprising: Determining a plurality of information sources to be sent to a plurality of user terminals; Based on several semantic feature extraction methods, semantic feature extraction is performed on each of the source information to obtain a first semantic feature set corresponding to each of the source information; wherein the first semantic feature set includes several semantic features obtained based on the several semantic feature extraction methods; Rearranging the plurality of semantic features in the first semantic feature set based on differences between the plurality of semantic features in the first semantic feature set to obtain a second semantic feature set; The plurality of first semantic feature sets and the plurality of second semantic feature sets are superimposed to obtain a superimposed semantic feature set, and the superimposed semantic feature set is sent to the plurality of user terminals.

2. The method according to claim 1, further comprising: The N first semantic feature sets are sorted in ascending order based on the distances between the user terminals and the base stations corresponding to the first semantic feature sets.

3. The method according to claim 2, wherein: The rearranging of the plurality of semantic features in the first semantic feature set based on the differences between the plurality of semantic features in the first semantic feature set includes: Rearranging the plurality of semantic features in the N-1th first semantic feature set based on differences between the plurality of semantic features in the Nth first semantic feature set to obtain a first second semantic feature set; Rearranging the semantic features in the N-3th first semantic feature set based on differences between the semantic features in the N-2th first semantic feature set to obtain a second second semantic feature set; and The above steps are repeated until there is no more first semantic feature sets to be rearranged, thereby obtaining a plurality of second semantic feature sets.

4. The method according to claim 1, wherein The rearranging of the plurality of semantic features in the first semantic feature set based on the differences between the plurality of semantic features in the first semantic feature set includes: Determining a sequence number of each of the semantic features in the first semantic feature set; determining a difference parameter between every two semantic features in the first semantic feature set; Pairing the two semantic features with the goal of maximizing the difference parameter between the two semantic features, and determining the sequence numbers of the paired semantic features; Based on the serial numbers of the paired semantic features, rearrange the serial number sequence composed of the serial numbers of each semantic feature in the first semantic feature set to obtain a rearranged serial number sequence; and Based on the rearranged sequence of sequence numbers, the order of the plurality of semantic features in the first semantic feature set to be rearranged is rearranged to obtain the second semantic feature set.

5. The method according to claim 4, wherein The step of rearranging the order of the plurality of semantic features in the first semantic feature set to be rearranged based on the rearranged sequence of sequence numbers includes: Determining a sequence number of each of the plurality of semantic features in the first semantic feature set to be rearranged; and Based on the rearranged sequence of sequence numbers, the order of the plurality of semantic features in the first semantic feature set to be rearranged is rearranged to obtain the second semantic feature set.

6. The method according to claim 1, wherein The superimposing the plurality of first semantic feature sets and the plurality of second semantic feature sets to obtain a superimposed semantic feature set, and sending the superimposed semantic feature set to the plurality of user terminals includes: determining a decoding order for the plurality of user terminals based on channel parameters of the plurality of user terminals; Allocating transmission power to the first semantic feature set and the second semantic feature set based on the decoding order; Superimposing a plurality of the first semantic feature sets and a plurality of the second semantic feature sets based on the transmission power to obtain the superimposed semantic feature sets; and Based on the transmission power, the superimposed semantic feature set is sent to the plurality of user terminals.

7. The method according to claim 1, wherein The determining of the plurality of information sources to be sent to the plurality of user terminals includes: The information source information to be sent to different user terminals is different.

8. The method according to claim 1, wherein The semantic feature extraction method includes an encoding function; The semantic feature extraction is performed on each of the source information based on the plurality of semantic feature extraction methods to obtain a first semantic feature set corresponding to each of the source information, wherein the first semantic feature set includes a plurality of semantic features obtained based on the plurality of semantic feature extraction methods, including: Based on the plurality of encoding functions, semantic features are extracted for each of the information source information to obtain the first semantic feature set corresponding to each of the information source information, wherein the first semantic feature set includes the plurality of semantic features obtained based on the plurality of encoding functions.

9. The method according to claim 1, wherein The semantic features obtained based on different semantic feature extraction methods are different.

10. A multi-user information transmission method based on semantic base, applied to a user terminal, comprising: receiving a superimposed semantic feature set sent by a base station; In response to determining that the decoding order corresponding to the user terminal ranks first, decoding the superimposed semantic feature set to obtain information source information corresponding to the user terminal; or, In response to determining that the decoding order corresponding to the user terminal is not ranked first, serial interference elimination is performed on the superimposed semantic feature set to obtain the semantic feature set corresponding to the user terminal, and the semantic feature set is decoded to obtain the source information corresponding to the user terminal.

11. The method according to claim 10, wherein: Decoding the semantic feature set to obtain information source information corresponding to the user terminal includes: In response to determining that the semantic feature set has been rearranged, restoring the semantic feature set to obtain a restored semantic feature set; and The restored semantic feature set is decoded to obtain the information source information corresponding to the user terminal.

12. A multi-user information transmission method based on semantic base, applied to a user terminal, comprising: Determining information source to be sent to a base station; performing semantic feature extraction on the source information based on a plurality of semantic feature extraction methods to obtain a third semantic feature set corresponding to the source information, wherein the third semantic feature set includes a plurality of semantic features obtained based on the plurality of semantic feature extraction methods; In response to determining that the user terminal does not participate in the reordering, sending the third semantic feature set to the base station; or, In response to determining that the user terminal participates in the rearrangement, the several semantic features in the third semantic feature set are rearranged based on the differences between the several semantic features in the third semantic feature set to obtain a fourth semantic feature set, and the fourth semantic feature set is sent to the base station.

13. A multi-user information transmission method based on semantic base, applied to a base station, comprising: receiving a superimposed semantic feature set sent by a plurality of user terminals; determining a decoding order of the plurality of user terminals according to channel parameters of the plurality of user terminals; as well as Based on the decoding order, serial interference cancellation is performed on the superimposed semantic feature sets to obtain a plurality of semantic feature sets, and the semantic feature sets are decoded to obtain source information corresponding to the semantic feature sets.

14. A multi-user information transmission device based on semantic base, applied to a base station, comprising: A first information source information determination module is configured to determine a plurality of information source information to be sent to a plurality of user terminals; A first semantic feature extraction module is configured to perform semantic feature extraction on each of the information sources based on a plurality of semantic feature extraction methods to obtain a first semantic feature set corresponding to each of the information sources, wherein the first semantic feature set includes a plurality of semantic features obtained based on the plurality of semantic feature extraction methods; a first semantic feature rearrangement module configured to rearrange the plurality of semantic features in the first semantic feature set based on differences between the plurality of semantic features in the first semantic feature set to obtain a second semantic feature set; as well as The first semantic feature sending module is configured to superimpose several first semantic feature sets and several second semantic feature sets to obtain superimposed semantic feature sets, and send the superimposed semantic feature sets to the several user terminals.

15. A multi-user information transmission device based on semantic base, applied to a user terminal, comprising: A first semantic feature receiving module is configured to receive a superimposed semantic feature set sent by a base station; A first semantic feature decoding module is configured to, in response to determining that the decoding order corresponding to the user terminal is ranked first, decode the superimposed semantic feature set to obtain source information corresponding to the user terminal; as well as The first serial interference elimination module is configured to, in response to determining that the decoding order corresponding to the user terminal is not ranked first, perform serial interference elimination on the superimposed semantic feature set to obtain the semantic feature set corresponding to the user terminal, decode the semantic feature set, and obtain the source information corresponding to the user terminal.

16. A multi-user information transmission device based on semantic base, applied to a user terminal, comprising: A second information source information determination module is configured to determine information source information to be sent to the base station; a second semantic feature extraction module configured to perform semantic feature extraction on the source information based on a plurality of semantic feature extraction methods to obtain a third semantic feature set corresponding to the source information, wherein the third semantic feature set includes a plurality of semantic features obtained based on the plurality of semantic feature extraction methods; a second semantic feature sending module, configured to send the third semantic feature set to the base station in response to determining that the user terminal does not participate in the reordering; as well as The second semantic feature rearrangement module is configured to, in response to determining that the user terminal participates in the rearrangement, rearrange the several semantic features in the third semantic feature set based on the differences between the several semantic features in the third semantic feature set to obtain a fourth semantic feature set, and send the fourth semantic feature set to the base station.

17. A multi-user information transmission device based on semantic base, applied to a base station, comprising: A second semantic feature receiving module is configured to receive a superimposed semantic feature set sent by a plurality of user terminals; a second semantic feature decoding module, configured to determine a decoding order of the plurality of user terminals according to channel parameters of the plurality of user terminals; as well as The second serial interference cancellation module is configured to perform serial interference cancellation on the superimposed semantic feature sets based on the decoding order to obtain several semantic feature sets, decode the semantic feature sets, and obtain source information corresponding to the semantic feature sets.

18. A multi-user information transmission system based on semantic base, comprising: Base station and user terminal; wherein, The base station is configured to perform the method according to any one of claims 1 to 8; the user terminal is configured to perform the method according to any one of claims 9 to 10; or, The user terminal is configured to perform the method according to claim 11; the base station is configured to perform the method according to claim 12.

19. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 8, the method according to any one of claims 9 to 10, the method according to claim 11, or the method according to claim 12 is implemented.

20. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method according to any one of claims 1 to 8, the method according to any one of claims 9 to 10, the method according to claim 11, or the method according to claim 12.

21. A computer program product comprising computer program instructions, which, when executed on a computer, cause the computer to perform the method according to any one of claims 1 to 8, the method according to any one of claims 9 to 10, the method according to claim 11, or the method according to claim 12.

Images