Electronic equipment, communication method and storage medium

Abstract

The invention relates to an electronic device, a communication method and a storage medium in a wireless communication system. An electronic device for a computing device, comprising processing circuitry configured to: receive, from a first user equipment (UE), a message comprising first indication information indicating that the first UE can transmit data of a first modality; receiving a message including second indication information from a second UE, the second indication information indicating that the second UE expects to receive data of a second modality, the second modality being different from the first modality; and determining a conversion model from the data of the first mode to the data of the second mode based on the first indication information and the second indication information.

Description

Technical Field

[0001] The present disclosure generally relates to the field of wireless communication technology, and more specifically, to an electronic device, a communication method, and a storage medium for cross-modality data transmission. Background Art

[0002] In recent years, with the development of the Internet and the popularization of various smart devices, more and more multimodal data are transmitted through the Internet, and more and more multimodal application scenarios are emerging. With the rapid development of multimedia and wireless communication technologies represented by social media, ultra-high-definition mobile video, and 5G, people's communication and audio-visual needs have been greatly met, and they have begun to pursue more dimensional and higher-level experiences.

[0003] As the core support for the ultimate realization of multimodal services, multimodal communication technology has emerged. Multimodal communication has been widely used in various fields, such as social media, online education, telemedicine, video conferencing, the Internet of Things, etc. It brings a more intuitive, more convenient, and more vivid way of communication, which can better meet people's communication needs.

[0004] With the continuous development of technology, multimodal communication is also constantly evolving and improving. For example, with the popularization of 5G wireless communication technology, the picture quality of video calls will be clearer, the delay will be smaller, and people will be able to communicate face to face more naturally. At the same time, the application of artificial intelligence technology can also provide more intelligent services for multimodal communication, such as voice translation, image recognition and other technologies. In general, multimodal communication is in a stage of rapid development. In the future, it will be widely used in more scenarios and will become more and more intelligent and convenient. Summary of the invention

[0005] A brief overview of the disclosure is given in this section to provide a basic understanding of some aspects of the disclosure. However, it should be understood that this overview is not an exhaustive overview of the disclosure. It is not intended to identify the key or important parts of the disclosure, nor is it intended to limit the scope of the disclosure. Its purpose is simply to give some concepts about the disclosure in a simplified form as a prelude to a more detailed description given later.

[0006] According to one aspect of the present disclosure, an electronic device for a computing device is provided, including a processing circuit, configured to: receive a message including first indication information from a first user equipment (UE), the first indication information indicating that the first UE can transmit data of a first modality; receive a message including second indication information from a second UE, the second indication information indicating that the second UE expects to receive data of a second modality, wherein the second modality is different from the first modality; and determine a conversion model from the data of the first modality to the data of the second modality based on the first indication information and the second indication information.

[0007] According to one aspect of the present disclosure, an electronic device for a user equipment (UE) is provided, including a processing circuit, configured to: send a message including first indication information to a computing device, the first indication information indicating that the UE can transmit data of a first modality; and send data of the first modality to the computing device, the data of the first modality being converted by the computing device into data of a second modality that another UE expects to receive, wherein the second modality is different from the first modality.

[0008] According to one aspect of the present disclosure, an electronic device for a user equipment (UE) is provided, including a processing circuit, configured to: send a message including second indication information to a computing device, the second indication information indicating that the UE expects to receive data in a second modality; and receive data in the second modality from the computing device, the data being converted from data in a first modality transmitted by the computing device from another UE, wherein the second modality is different from the first modality.

[0009] According to one aspect of the present disclosure, a communication method is provided, including operations performed by a processing circuit of any of the above electronic devices.

[0010] According to one aspect of the present disclosure, a non-transitory computer-readable storage medium storing executable instructions is provided, wherein the executable instructions implement the above communication method when executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The present disclosure may be better understood by referring to the detailed description given below in conjunction with the accompanying drawings, wherein the same or similar reference numerals are used throughout the drawings to represent the same or similar elements. All drawings together with the following detailed description are included in this specification and form a part of the specification to further illustrate the embodiments of the present disclosure and explain the principles and advantages of the present disclosure. Among them:

[0012] Figure 1 is a flow chart illustrating cross-modal data transmission according to the present disclosure;

[0013] Figure 2A and 2BAn example of determining a conversion model by a computing device according to the present disclosure is shown;

[0014] Figure 3 is a schematic diagram showing an Internet of Vehicles application architecture according to an exemplary embodiment of the present disclosure;

[0015] Figure 4 is a flow chart illustrating cross-modal sensory data transmission according to an exemplary embodiment of the present disclosure;

[0016] Figure 5 is a flow chart illustrating cross-modal IMS session negotiation according to an exemplary embodiment of the present disclosure;

[0017] Fig. 6A and 6B An electronic device for a computing device and a communication method thereof according to the present disclosure are respectively illustrated;

[0018] Fig. 7A and 7B An electronic device for a user device and a communication method thereof according to the present disclosure are respectively illustrated;

[0019] Fig. 8A and 8B An electronic device for a user device and a communication method thereof according to the present disclosure are respectively illustrated;

[0020] Fig. 9 An example block diagram of a computer that can be implemented as a user device or a control device according to an embodiment of the present disclosure is shown.

[0021] Fig.10 A first example of a schematic configuration of a base station according to the present disclosure is illustrated;

[0022] Fig.11 A second example of a schematic configuration of a base station according to the present disclosure is illustrated;

[0023] Fig.12 illustrates a schematic configuration example of a smartphone according to the present disclosure;

[0024] Fig.13 A schematic configuration example of a car navigation device according to the present disclosure is illustrated.

[0025] The features and aspects of the present disclosure will be clearly understood by reading the following detailed description with reference to the accompanying drawings. DETAILED DESCRIPTION

[0026] Various exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. For the sake of clarity and conciseness, not all implementations of the embodiments are described in this specification. However, it should be noted that when implementing the embodiments of the present disclosure, many implementation-specific settings can be made according to specific needs in order to achieve the specific goals of the developers. For ease of explanation only, examples of one or more aspects of the present disclosure may be described below in conjunction with certain specific application scenarios. However, it should be noted that this is not a limitation on the scope of application of the present disclosure, and one or more aspects of the present disclosure may also be applied to various application architectures that are already in common use or will be developed in the future.

[0027] In addition, it should be noted that in order to avoid obscuring the present disclosure due to unnecessary details, only the processing steps and / or device structures closely related to the technical solution of the present disclosure are shown in the drawings. The following description of the exemplary embodiments is merely illustrative and is not intended to be any limitation on the present disclosure and its applications.

[0028] In traditional communication services, whether it is telephone voice services or data transmission services, a certain fixed encoding method is generally used. The two parties in communication have clear needs and have reached an agreement on the transmission format or content based on the protocol or through negotiation before initiating the service. The two parties only use the network as a pipeline to carry the corresponding data transmission.

[0029] Multimodal communication is in a stage of rapid development. In the future, it will be widely used in more scenarios and will become more and more intelligent and convenient. Data of multiple modes can be transmitted on the Internet. However, the two communicating parties may process data of different modes. In business scenarios such as computer image sensor fusion, cross-language and even cross-expression calls, there is a common feature, that is, the transmitted data can exist in semantic / feature form, rather than traditional voice data encoding or image data encoding data, and the data can have different modes at the production and consumption ends. There are even communication modes with uncertain data requirements at the production and consumption ends, such as randomly matched chat rooms, where users of such chat rooms / software may use various languages, including spoken languages, text, sign language, facial expression language, etc.

[0030] In such cross-modal data communication, there may be a situation where the stable matching relationship between the consumer end and the producer end is unclear. The usual solution is to implement the conversion between modes at the application layer and transparent transmission of data at the bottom layer. However, this may place high demands on the data processing capabilities of the terminal device. In addition, in order to meet the real-time requirements, it may be expected to complete some matching processing on the data on the network side or the edge side. Such a business model has not yet appeared, and the Third Generation Partnership Project (3GPP) has not used the corresponding functional definition to implement such a function.

[0031] In view of this, one or more aspects of the present disclosure provide techniques for facilitating cross-modal data communication.

[0032] Figure 1 is a flowchart showing cross-modal data transmission according to the present disclosure. As shown in the figure, the system for cross-modal data transmission includes at least a first user equipment (UE) 101, a computing device 102, and a second UE 103. As used in the present disclosure, the term "UE" has the full breadth of its usual meaning, including various terminal devices or vehicle-mounted devices. UEs can communicate with each other via infrastructure such as base stations, or can communicate directly between UEs. As an example, a UE can be, for example, a mobile phone, a handheld device, a media player, a laptop, a tablet computer, an on-board unit (OBU) or a vehicle, a machine-to-machine (M2M) or Internet of Things (IOT) terminal, a roadside unit (RSU), etc. or an element thereof. The following sections will describe the application examples of UE in detail.

[0033] The first UE 101 can be operated as a production-end UE that generates and sends data, and the second UE 103 can be operated as a consumer-end UE that receives and uses data. It should be noted that the production-end and the consumer-end are for a specific data transmission direction. In the opposite direction, the first UE 101 may also be operated as a consumer-end UE, and the second UE 103 may also be operated as a production-end UE. Therefore, Figure 1 It is not intended to limit the functionality of each UE.

[0034] According to the present disclosure, for example, the first UE 101 can output data of the first modality. As used in the present disclosure, the "modality" of data refers to the attributes of the existence form, source, state, etc. of the data, and similar expressions include "mode", "format", "morphology", etc. If the first UE 101 itself has performed data fusion, the first modality can represent a combination of multiple modalities. The second UE 103 can input data of the second modality, wherein the second modality is different from the first modality. The difference between the first modality and the second modality may be one or more of the following:

[0035] 1) Type of data. For example, the data of the first modality may be a combination of one or more of audio data, image / video data, radar data, lidar data, extracted feature data, extracted semantic data, etc., while the data of the second modality may be a combination of another one or more of the above;

[0036] 2) Feature data category. The data of the first modality may be, for example, feature data obtained by the first UE 101 performing preliminary processing on the original data using the model, and / or the data of the second modality may be, for example, feature data to be further processed by the model of the second UE 103. In this case, the modality of the data may include attributes related to the feature, such as the name of the model used to process the data, the channel or layer of the output data, the format or dimension of the feature matrix, etc.;

[0037] 3) Semantic data category. The data of the first modality may be, for example, semantic data extracted from the original data by the first UE 101 using a model, and / or the data of the second modality may be, for example, semantic data to be further processed by the model of the second UE 103. Semantics can be simply regarded as the meaning of the concepts represented by the things in the real world corresponding to the data and the relationship between these meanings, which is the interpretation and logical representation of the data in a certain field. The modality of semantic data may include semantic-related attributes, such as semantic type, name of semantic extraction model, channel or layer of output semantic data, format or dimension of semantic data, etc.;

[0038] 4) Data encoding method. The data of the first modality may be encoded by a codec at the first UE 101 to have a first format, while the data of the second modality may have a second format acceptable to the codec at the second UE 103;

[0039] 5) Data content characteristics. Content characteristics here refer to attributes related to the content of the data itself, such as the language of the call (e.g., Chinese, English, French, etc.), the expression method (e.g., voice, text, sign language, expression, body posture, etc.), the physical location setting of the data source (e.g., the field of view (FOV), height, viewing direction, ambient lighting conditions, etc. of the sensor source), the range of the data (e.g., the value range and unit of the sensor data, etc.), or other content characteristics.

[0040] The difference between the first modality and the second modality can also be reflected in any other aspects. Typically, whether the modality of the data is different is in terms of processing requirements. In other words, if the data output by the first UE 101 does not meet the processing requirements of the second UE 103, it can be considered that there is a modality difference. On the contrary, if the data of the first UE 101 can be directly processed by the second UE 103, it is considered that their modalities are matched or compatible. In this case, the data of the first UE 101 can be transmitted to the second UE 103 for consumption without intermediate conversion.

[0041] The computing device 102 may be configured to relay data transmission between the first UE 101 and the second UE 103. In the scenario of infrastructure communication, the computing device 102 may be implemented as a network side device, and the first UE 101 or the second UE 103 may communicate with the computing device 102, for example, via a Uu interface. In one example, the computing device 102 may be deployed in a base station, for example, in a central unit (CU). As used in the present disclosure, the term "base station" is a control device in a wireless communication system, with the full breadth of its usual meaning. For example, in addition to the gNB and ng-eNB specified in the 5G communication standard, depending on the scenario in which the technical solution of the present disclosure is applied, the "base station" may also be, for example, an eNB in ​​an LTE communication system, a remote radio head, a wireless access point, a relay node, a drone control tower, or a communication device or element thereof that performs similar control functions. The following sections will describe in detail the application examples of the base station. In another example, the computing device 102 may be deployed in, for example, a core network, for example, as a network function (NF) unit that provides the functions described in the present disclosure.

[0042] In the scenario of device-to-device (D2D) communication, the computing device 102 can be implemented as an edge device with computing capabilities, such as a roadside computing unit (RSCU) and a multi-access edge computing (MEC) platform, which will be described in detail later. The first UE 101 or the second UE 103 can communicate with the computing device 102, for example, through a PC5 interface. As an example of D2D communication, vehicle-to-everything (V2X) technology can be used. The vehicle provides vehicle information through sensors, vehicle terminals, and electronic tags installed on the vehicle, and realizes the interconnection and intercommunication between vehicles (V2V), vehicles and people (V2P), vehicles and facilities (V2I), and vehicles and networks (V2N) through a variety of communication methods including direct links (Sidelink).

[0043] like Figure 1 As shown in , the method for facilitating cross-modal data transmission according to the present disclosure includes a computing device 102 receiving a message including first indication information from a first UE 101 (step S1). The first indication information indicates that the first UE 101 can transmit data of the first modality. For example, the first indication information can be used to identify the first modality, including the type of data, encoding method, content characteristics, etc. For feature data / semantic data generated by the first UE 101 through a model, the first indication information may include information about the model name, feature size (e.g., the number of channels, the output format of the output feature matrix, feature layer identification), etc.

[0044] The computing device 102 also receives a message including second indication information from the second UE 103 (step S2). The second indication information indicates that the second UE 103 expects to receive data of the second modality. For example, the second indication information can be used to identify the second modality, including the type of data, encoding method, content characteristics, etc. For feature data / semantic data, the second indication information can include information about the model name, feature size (e.g., the number of channels, the output format of the output feature matrix, feature layer identification), etc.

[0045] Optionally, the first indication information and / or the second indication information may also include various side information to facilitate subsequent data transmission, including but not limited to: identification information of the producer or consumer of the data (for example, the ID of the first UE 101 or the second UE 103), timeliness requirements of the data (especially for sensor data), source requirements of the data, and the like.

[0046] The message transmitting the first indication information and / or the second indication information may include various control signaling messages. In one example, the first indication information and / or the second indication information may be reported to the computing device 102 as user capability information via a radio resource control (RRC) message. In another example, the first indication information and / or the second indication information may be sent to the computing device 102 via a control channel on an uplink or a direct link.

[0047] The sending of the first / second indication information in steps S1 and S2 may be performed according to the protocol or pre-configured periodicity. Alternatively, steps S1 and S2 may also be dynamically triggered, for example, in response to a request from the first UE 101 to establish a data transmission connection with the second UE 103 ( Figure 1 ) or a request by the second UE 103 for data that can be provided by the first UE 101 ( Figure 1 ), the first UE 101 and the second UE 103 respectively send the first indication information and the second indication information to the computing device 102. The sending of the first indication information and the second indication information may not occur during the current connection session. For example, the first indication information may be sent to the computing device 102 during the registration of the first UE 101 or during the previous connection session, and recorded as historical information at the computing device 102. Similarly, the second indication information may be sent to the computing device 102 during the registration of the second UE 103 or during the previous connection session, and recorded as historical information at the computing device 102.

[0048] Optionally, the modality information of the data of the first UE 101 may be implicitly indicated to the computing device 102. In one example, the first UE 101 directly transmits data to the computing device 102, the data packet header carries a specific field to indicate information about the first modality, and the computing device 102 analyzes the header to obtain the indicated information. In another example, the first UE 101 directly transmits data to the computing device 102, and the computing device 102 analyzes the content of the data to determine the modality-related information corresponding to the data transmission, which is possible for a scenario where, for example, the computing device 102 itself is a consumer of the data of the first UE 101.

[0049] Subsequently, based on the first indication information and the second indication information, the computing device 102 may determine a conversion model from the data of the first modality to the data of the second modality (step S3). If the first indication information and the second indication information indicate that the first modality matches the second modality, that is, the data of the first UE 101 can be directly consumed by the second UE 103, the computing device 102 may determine that no modality conversion is required. On the contrary, if the first modality and the second modality do not match, the computing device 102 determines a conversion model for converting the data from the first modality to the second modality based on the first indication information and the second indication information.

[0050] According to the present disclosure, the conversion model includes a transformer that couples the first modality to the second modality. For example, for feature / semantic data, the conversion model can implement coupling between features of different formats so that data provided by the first UE 101 can be used for the second UE 103. For the first UE 101 outputting data encoded by a codec that is incompatible with the codec of the second UE 103, the computing device 102 can determine the conversion model as a transcoder between the two codecs.

[0051] The conversion model can be implemented as an artificial intelligence (AI) model, including but not limited to: convolutional neural network (CNN), generative adversarial network (GAN), recent neural network (RNN), transformer model, etc. Alternatively, other deep learning models can also be used as long as the model can achieve modal conversion.

[0052] Figure 2A An example of determining a conversion model by computing device 102 is shown. Figure 2AAs shown in , the computing device 102 may pre-configure a set of conversion models, each of which may implement conversion from one or more modalities to another one or more modalities. Based on the first modality of the data of the first UE 101 indicated by the first indication information and the second modality of the data of the second UE 103 indicated by the second indication information, the computing device 102 may select a conversion model (e.g., Figure 2A The conversion model deployed and run by the computing device 102 may be a standardized model, that is, a model that can be used by a series of standardized computing devices, and the output of the first UE 101 and the input of the second UE 103 are also standardized (in a data mode that can be sent and accepted by a series of standardized products). Such standardization is beneficial for the convenient implementation of the system.

[0053] Figure 2B Another example of determining a conversion model by computing device 102 is shown. Figure 2B As shown in the figure, the computing device 102 can pre-configure at least two groups of conversion models, namely the input conversion model and the output conversion model shown in the figure, each of which can realize the conversion from one or more modalities to another one or more modalities. Based on the first modality of the data of the first UE 101 indicated by the first indication information, the computing device 102 can select a conversion model (for example, a conversion model) that takes the data of the first modality as input and outputs the data of the intermediate modality from a group of input conversion models. Figure 2B Based on the second modality of the data of the second UE 103 indicated by the second indication information, the computing device 102 may select a conversion model (e.g., Figure 2B With the help of the intermediate modality, the computing device 102 can implement the conversion from the first modality to the second modality through two (or even more) conversion models.

[0054] Back to Figure 1 , as a response to a data transmission request from the first UE 101 or the second UE 103, the computing device 102 can establish a data transmission connection from the first UE 101 to the second UE 103. In particular, transmission resources, such as time-frequency resources of a wireless link, can be allocated for the link from the first UE 101 to the computing device 102 and the link from the computing device 102 to the second UE 103. In addition, the computing device 102 can allocate computing resources for the determined conversion model and instantiate the conversion model using the allocated computing resources. The allocation of computing resources for the conversion model may be subject to constraints related to the service level to meet, for example, corresponding delay requirements, reliability requirements, priority requirements, etc.

[0055] Using the established connection, the first UE 101 can generate and transmit data of the first modality (step S4), for example, the first UE 101 can use its specific AI model to generate feature / semantic data of the first modality, or use a specific codec to generate coded data (such as audio coded data, image coded data, video coded data, etc.). The computing device 102 receives the data of the first modality, converts it into data of the second modality using the conversion model, and transmits the converted data to the second UE 103 (step S5). The second UE 103 can use a specific AI model or codec to process the data of the second modality that meets its processing needs.

[0056] Optionally, the computing device 102 may also instruct the first UE 101 to adjust settings to change its data transmission mode. This may be useful in some cases. For example, the computing device 102 may not be able to select a conversion model that implements the conversion from the first mode to the second mode, but can select a conversion model for the conversion from the third mode supported by the first UE 101 to the second mode, then the computing device 102 may instruct the first UE 101 to generate data in the third mode. For another example, the second UE 103 may change its content needs to be consumed, then the computing device 102 may also instruct the first UE 101 to generate data of the corresponding mode. For another example, an additional third UE may issue a data request, then the computing device 102 may instruct the first UE to adjust the data generation settings to generate data of the corresponding mode according to the processing requirements of the third UE.

[0057] Next, exemplary embodiments according to the present disclosure are described.

[0058] An exemplary embodiment of the present disclosure relates to a D2D communication scenario. For ease of explanation, the following description is made by taking a sensor data fusion service as an example, but it should be understood that this is not a limitation on the application scenario of the present disclosure.

[0059] In the sensor data fusion service, edge-side devices (such as RSU, MEC platform, etc.) or network-side devices (base stations, core networks) match and couple the supply and demand of sensor data of production-end UE and consumer-end UE within a certain geographical range. For UEs with high mobility characteristics such as vehicles / drones, the initiation and implementation of sensor data fusion services are often extremely unstable. Two UEs with matched sensor data supply relationships may be interrupted at any time according to changes in road traffic conditions (such as changes in the blind spot range of the vehicle's visual sensor system, interference from external factors to sensors of different modalities, and changes in the direction of vehicle travel). Another situation is that there are many types of vehicles on the road, and even if the sensors they are equipped with belong to the same type (i.e., lidar, camera, radar, etc.), it is extremely difficult to unify the specifications. In addition, there is also the problem of different sensor data processing algorithms. In this regard, the cross-modal data transmission technology disclosed in the present invention can be utilized.

[0060] Figure 3 is a schematic diagram showing a vehicle networking application architecture according to an exemplary embodiment of the present disclosure. Figure 3 As shown in , an exemplary system may include a central subsystem (CSS), a multi-access edge computing (MEC) platform, an on-board subsystem, a roadside unit (RSU), and a vulnerable traffic participant (VRU). The central subsystem has the ability to communicate with the on-board subsystem, the road subsystem, the VRU, etc. The central subsystem has the ability to receive, store, process, and distribute global data, and is responsible for global information perception and global business policy control. Typically, the central subsystem can be implemented in a core network unit (e.g., an application function (AF)) or an application server.

[0061] The road subsystem may include one or more of the RSU, MEC platform, and roadside sensing equipment. Roadside sensing equipment is deployed at intersections, overpasses, main roads, and auxiliary roads. The roadside sensing equipment has a specific sensing range, such as extending a certain range around the pedestrian part of the intersection as the sensing range, or taking the junction of the non-motorized vehicle lane and the motor vehicle lane of the overpass as the sensing range. The roadside sensing equipment has the ability to identify VRUs and vehicles within its sensing range by equipping it with sensor equipment such as camera sensors, lidar sensors, and radar sensors.

[0062] RSU is a communication unit that has a (wired) connection with the roadside sensing device and the roadside MEC platform, and has roadside sensing and communication capabilities (based on 4G / 5G cellular wireless communication, 4G / 5G Sidelink communication). RSU can transmit the sensed information to the vehicle subsystem, multi-access edge computing platform and central subsystem; it can receive application layer information sent from the vehicle subsystem, multi-access edge computing platform and central subsystem. RSU can even be integrated with roadside sensing devices and / or MEC platforms.

[0063] The roadside MEC platform is generally deployed at a roadside location close to the RSU and related roadside sensing equipment. It provides various services by analyzing the road information collected by the RSU (including vehicle information, VRU information, traffic light information, etc.) and the road information captured by the roadside sensing equipment (movement information of various vehicles and VRUs, including original or processed video data streams, 3D point cloud data, etc.). In the MEC platform, computing resources can be deployed to provide support for meeting the computing needs of communication services (such as communication computing fusion services). As an example, the MEC platform can be implemented in a radio access network (RAN). The MEC platform and RSU can exist as two physically separated entities or as the same physical entity device (RSCU).

[0064] Figure 4 is a flow chart showing a cross-modal sensory data transmission according to an exemplary embodiment of the present disclosure. It should be noted that Figure 4 There is no strict order for the steps in the process; the order of some steps can be reversed or performed in parallel.

[0065] As shown in the figure, the first UE as the producer of the sensor data indicates to the edge device (RSU or RSCU) the modal information of the sensor data that it can provide (step S11). This information can be referred to as Figure 1 The first indication information described is sent to the RSU or RSCU in the form of the first indication information. The first UE may have one or more sensors, such as a camera sensor, a lidar sensor, a radar sensor, etc.

[0066] In step S11, the difference between RSU and RSCU is that RSU only has sensing and communication capabilities (so it may need to interact with the matching MEC platform to realize necessary computing functions such as data processing and logic processing), while RSCU has the ability to analyze and calculate the sensing data (i.e., the functional entity connected by the MEC platform and RSU). The edge side device needs to obtain information about the sensing data of the first UE, for example:

[0067] 1) The type of data, that is, the sensor source type of the data itself is a specific type of laser radar, camera, radar. If the first UE itself has performed data fusion, it can contain more than one modality;

[0068] 2) Category of feature / semantic data. If the first UE performs data processing on the original data to output feature / semantic data, the model name corresponding to the data processing and the size of the feature (e.g., the number of channels, the output format of the output feature matrix, such as feature size, feature layer identifier of the corresponding model, etc.) may be indicated;

[0069] 3) The content characteristics of the data, such as the range of the sensor data, the location information of the data source itself, the environment and conditions of data collection, etc., including the physical location settings of the sensor source, such as field of view, height, viewing direction, ambient lighting conditions, etc.

[0070] The first indication information may also include other secondary information, such as ID information of the first UE, timeliness, source, and usage restrictions of the data.

[0071] Similarly, the second UE may indicate to the edge device the modality information of the data it desires, that is, the second indication information. Similar to the first indication information, the second indication information may indicate the type, feature / semantic category, content characteristics, etc. of the data required by the second UE. Figure 4 The second UE is shown to send a data request together with the second indication information to the edge device, but this is only an example. As an alternative, the second UE may also send the second indication information to the edge device in advance, and then send the data request when data is needed. An edge device such as an RSCU or MEC platform may store the first indication information from the first UE and the second indication information from the second UE.

[0072] In response to the second UE's request for the sensing data of the first UE, the edge device may determine whether the data that the first UE can provide matches the data expected by the second UE based on the first indication information and the second indication information. This determination may be performed by the RSCU based on computing power. As an alternative, Figure 4 As shown in steps S13 and S14 indicated by dotted arrows, the RSU can send a matching check request (S13) to the MEC platform. The MEC platform determines whether the data modalities of the first UE and the second UE match based on the first indication information and the second indication information, and returns a matching check response (S14) to the RSU.

[0073] If there is no match, the edge side device (such as RSCU or MEC platform) can also determine the modal conversion model from the data of the first UE to the second data based on the first indication information and the second indication information, as described above with reference to Figure 1 as described.

[0074] In response to the data request of the second UE, the edge device (eg, RSU or RSCU) returns a message indicating service acceptance to the second UE (S15). In addition, the edge device may also send a data request to the first UE (S16) to request the sensing data required by the second UE.

[0075] The first UE transmits the generated sensing data of the first modality to the edge device. These transmission data can be converted into data of the second modality by the RSCU or MEC platform using a previously determined conversion model and transmitted to the second UE. Subsequently, the second UE can further process the data to provide corresponding services.

[0076] As an example and not a limitation, an application example of cross-modal data transmission in the Internet of Vehicles will be described. Assume that vehicle 1 relies on a visual camera sensor to perceive surrounding objects (i.e., the original sensor data is a 2D image), while vehicle 2 relies on sensors such as a lidar or radar to perceive surrounding objects (i.e., the original sensor data is 3D point cloud data, or even 4D data). The respective sensor systems of vehicle 1 and vehicle 2 will also process the raw data using different sensor data processing models. For example, vehicle 1 directly performs operations such as segmentation, object recognition, and classification on the image from the camera sensor, or uses multiple camera sensors equipped with the vehicle to predict the depth / distance / size of the object to obtain three-dimensional information. Vehicle 2 can process the point cloud data obtained by, for example, a lidar or radar into a 3D data format such as pillar or voxel or an image format such as BEV (bird's eye view).

[0077] When it comes to sensor data fusion of multiple vehicles, heterogeneous vehicles are likely to adopt heterogeneous perception system structure settings. At this time, it is necessary to use the server on the edge side or the core network side to deploy the corresponding cross-modal data conversion processing model to match the feature data of different vehicle UEs (features are often the intermediate results of nonlinear processing of raw data, which can be more intuitively understood as the outline, position, and material of an object being abstracted into a mathematical expression in the form of a matrix, with one or more matrices expressing the features of an object captured by a sensor in multiple dimensions). Therefore, vehicle 1 and vehicle 2 can refer to Figure 4 The process indicates the modal information of the data to the edge device to facilitate the selection of an appropriate modal conversion model.

[0078] In addition, for less stringent cross-modal data, such as differences in the height of the sensor source and the light-sensitive environment, or situations where the sensor source type is the same but the models used by the producer and consumer are inconsistent, these additional parameters (height, light intensity, etc.) also need to be indicated to the RSU. Similarly, the parameters of the perception system of vehicle 2, as a consumer, can also be provided to the edge-side device, which inputs these parameters into the conversion processing model to obtain input that better matches the consumer-side perception system. Edge-side devices such as RSCU or MEC platforms convert data of different modalities so that vehicle 2 can obtain input data acceptable to its perception system / perception model to facilitate data fusion.

[0079] Another exemplary embodiment of the present disclosure relates to an infrastructure communication scenario. For ease of explanation, the following description is made by taking an IP Multimedia Subsystem (IMS) service as an example, but it should be understood that this is not a limitation on the application scenario of the present disclosure.

[0080] Figure 5 1 is a flowchart showing a cross-modal IMS session negotiation according to an exemplary embodiment of the present disclosure. As shown in the figure, in addition to the first UE and the second UE participating in the session, the network elements involved are mainly call session control functions (CSCFs). CSCF is the core part of IMS and is mainly used for SIP session control based on packet switching. In IMS, CSCF is responsible for processing user multimedia sessions and can be regarded as a SIP server in the IETF architecture. According to their different main functions, CSCF can be divided into a proxy call session control function (P-CSCF), an inquiry call session control function (I-CSCF) and a service call session control function (S-CSCF), and the three functions can be physically separated or independent. In short, P-CSCF mainly checks whether the media parameters related to the IMS call can be supported (based on whether the current network resources can support the QoS requirements corresponding to the IMS service), while S-CSCF mainly checks whether the IMS voice service initiated by the UE is consistent with the user profile of the UE (for example, whether the service is opened, the service level, etc.).

[0081] In step S21, the first UE as the session initiator reflects the UE capabilities and user preferences to the communication network by carrying a set of codecs supported by the first UE in the Session Description Protocol (SDP) payload. This set of codecs contains data modalities corresponding to the codecs that the first UE currently prefers to use, including but not limited to:

[0082] 1) The specific languages ​​that users tend to use (various ethnic languages);

[0083] 2) Specific forms of expression, such as video, voice, text, sign language, facial expressions, body gestures, etc., or a combination of multiple forms of expression.

[0084] In step S22, the first UE sends the determined codec information (equivalent to reference Figure 1 The first indication information) is sent to CSCF#1 (including P-CSCF and C-CSCF, where the two functional entities are regarded as one, but they can also be deployed separately, and the operation is performed by the corresponding functional entities) serving the first UE through INVITE signaling;

[0085] In step S23, CSCF#1 confirms the type of transcoder that can be provided for the first UE (including confirming that the codec listed by the first UE can be used as the input of the transcoder, and the corresponding output format) and the corresponding computing resources according to the codec indicated by the first UE, because the corresponding computing resources need to be instantiated to run such a transcoder;

[0086] In step S24, CSCF#2 corresponding to the second UE receives and parses the INVITE message, and in step S25 converts the message to the second UE;

[0087] In step S26, the second UE determines a set of codecs and other parameters that it can support for the IMS session according to its own capabilities;

[0088] In step S27, the second UE sends a message to CSCF#2 containing the codec information and parameters determined in step S26 (equivalent to reference Figure 1 the SDP response of the second indication information);

[0089] In step S28, CSCF#2 determines a transcoder corresponding to a set of codecs supported by the second UE based on the information from the second UE;

[0090] In step S29, CSCF#2 forwards the SDP response to CSCF#1 and additionally indicates the transcoder list corresponding to the codec supported by the second UE. In this step, CSCF#1 needs to combine the transcoder corresponding to the codec of the first UE, the codec of the second UE, and the transcoder supported by CSCF#2 to confirm the actual executable codec list of the first UE and the transcoder solution corresponding to each codec in the list (including the case where only one CSCF instantiates the transcoder and the case where both parties need to instantiate the transcoder);

[0091] In step S210, CSCF#1 sends a codec list actually suitable for the first UE to the first UE;

[0092] In step S211, the first UE selects a specific codec to be used in the next IMS service;

[0093] In step S212, the first UE indicates its selected codec to CSCF#1 through an SDP offer;

[0094] In step S213, CSCF#1 and CSCF#2 finally determine the instantiation of the transcoder and related computing resources corresponding to the codec according to the codec finally determined by the first UE.

[0095] It should be noted that in Figure 5In the example shown, CSCF#1 provides the first UE with a list of codecs that can actually run, so that the first UE can determine the codec (and the corresponding mode) to be used, and determine the encoder and transcoder actually used by the second UE based on the codec of the first UE. However, this is not the only implementation method.

[0096] An alternative implementation is that the transcoder and modality selection is determined by the CSCF negotiation between the first UE and the second UE, and both UEs only need to carry information indicating the data modality they can accept in the messages of steps S22 and S27. CSCF#2 corresponding to the second UE learns the modalities acceptable to both UEs, as well as the transcoders that can be instantiated in combination with the CSCFs on both sides, confirms the modality matching relationship and the list of transcoders that the called party should actually deploy in step S28, and indicates it to the other party CSCF#1 in step S29. Subsequently, based on the codec actually used determined by the first UE in step S211, CSCF#1 selects its own transcoder and completes the instantiation.

[0097] Another alternative implementation is that in step S26, the second UE selects to determine a unique codec (and corresponding mode), and then the CSCF#2 corresponding to the second UE selects to determine a unique transcoder in step S28, where the transcoder has a matching transcoder at the CSCF#1 corresponding to the first UE. Then, based on the actual codec used determined by the first UE in step S211, the CSCF#1 corresponding to the first UE selects its own transcoder and completes the instantiation.

[0098] As an example but not a limitation, the application of the IMS communication service according to the present disclosure may involve calls across modes of expression, such as voice, text, body language (e.g., sign language, body movements, tension and relaxation of body postures, etc.), facial expressions and other modes of expression, so it is necessary to consider the semantics of connecting these different modes of expression, that is, the mathematical expression abstracted from the meaning that people want to express. With the help of the intermediate processing unit in the communication network, the call data of one mode generated by the first UE can be converted into a common semantic feature, and then converted from the common semantic feature to another specific mode.

[0099] Another possible application is cross-language communication. For example, translation from some minority languages ​​to English and then to other non-English languages ​​is often better than translation from minority languages ​​directly to other languages. Therefore, in some cases, a universal language or a language with translation research results for multiple languages ​​can be selected as the intermediate language. Two possible implementations can be imagined:

[0100] 1) GPT-type large model deployment, all modal data features can be processed by a comprehensive large model, then the steps required by CSCF and core network include using the large model to perform real-time conversion of cross-modal data (the abstract expression of semantics is located in a middle layer of the large model). However, the deployment energy consumption, training cost, and real-time performance of the large model are all seriously flawed;

[0101] 2) Deployment of a specific model, that is, the model has a transcoder function from a specific modality to an intermediate modality, and the training and deployment costs are low, so it can be applied in the process set by the present invention, and the CSCF corresponding to each UE selects the corresponding transcoder to couple cross-modal data.

[0102] Regardless of the method, the transcoder can be bidirectional. That is, unlike the vehicle network scenario where there are clear production and consumption ends, IMS services usually involve the two parties sending data to each other, so the input and output of the transcoder are preferably bidirectional.

[0103] In the actual deployment process, users can choose the mode they want. For example, users can manually set their currently acceptable expression mode and commonly used language types in the mode settings of their UE (such as mobile phones or other terminals). For example, if the user only accepts text mode expressions in the Do Not Disturb mode, then the incoming call can be directly connected, the other party's voice information will be converted into text in real time and displayed on the screen, and the text entered by the user can also be converted into voice in real time and sent to the other party.

[0104] The UE operator can record the user's modal usage habits and form a corresponding user profile. When the user uses or passively receives a call for the corresponding service, the operator selects the model or transcoder corresponding to the user's commonly used modality. In this mode, there is no user selection / decision-making process. The CSCFs of both parties can match and select based on the user's profile record.

[0105] Alternatively, the user may be provided with an opportunity to select a mode when the service actually occurs. For example, when an IMS call is ringing, the screen pops up options for the mode, such as a normal voice call mode, a do-not-disturb text mode, etc., and the user can then select the desired mode (this can be done in, for example, Figure 5 (implemented in step S26 in ).

[0106] Next, an electronic device and a communication method according to an embodiment of the present disclosure are described.

[0107] Fig. 6A is an example of a method for computing a device (eg Figure 1 102) in a block diagram of an electronic device 600, Figure 6Bis a flow chart showing a communication method executable by an electronic device 600. The electronic device 600 may be included in an edge side device (e.g., an RSCU or an MEC platform), or may be included in a network side device (e.g., a base station or a core network). The electronic device 600 may communicate with electronic devices 700 and 800 described below.

[0108] like Fig. 6A As shown in FIG. 6 , the electronic device 600 includes a processing circuit 601, and the processing circuit 601 includes at least a receiving unit 602 and a determining unit 603. The processing circuit 601 may be configured to execute Figure 6B The communication method shown in . Processing circuit 101 may refer to various implementations of digital circuit systems, analog circuit systems, or mixed-signal (a combination of analog and digital signals) circuit systems that perform functions in a computing system. Processing circuits may include, for example, circuits such as integrated circuits (ICs), application specific integrated circuits (ASICs), portions or circuits of separate processor cores, entire processor cores, separate processors, programmable hardware devices such as field programmable gate arrays (FPGAs), and / or systems including multiple processors.

[0109] The receiving unit 602 of the processing circuit 601 is configured to receive a message including first indication information from the first UE, wherein the first indication information indicates that the first UE can transmit data of the first modality (ie, perform Figure 6B In addition, the receiving unit 602 is further configured to receive a message including second indication information from the second UE, wherein the second indication information indicates that the second UE expects to receive data of the second modality (ie, to perform Figure 6B In step S602), the second modality is different from the first modality.

[0110] The determining unit 603 is configured to determine a conversion model from the data of the first modality to the data of the second modality based on the first indication information and the second indication information received by the receiving unit 602 (ie, perform Figure 6B In step S603 of the embodiment, the determining unit 603 may determine an AI model that takes the data of the first modality as input and takes the data of the second modality as output. In another example, the determining unit 603 determines a first AI model that takes the data of the first modality as input and takes the data of the intermediate modality as output, and a second AI model that takes the data of the intermediate modality as input and takes the data of the second modality as output.

[0111] The determining unit 603 may also allocate computing resources to the determined conversion model, and instantiate the conversion model using the allocated computing resources. Through the conversion model, a cross-modal data transmission connection may be established between the first UE and the second UE.

[0112] The electronic device 600 may further include, for example, a communication unit 605. The communication unit 605 may be configured to communicate under the control of the processing circuit 601. In one example, the communication unit 605 may be implemented as a transceiver, including communication components such as an antenna array and / or a radio frequency link. The communication unit 105 is drawn with a dotted line because it may also be located outside the electronic device 600.

[0113] The electronic device 600 may further include a memory 606. The memory 606 may store various data and instructions, such as programs and data for the operation of the electronic device 600, various data generated by the processing circuit 601, etc. The memory 606 is drawn with a dotted line because it may also be located within the processing circuit 601 or outside the electronic device 600. The memory 106 may be a volatile memory and / or a non-volatile memory. For example, the memory may include, but is not limited to, a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a read-only memory (ROM), and a flash memory.

[0114] Fig. 7A is an example of a method for a user device (eg Figure 1 101) in a block diagram of an electronic device 700, Figure 7B is a flow chart showing a communication method that can be performed by the electronic device 700. The electronic device 700 can communicate with the electronic device 600 described above.

[0115] like Fig. 7A As shown in FIG. 1 , the electronic device 700 includes a processing circuit 701, and the processing circuit 701 includes at least an indication information sending unit 702 and a data sending unit 703. The processing circuit 701 may be configured to execute Figure 7B The communication method shown in . Processing circuit 701 may refer to various implementations of digital circuit systems, analog circuit systems, or mixed-signal (a combination of analog and digital signals) circuit systems that perform functions in a computing system. Processing circuits may include, for example, circuits such as integrated circuits (ICs), application specific integrated circuits (ASICs), portions or circuits of separate processor cores, entire processor cores, separate processors, programmable hardware devices such as field programmable gate arrays (FPGAs), and / or systems including multiple processors.

[0116] The instruction information sending unit 702 of the processing circuit 701 is configured to send a message including the first instruction information to the computing device (i.e., execute Figure 7B The first indication information indicates that the UE can transmit data of the first modality.

[0117] The data sending unit 703 is configured to send data of the first modality to the computing device (i.e., execute Figure 7B The data of the first modality is converted by the computing device into data of the second modality that another UE expects to receive, wherein the second modality is different from the first modality.

[0118] The electronic device 700 may further include, for example, a communication unit 705. The communication unit 705 may be configured to communicate under the control of the processing circuit 701. In one example, the communication unit 705 may be implemented as a transceiver, including communication components such as an antenna array and / or a radio frequency link. The communication unit 705 is drawn with a dotted line because it may also be located outside the electronic device 700.

[0119] The electronic device 700 may further include a memory 706. The memory 706 may store various data and instructions, such as programs and data for the operation of the electronic device 700, various data generated by the processing circuit 701, etc. The memory 706 is drawn with a dotted line because it may also be located within the processing circuit 701 or outside the electronic device 700. The memory 706 may be a volatile memory and / or a non-volatile memory. For example, the memory may include, but is not limited to, a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a read-only memory (ROM), and a flash memory.

[0120] Fig. 8A is an example of a method for a user device (eg Figure 1 A block diagram of an electronic device 800 of a second UE 103 in FIG. Figure 8B is a flow chart showing a communication method that can be performed by the electronic device 800. The electronic device 800 can communicate with the electronic device 600 described above.

[0121] like Fig. 8A As shown in FIG. 8 , the electronic device 800 includes a processing circuit 801, and the processing circuit 801 includes at least an indication information sending unit 802 and a data receiving unit 803. The processing circuit 801 may be configured to execute Figure 8B The communication method shown in . Processing circuit 801 may refer to various implementations of digital circuit systems, analog circuit systems, or mixed-signal (a combination of analog and digital signals) circuit systems that perform functions in a computing system. Processing circuits may include, for example, circuits such as integrated circuits (ICs), application specific integrated circuits (ASICs), portions or circuits of separate processor cores, entire processor cores, separate processors, programmable hardware devices such as field programmable gate arrays (FPGAs), and / or systems including multiple processors.

[0122] The instruction information sending unit 802 of the processing circuit 801 is configured to send a message including the second instruction information to the computing device (i.e., execute Figure 8B The second indication information indicates that the UE expects to receive data of the second modality.

[0123] The data receiving unit 803 is configured to receive data of the second modality from the computing device (i.e., execute Figure 8B The data of the second modality is converted by the computing device from the data of the first modality transmitted from another UE, wherein the second modality is different from the first modality.

[0124] The electronic device 800 may further include, for example, a communication unit 805. The communication unit 805 may be configured to communicate under the control of the processing circuit 801. In one example, the communication unit 805 may be implemented as a transceiver, including communication components such as an antenna array and / or a radio frequency link. The communication unit 805 is drawn with a dotted line because it may also be located outside the electronic device 800.

[0125] The electronic device 800 may further include a memory 806. The memory 806 may store various data and instructions, such as programs and data for the operation of the electronic device 800, various data generated by the processing circuit 801, etc. The memory 806 is drawn with a dotted line because it may also be located within the processing circuit 801 or outside the electronic device 800. The memory 806 may be a volatile memory and / or a non-volatile memory. For example, the memory may include, but is not limited to, a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a read-only memory (ROM), and a flash memory.

[0126] It should be understood that the various units of the electronic devices 600, 700, and 800 described in the above embodiments are only logical modules divided according to the specific functions implemented by them, rather than being used to limit the specific implementation methods. In actual implementation, the above units can be implemented as independent physical entities, or can also be implemented by a single entity (for example, a processor (CPU or DSP, etc.), an integrated circuit, etc.).

[0127] [Exemplary Implementation of the Present Disclosure]

[0128] According to the embodiments of the present disclosure, various implementations of the concepts of the present disclosure can be conceived, including but not limited to:

[0129] 1) An electronic device for a computing device, comprising:

[0130] The processing circuit is configured to:

[0131] Receiving a message including first indication information from a first user equipment (UE), the first indication information indicating that the first UE can transmit data of a first modality;

[0132] receiving a message including second indication information from a second UE, the second indication information indicating that the second UE desires to receive data in a second modality, wherein the second modality is different from the first modality; and

[0133] Based on the first indication information and the second indication information, a conversion model from data in the first modality to data in the second modality is determined.

[0134] 2) The electronic device as described in 1), wherein determining a conversion model from data in the first modality to data in the second modality comprises:

[0135] An artificial intelligence (AI) model is determined that takes data of a first modality as input and takes data of a second modality as output.

[0136] 3) The electronic device as described in 1), wherein determining a conversion model from data in the first modality to data in the second modality comprises:

[0137] determining a first artificial intelligence (AI) model that takes the data of the first modality as input and takes the data of the intermediate modality as output; and

[0138] A second AI model is determined that has the data of the intermediate modality as input and has the data of the second modality as output.

[0139] 4) The electronic device as described in any one of 1) to 3), wherein the processing circuit is further configured to:

[0140] receiving a request for data transmission from the first UE or the second UE; and

[0141] In response to the request, a data connection is established from the first UE to the second UE based on the determined transition model.

[0142] 5) The electronic device as described in 4), wherein the processing circuit is further configured to:

[0143] allocating computing resources to the determined transformation model; and

[0144] The transformation model is instantiated using the allocated computing resources.

[0145] 6) The electronic device as described in any one of 1) to 3), wherein the first mode and the second mode differ in one or more of the following:

[0146] the type of data;

[0147] Feature data category;

[0148] semantic data categories;

[0149] How the data is encoded;

[0150] Content characteristics of the data.

[0151] 7) An electronic device as described in any one of 1) to 3), wherein the electronic device is included in one or more of a road side computing unit (RSCU), a multi-access edge computing platform (MEC), a base station, and a core network.

[0152] 8) The electronic device as described in any one of 1) to 3), wherein the conversion model includes a transformer configured to couple the first mode to the second mode.

[0153] 9) The electronic device as described in any one of 1) to 3), wherein the conversion model includes a transcoder between a codec associated with the first modality and a codec associated with the second modality.

[0154] 10) The electronic device as described in 9), wherein the processing circuit is further configured to:

[0155] Instruct the first UE to adjust settings to generate data of a first modality.

[0156] 11) An electronic device for a user equipment (UE), comprising:

[0157] The processing circuit is configured to:

[0158] Sending a message including first indication information to a computing device, where the first indication information indicates that the UE can transmit data of a first modality; and

[0159] Data in a first modality is sent to the computing device, and the data in the first modality is converted by the computing device into data in a second modality that another UE expects to receive, wherein the second modality is different from the first modality.

[0160] 12) The electronic device as described in 11), wherein the processing circuit is further configured to:

[0161] The data of the first modality is generated using a specific AI model or codec.

[0162] 13) The electronic device as described in 12), wherein the processing circuit is further configured to:

[0163] receiving information about one or more AI models or codecs from the computing device; and

[0164] The specific AI model or codec is selected from the one or more AI models or codecs.

[0165] 14) An electronic device for a user equipment (UE), comprising:

[0166] The processing circuit is configured to:

[0167] Sending a message including second indication information to the computing device, where the second indication information indicates that the UE expects to receive data in a second modality; and

[0168] Data in a second modality is received from the computing device, the data being converted from data in a first modality transmitted by the computing device from another UE, wherein the second modality is different from the first modality.

[0169] 15) The electronic device as described in 14), wherein the processing circuit is further configured to:

[0170] The data of the second modality is processed using a specific AI model or codec.

[0171] 16) The electronic device as described in 14), wherein the processing circuit is further configured to:

[0172] receiving information about one or more AI models or codecs from the computing device; and

[0173] The specific AI model or codec is selected from the one or more AI models or codecs.

[0174] 17) The electronic device as described in 14), wherein the processing circuit is further configured to:

[0175] The second modality is changed in response to a setting of a user of the UE.

[0176] 18) A communication method, comprising:

[0177] Receiving a message including first indication information from a first user equipment (UE), the first indication information indicating that the first UE can transmit data of a first modality;

[0178] receiving a message including second indication information from a second UE, the second indication information indicating that the second UE desires to receive data in a second modality, wherein the second modality is different from the first modality; and

[0179] Based on the first indication information and the second indication information, a conversion model from data in the first modality to data in the second modality is determined.

[0180] 19) A communication method, comprising:

[0181] Sending a message including first indication information to a computing device, where the first indication information indicates that the UE can transmit data of a first modality; and

[0182] Data in a first modality is sent to the computing device, and the data in the first modality is converted by the computing device into data in a second modality that another UE expects to receive, wherein the second modality is different from the first modality.

[0183] 20) A communication method, comprising:

[0184] Sending a message including second indication information to the computing device, where the second indication information indicates that the UE expects to receive data in a second modality; and

[0185] Data in a second modality is received from the computing device, the data being converted from data in a first modality transmitted by the computing device from another UE, wherein the second modality is different from the first modality.

[0186] 21) A non-transitory computer-readable storage medium storing executable instructions, wherein the executable instructions, when executed, implement the communication method as described in 18) to 20).

[0187] Fig. 9 An example block diagram of a computer that can be implemented as a first UE 101, a computing device 102, or a second UE 103 according to an embodiment of the present disclosure is shown.

[0188] exist Fig. 9 In the embodiment, a central processing unit (CPU) 1301 executes various processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage section 1308 to a random access memory (RAM) 1303. In the RAM 1303, data required when the CPU 1301 executes various processes and the like is also stored as needed.

[0189] The CPU 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. To the bus 1304, an input / output interface 1305 is also connected.

[0190] The following components are connected to the input / output interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet.

[0191] A drive 1310 is also connected to the input / output interface 1305 as needed. A removable medium 1311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 1310 as needed so that a computer program read therefrom is installed into the storage section 1308 as needed.

[0192] In the case where the above-described series of processing is realized by software, a program constituting the software is installed from a network such as the Internet or a storage medium such as the removable medium 1311 .

[0193] Those skilled in the art should understand that such storage media is not limited to Fig. 9 The removable medium 1311 shown has a program stored therein and is distributed separately from the device to provide the program to the user. Examples of the removable medium 1311 include magnetic disks (including floppy disks (registered trademark)), optical disks (including compact disk read-only memory (CD-ROM) and digital versatile disks (DVD)), magneto-optical disks (including minidiscs (MD) (registered trademark)), and semiconductor memories. Alternatively, the storage medium may be the ROM 1302, a hard disk included in the storage section 1308, or the like, in which the program is stored and distributed to the user together with the device containing them.

[0194] [Application examples of the present disclosure]

[0195] The techniques described in this disclosure can be applied to a variety of products.

[0196] For example, the electronic device 600 according to an embodiment of the present disclosure may be implemented as various edge-side devices or network-side devices.

[0197] The communication method according to the embodiments of the present disclosure can be implemented by various base stations or user equipment; the methods and operations according to the embodiments of the present disclosure can be embodied as computer-executable instructions, stored in a non-temporary computer-readable storage medium, and can be executed by various base stations or user equipment to implement one or more functions described above.

[0198] The technology according to the embodiments of the present disclosure can be made into various computer program products, which can be used in various base stations or user equipments to implement one or more functions described above.

[0199] The base station mentioned in the present disclosure may be implemented as any type of base station, preferably, such as macro gNB and ng-eNB defined in the 5G NR standard of 3GPP. The gNB may be a gNB that covers a cell smaller than a macro cell, such as a pico gNB, a micro gNB, and a home (femto) gNB. Alternatively, the base station may be implemented as any other type of base station, such as a NodeB, an eNodeB, and a base transceiver station (BTS). The base station may also include: a main body configured to control wireless communication and one or more remote radio heads (RRHs) arranged at a place different from the main body, a wireless relay station, a drone tower, a control node in an automated factory, and the like.

[0200] The user equipment may be implemented as a mobile terminal (such as a smart phone, a tablet personal computer (PC), a notebook PC, a portable game terminal, a portable / dongle-type mobile router, and a digital camera) or a vehicle-mounted terminal (such as a car navigation device). The user equipment may also be implemented as a terminal that performs machine-to-machine (M2M) communication (also known as a machine type communication (MTC) terminal), a drone, a sensor and an actuator in an automated factory, etc. In addition, the user equipment may be a wireless communication module (such as an integrated circuit module including a single chip) installed on each of the above terminals.

[0201] Examples of base stations and user equipment to which the technology of the present disclosure can be applied are briefly introduced below.

[0202] First application example of base station

[0203] Fig.10 is a block diagram showing a first example of a schematic configuration of a base station to which the technology of the present disclosure can be applied. Fig.10 In the embodiment, the base station can be implemented as a gNB 1400. The gNB 1400 includes a plurality of antennas 1410 and a base station device 1420. The base station device 1420 and each antenna 1410 can be connected to each other via an RF cable. In one implementation, the gNB 1400 (or base station device 1420) here can correspond to the above-mentioned electronic device 600.

[0204] Antenna 1410 includes multiple antenna elements, such as multiple antenna arrays for massive MIMO. Antenna 1410 can be arranged into an antenna array matrix, for example, and used for base station device 1420 to send and receive wireless signals. For example, multiple antennas 1410 can be compatible with multiple frequency bands used by gNB 1400.

[0205] The base station device 1420 includes a controller 1421 , a memory 1422 , a network interface 1423 , and a wireless communication interface 1425 .

[0206] The controller 1421 may be, for example, a CPU or a DSP, and operates various functions of the higher layers of the base station device 1420. For example, the controller 1421 may include the processing circuit 601 described above, executing Figure 6B , or controls the various components of the electronic device 600. For example, the controller 1421 generates a data packet based on the data in the signal processed by the wireless communication interface 1425, and transmits the generated packet via the network interface 1423. The controller 1421 can bundle data from multiple baseband processors to generate a bundled packet, and transmit the generated bundled packet. The controller 1421 may have a logical function to perform the following control: the control is such as radio resource control, radio bearer control, mobility management, admission control and scheduling. The control can be performed in conjunction with a nearby gNB or core network node. The memory 1422 includes RAM and ROM, and stores programs executed by the controller 1421 and various types of control data (such as terminal lists, transmission power data, and scheduling data).

[0207] The network interface 1423 is a communication interface for connecting the base station device 1420 to the core network 1424 (e.g., a 5G core network). The controller 1421 can communicate with the core network node or another gNB via the network interface 1423. In this case, the gNB 1400 and the core network node or other gNBs can be connected to each other through logical interfaces such as NG interfaces and Xn interfaces. The network interface 1423 can also be a wired communication interface or a wireless communication interface for a wireless backhaul line. If the network interface 1423 is a wireless communication interface, the network interface 1423 can use a higher frequency band for wireless communication compared to the frequency band used by the wireless communication interface 1425.

[0208] The wireless communication interface 1425 supports any cellular communication scheme (such as 5G NR) and provides wireless connection to a terminal located in the cell of the gNB 1400 via the antenna 1410. The wireless communication interface 1425 may generally include, for example, a baseband (BB) processor 1426 and an RF circuit 1427. The BB processor 1426 may perform, for example, encoding / decoding, modulation / demodulation, and multiplexing / demultiplexing, and perform various types of signal processing of each layer (e.g., physical layer, MAC layer, RLC layer, PDCP layer, SDAP layer). Instead of the controller 1421, the BB processor 1426 may have part or all of the above-mentioned logical functions. The BB processor 1426 may be a memory for storing a communication control program, or a module including a processor configured to execute a program and related circuits. Updating the program may change the function of the BB processor 1426. The module may be a card or a blade inserted into a slot of the base station device 1420. Alternatively, the module may also be a chip mounted on a card or a blade. Meanwhile, the RF circuit 1427 may include, for example, a mixer, a filter, and an amplifier, and transmit and receive wireless signals via the antenna 1410. Fig.10 An example is shown in which one RF circuit 1427 is connected to one antenna 1410 , but the present disclosure is not limited to this illustration, and one RF circuit 1427 may be connected to a plurality of antennas 1410 at the same time.

[0209] like Fig.10 As shown, the wireless communication interface 1425 may include multiple BB processors 1426. For example, the multiple BB processors 1426 may be compatible with multiple frequency bands used by the gNB 1400. Fig.10 As shown, the wireless communication interface 1425 may include multiple RF circuits 1427. For example, the multiple RF circuits 1427 may be compatible with multiple antenna elements. Fig.10 An example is shown in which the wireless communication interface 1425 includes a plurality of BB processors 1426 and a plurality of RF circuits 1427 , but the wireless communication interface 1425 may also include a single BB processor 1426 or a single RF circuit 1427 .

[0210] exist Fig.10 In the gNB 1400 shown in FIG. Fig. 6AOne or more units included in the processing circuit 601 shown in FIG. 14 may be implemented in the wireless communication interface 1425. Alternatively, at least a portion of these components may be implemented in the controller 1421. For example, the gNB 1400 includes a portion (e.g., BB processor 1426) or the entirety of the wireless communication interface 1425, and / or a module including the controller 1421, and one or more components may be implemented in the module. In this case, the module may store a program for allowing the processor to function as one or more components (in other words, a program for allowing the processor to perform the operation of one or more components), and the program may be executed. As another example, a program for allowing the processor to function as one or more components may be installed in the gNB 1400, and the wireless communication interface 1425 (e.g., BB processor 1426) and / or the controller 1421 may execute the program. As described above, as a device including one or more components, the gNB 1400, the base station device 1420, or the module may be provided, and a program for allowing the processor to function as one or more components may be provided. In addition, a readable medium in which the program is recorded may be provided.

[0211] Second application example of base station

[0212] Fig.11 is a block diagram showing a second example of a schematic configuration of a base station to which the technology of the present disclosure can be applied. Fig.11 In the figure, the base station is shown as gNB 1530. The gNB 1530 includes multiple antennas 1540, a base station device 1550, and an RRH 1560. The RRH 1560 and each antenna 1540 can be connected to each other via an RF cable. The base station device 1550 and the RRH 1560 can be connected to each other via a high-speed line such as an optical fiber cable. In one implementation, the gNB 1530 (or base station device 1550) here can correspond to the above-mentioned electronic device 600.

[0213] Antenna 1540 includes multiple antenna elements, such as multiple antenna arrays for massive MIMO. Antenna 1540 can be arranged in an antenna array matrix, for example, and is used for base station device 1550 to send and receive wireless signals. For example, multiple antennas 1540 can be compatible with multiple frequency bands used by gNB 1530.

[0214] The base station device 1550 includes a controller 1551, a memory 1552, a network interface 1553, a wireless communication interface 1555, and a connection interface 1557. The controller 1551, the memory 1552, and the network interface 1553 are similar to the reference Fig.10 The controller 1421, memory 1422, and network interface 1423 described are the same.

[0215] The wireless communication interface 1555 supports any cellular communication scheme (such as 5G NR) and provides wireless communication to a terminal located in a sector corresponding to the RRH 1560 via the RRH 1560 and the antenna 1540. The wireless communication interface 1555 may generally include, for example, a BB processor 1556. In addition to the BB processor 1556 being connected to the RF circuit 1564 of the RRH 1560 via the connection interface 1557, the BB processor 1556 is connected to the reference RF circuit 1564 of the RRH 1560. Fig.10 The same as the BB processor 1426 described above. Fig.11 As shown, the wireless communication interface 1555 may include multiple BB processors 1556. For example, the multiple BB processors 1556 may be compatible with multiple frequency bands used by the gNB 1530. Fig.11 An example is shown in which the wireless communication interface 1555 includes a plurality of BB processors 1556 , but the wireless communication interface 1555 may also include a single BB processor 1556 .

[0216] The connection interface 1557 is an interface for connecting the base station device 1550 (wireless communication interface 1555) to the RRH 1560. The connection interface 1557 may also be a communication module for connecting the base station device 1550 (wireless communication interface 1555) to the RRH 1560 for communication in the above-mentioned high-speed line.

[0217] The RRH 1560 includes a connection interface 1561 and a wireless communication interface 1563 .

[0218] The connection interface 1561 is an interface for connecting the RRH 1560 (wireless communication interface 1563) to the base station device 1550. The connection interface 1561 may also be a communication module for communication in the above-mentioned high-speed line.

[0219] The wireless communication interface 1563 transmits and receives wireless signals via the antenna 1540. The wireless communication interface 1563 may generally include, for example, an RF circuit 1564. The RF circuit 1564 may include, for example, a mixer, a filter, and an amplifier, and transmits and receives wireless signals via the antenna 1540. Fig.11 An example is shown in which one RF circuit 1564 is connected to one antenna 1540 , but the present disclosure is not limited to this illustration, and one RF circuit 1564 may be connected to a plurality of antennas 1540 at the same time.

[0220] like Fig.11 As shown, the wireless communication interface 1563 may include multiple RF circuits 1564. For example, the multiple RF circuits 1564 may support multiple antenna elements. Fig.11An example is shown in which the wireless communication interface 1563 includes a plurality of RF circuits 1564 , but the wireless communication interface 1563 may also include a single RF circuit 1564 .

[0221] exist Fig.11 In the gNB 1500 shown in FIG. 1 , one or more units included in the processing circuit 601 may be implemented in the wireless communication interface 1525. Alternatively, at least a portion of these components may be implemented in the controller 1521. For example, the gNB 1500 includes a portion (e.g., BB processor 1526) or the entirety of the wireless communication interface 1525, and / or a module including the controller 1521, and one or more components may be implemented in the module. In this case, the module may store a program for allowing the processor to function as one or more components (in other words, a program for allowing the processor to perform the operation of one or more components), and the program may be executed. As another example, a program for allowing the processor to function as one or more components may be installed in the gNB 1500, and the wireless communication interface 1525 (e.g., BB processor 1526) and / or the controller 1521 may execute the program. As described above, as a device including one or more components, the gNB 1500, the base station device 1520, or the module may be provided, and a program for allowing the processor to function as one or more components may be provided. In addition, a readable medium in which the program is recorded may be provided.

[0222] First application example of user equipment

[0223] Fig.12 1600 is a block diagram showing an example of a schematic configuration of a smart phone 1600 to which the technology of the present disclosure can be applied. In one example, the smart phone 1600 can be implemented as the electronic device 700 or 800.

[0224] The smart phone 1600 includes a processor 1601, a memory 1602, a storage device 1603, an external connection interface 1604, a camera 1606, a sensor 1607, a microphone 1608, an input device 1609, a display device 1610, a speaker 1611, a wireless communication interface 1612, one or more antenna switches 1615, one or more antennas 1616, a bus 1617, a battery 1618 and an auxiliary controller 1619.

[0225] The processor 1601 may be, for example, a CPU or a system on a chip (SoC), and controls the functions of the application layer and other layers of the smartphone 1600. The processor 1601 may include or function as the processing circuit 701 or 801 described with reference to the drawings. The memory 1602 includes a RAM and a ROM, and stores data and a program executed by the processor 1601. The storage device 1603 may include a storage medium such as a semiconductor memory and a hard disk. The external connection interface 1604 is an interface for connecting an external device (such as a memory card and a universal serial bus (USB) device) to the smartphone 1600.

[0226] The camera 1606 includes an image sensor such as a charge coupled device (CCD) and a complementary metal oxide semiconductor (CMOS), and generates a captured image. The sensor 1607 may include a group of sensors such as a measurement sensor, a gyro sensor, a geomagnetic sensor, and an acceleration sensor. The microphone 1608 converts the sound input to the smart phone 1600 into an audio signal. The input device 1609 includes, for example, a touch sensor, a keypad, a keyboard, a button, or a switch configured to detect a touch on the screen of the display device 1610, and receives an operation or information input from a user. The display device 1610 includes a screen such as a liquid crystal display (LCD) and an organic light emitting diode (OLED) display, and displays an output image of the smart phone 1600. The speaker 1611 converts an audio signal output from the smart phone 1600 into sound.

[0227] The wireless communication interface 1612 supports any cellular communication scheme (such as 4G LTE or 5G NR, etc.) and performs wireless communication. The wireless communication interface 1612 may generally include, for example, a BB processor 1613 and an RF circuit 1614. The BB processor 1613 may perform, for example, encoding / decoding, modulation / demodulation, and multiplexing / demultiplexing, and perform various types of signal processing for wireless communication. Meanwhile, the RF circuit 1614 may include, for example, a mixer, a filter, and an amplifier, and transmit and receive wireless signals via an antenna 1616. The wireless communication interface 1612 may be a chip module on which the BB processor 1613 and the RF circuit 1614 are integrated. Fig.12 As shown, the wireless communication interface 1612 may include multiple BB processors 1613 and multiple RF circuits 1614. Fig.12 An example is shown in which the wireless communication interface 1612 includes a plurality of BB processors 1613 and a plurality of RF circuits 1614 , but the wireless communication interface 1612 may also include a single BB processor 1613 or a single RF circuit 1614 .

[0228] In addition, in addition to the cellular communication scheme, the wireless communication interface 1612 can support other types of wireless communication schemes, such as short-range wireless communication schemes, near field communication schemes, and wireless local area network (LAN) schemes. In this case, the wireless communication interface 1612 can include a BB processor 1613 and an RF circuit 1614 for each wireless communication scheme.

[0229] Each of the antenna switches 1615 switches a connection destination of the antenna 1616 between a plurality of circuits (eg, circuits for different wireless communication schemes) included in the wireless communication interface 1612 .

[0230] Antenna 1616 includes multiple antenna elements, such as multiple antenna arrays for massive MIMO. Antenna 1616 can be arranged in an antenna array matrix, for example, and used for wireless communication interface 1612 to transmit and receive wireless signals. Smartphone 1600 may include one or more antenna panels (not shown).

[0231] In addition, the smartphone 1600 may include an antenna 1616 for each wireless communication scheme. In this case, the antenna switch 1615 may be omitted from the configuration of the smartphone 1600.

[0232] The bus 1617 connects the processor 1601, the memory 1602, the storage device 1603, the external connection interface 1604, the camera 1606, the sensor 1607, the microphone 1608, the input device 1609, the display device 1610, the speaker 1611, the wireless communication interface 1612, and the auxiliary controller 1619 to each other. Fig.12 The various blocks of the smartphone 1600 shown are supplied with power, the feed lines being partially shown as dashed lines in the figure. The auxiliary controller 1619 operates the minimum necessary functions of the smartphone 1600, for example in a sleep mode.

[0233] exist Fig.12In the smart phone 1600 shown in FIG. 1 , one or more units included in the processing circuit 701 or 801 may be implemented in the wireless communication interface 1612. Alternatively, at least a portion of these components may be implemented in the processor 1601 or the auxiliary controller 1619. As an example, the smart phone 1600 includes a portion (e.g., BB processor 1613) or the entirety of the wireless communication interface 1612, and / or a module including the processor 1601 and / or the auxiliary controller 1619, and one or more components may be implemented in the module. In this case, the module may store a program that allows the processor to function as one or more components (in other words, a program for allowing the processor to perform the operation of one or more components), and the program may be executed. As another example, a program for allowing the processor to function as one or more components may be installed in the smart phone 1600, and the wireless communication interface 1612 (e.g., BB processor 1613), the processor 1601, and / or the auxiliary controller 1619 may execute the program. As described above, as a device including one or more components, the smart phone 1600 or a module may be provided, and a program for allowing a processor to function as one or more components may be provided. In addition, a readable medium having the program recorded therein may be provided.

[0234] Second application example of user equipment

[0235] Fig.13 17 is a block diagram showing an example of a schematic configuration of a car navigation device 1720 to which the technology of the present disclosure can be applied. The car navigation device 1720 can be implemented as a reference Fig. 7A The electronic device 700 described or referred to Fig. 8A The electronic device 800 described. The car navigation device 1720 includes a processor 1721, a memory 1722, a global positioning system (GPS) module 1724, a sensor 1725, a data interface 1726, a content player 1727, a storage medium interface 1728, an input device 1729, a display device 1730, a speaker 1731, a wireless communication interface 1733, one or more antenna switches 1736, one or more antennas 1737, and a battery 1738. In an example, the car navigation device 1720 can be implemented as any of the electronic devices 1000, 2000, 3000, 4000 described in the present disclosure.

[0236] The processor 1721 may be, for example, a CPU or a SoC, and controls a navigation function and other functions of the car navigation device 1720. The memory 1722 includes a RAM and a ROM, and stores data and a program executed by the processor 1721.

[0237] The GPS module 1724 measures the position (such as latitude, longitude and altitude) of the car navigation device 1720 using GPS signals received from GPS satellites. The sensor 1725 may include a group of sensors such as a gyro sensor, a geomagnetic sensor and an air pressure sensor. The data interface 1726 is connected to, for example, the vehicle network 1741 via an unshown terminal and acquires data (such as vehicle speed data) generated by the vehicle.

[0238] The content player 1727 reproduces content stored in a storage medium such as a CD and a DVD, which is inserted into the storage medium interface 1728. The input device 1729 includes, for example, a touch sensor, a button, or a switch configured to detect a touch on the screen of the display device 1730, and receives an operation or information input from a user. The display device 1730 includes a screen such as an LCD or an OLED display, and displays an image of a navigation function or reproduced content. The speaker 1731 outputs a sound of a navigation function or reproduced content.

[0239] The wireless communication interface 1733 supports any cellular communication scheme (such as 4G LTE or 5G NR) and performs wireless communication. The wireless communication interface 1733 may generally include, for example, a BB processor 1734 and an RF circuit 1735. The BB processor 1734 may perform, for example, encoding / decoding, modulation / demodulation, and multiplexing / demultiplexing, and perform various types of signal processing for wireless communication. Meanwhile, the RF circuit 1735 may include, for example, a mixer, a filter, and an amplifier, and transmit and receive wireless signals via an antenna 1737. The wireless communication interface 1733 may also be a chip module on which the BB processor 1734 and the RF circuit 1735 are integrated. Fig.13 As shown, the wireless communication interface 1733 may include multiple BB processors 1734 and multiple RF circuits 1735. Fig.13 An example is shown in which the wireless communication interface 1733 includes a plurality of BB processors 1734 and a plurality of RF circuits 1735 , but the wireless communication interface 1733 may also include a single BB processor 1734 or a single RF circuit 1735 .

[0240] In addition, in addition to the cellular communication scheme, the wireless communication interface 1733 can support other types of wireless communication schemes, such as short-range wireless communication schemes, near field communication schemes, and wireless LAN schemes. In this case, for each wireless communication scheme, the wireless communication interface 1733 can include a BB processor 1734 and an RF circuit 1735.

[0241] Each of the antenna switches 1736 switches a connection destination of the antenna 1737 between a plurality of circuits included in the wireless communication interface 1733 , such as circuits for different wireless communication schemes.

[0242] The antenna 1737 includes multiple antenna elements, such as multiple antenna arrays for massive MIMO. The antenna 1737 can be arranged in an antenna array matrix, for example, and is used for the wireless communication interface 1733 to transmit and receive wireless signals.

[0243] In addition, the car navigation device 1720 may include an antenna 1737 for each wireless communication scheme. In this case, the antenna switch 1736 may be omitted from the configuration of the car navigation device 1720.

[0244] Battery 1738 is fed via a feed line to Fig.13 The respective blocks of the illustrated car navigation device 1720 are supplied with electric power, and feed lines are partially illustrated as dotted lines in the figure. The battery 1738 accumulates electric power supplied from the vehicle.

[0245] exist Fig.13 In the car navigation device 1720 shown in FIG. 1 , one or more units included in the processing circuit 701 or 801 may be implemented in the wireless communication interface 1733. Alternatively, at least a portion of these components may be implemented in the processor 1721. As an example, the car navigation device 1720 includes a portion (e.g., BB processor 1734) or the entirety of the wireless communication interface 1733, and / or a module including the processor 1721, and one or more components may be implemented in the module. In this case, the module may store a program that allows the processor to function as one or more components (in other words, a program for allowing the processor to perform the operation of one or more components), and the program may be executed. As another example, a program for allowing the processor to function as one or more components may be installed in the car navigation device 1720, and the wireless communication interface 1733 (e.g., BB processor 1734) and / or the processor 1721 may execute the program. As described above, as a device including one or more components, the car navigation device 1720 or a module may be provided, and a program for allowing the processor to function as one or more components may be provided. In addition, a readable medium in which the program is recorded may be provided.

[0246] The technology of the present disclosure may also be implemented as an in-vehicle system (or vehicle) 1740 including a car navigation device 1720, an in-vehicle network 1741, and one or more blocks in a vehicle module 1742. The vehicle module 1742 generates vehicle data (such as vehicle speed, engine speed, and fault information), and outputs the generated data to the in-vehicle network 1741.

[0247] The exemplary embodiments of the present disclosure are described above with reference to the accompanying drawings, but the present disclosure is certainly not limited to the above examples. Those skilled in the art may obtain various changes and modifications within the scope of the appended claims, and it should be understood that these changes and modifications will naturally fall within the technical scope of the present disclosure.

[0248] For example, a plurality of functions included in one unit in the above embodiments may be implemented by separate devices. Alternatively, a plurality of functions implemented by a plurality of units in the above embodiments may be implemented by separate devices, respectively. In addition, one of the above functions may be implemented by a plurality of units. Needless to say, such a configuration is included in the technical scope of the present disclosure.

[0249] In this specification, the steps described in the flowchart include not only the processing performed in time series in the order described, but also the processing performed in parallel or individually rather than necessarily in time series. In addition, even in the steps processed in time series, it goes without saying that the order can be appropriately changed.

[0250] Although the present disclosure and its advantages have been described in detail, it should be understood that various changes, substitutions and transformations can be made without departing from the spirit and scope of the present disclosure as defined by the appended claims. Moreover, the terms "including", "comprising" or any other variants of the embodiments of the present disclosure are intended to cover non-exclusive inclusions, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of further restrictions, the elements defined by the statement "including one..." do not exclude the presence of other identical elements in the process, method, article or equipment including the elements.

Claims

1. An electronic device for a computing device, comprising: The processing circuit is configured to: Receiving a message including first indication information from a first user equipment (UE), the first indication information indicating that the first UE can transmit data of a first modality; receiving a message including second indication information from a second UE, the second indication information indicating that the second UE desires to receive data in a second modality, wherein the second modality is different from the first modality; as well as Based on the first indication information and the second indication information, a conversion model from data in the first modality to data in the second modality is determined.

2. The electronic device according to claim 1, wherein: Determining a transformation model from data of a first modality to data of a second modality includes: An artificial intelligence (AI) model is determined that takes data of a first modality as input and takes data of a second modality as output.

3. The electronic device according to claim 1, wherein: Determining a transformation model from data of a first modality to data of a second modality includes: determining a first artificial intelligence (AI) model that takes the data of the first modality as input and takes the data of the intermediate modality as output; and A second AI model is determined that has the data of the intermediate modality as input and has the data of the second modality as output.

4. The electronic device according to any one of claims 1 to 3, wherein: The processing circuit is further configured to: receiving a request for data transmission from the first UE or the second UE; and In response to the request, a data connection is established from the first UE to the second UE based on the determined transition model.

5. The electronic device according to claim 4, wherein: The processing circuit is further configured to: allocating computing resources to the determined transformation model; and The transformation model is instantiated using the allocated computing resources.

6. The electronic device according to any one of claims 1 to 3, wherein: The first modality and the second modality differ in one or more of the following: the type of data; Feature data category; semantic data categories; How the data is encoded; Content characteristics of the data.

7. The electronic device according to any one of claims 1 to 3, wherein: The electronic device is included in one or more of a road side computing unit (RSCU), a multi-access edge computing platform (MEC), a base station, and a core network.

8. The electronic device according to any one of claims 1 to 3, wherein: The conversion model includes a transformer configured to couple a first mode to a second mode.

9. The electronic device according to any one of claims 1 to 3, wherein: The conversion model includes a transcoder between a codec associated with a first modality and a codec associated with a second modality.

10. The electronic device according to claim 9, wherein: The processing circuit is further configured to: Instruct the first UE to adjust settings to generate data of a first modality.

Images