Method and apparatus for integrating communication and sensing

Abstract

Embodiments of the present application relate to the field of communications, and provide a method and apparatus for integrating communication and sensing, which are capable of implementing communication and sensing integration on the basis of limited processing resources of a terminal device, thereby reducing the costs of the terminal device implementing communication and sensing integration. The method comprises: a terminal device reports terminal capability information to a network device, the terminal capability information comprising information used for indicating that the terminal device supports simultaneous execution of a sensing task and a communication task; the terminal device executes a sensing task within a first time period, and executes a first-level communication task within the first time period; and the terminal device executes a second-level communication task within a second time period, wherein the terminal device does not execute a sensing task within the second time period, and processing resources required by the first-level communication task are less than processing resources required by the second-level communication task.

Description

Method and device for communication and sensing fusion

[0001] The present application claims priority to the Chinese Patent Application No. 202410897870.7, filed on July 5, 2024, and entitled "Method and device for communication and sensing fusion", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] Embodiments of the present application relate to the field of communication, and in particular to a method and device for communication and sensing fusion. BACKGROUND

[0003] With the development of mobile communication technology, higher frequency bands, wider bandwidths, and larger antenna arrays make high-precision and high-resolution sensing possible, thereby enabling integrated sensing and communication (ISAC).

[0004] Among them, sensing refers to determining the position, distance, speed, etc. of a target object through the transmission and reception of wireless signals, or detecting, tracking, identifying, imaging, etc. of a target object, event or environment, etc.

[0005] However, current sensing is mostly integrated on the network side (e.g., the base station side), and there is less sensing on the terminal side. The prior art does not provide a related solution for how the terminal performs communication and sensing fusion. SUMMARY

[0006] Embodiments of the present application provide a method and device for communication and sensing fusion, which can implement communication and sensing fusion based on the limited processing resources of a terminal device, thereby reducing the cost of implementing communication and sensing fusion by the terminal device.

[0007] To achieve the above-mentioned purpose, embodiments of the present application adopt the following technical solutions:

[0008] In a first aspect, a method for communication and sensing fusion is provided, applied to a terminal device, comprising: reporting, by the terminal device, terminal capability information to a network device, the terminal capability information comprising information indicating that the terminal device supports simultaneously performing a sensing task and a communication task; performing, by the terminal device, the sensing task in a first time period, and performing a first level of communication task in the first time period; performing, by the terminal device, a second level of communication task in a second time period, and the terminal device does not perform the sensing task in the second time period, and the first level of communication task requires less processing resource than the second level of communication task.

[0009] Based on the method provided in the embodiments of the present application, the terminal device can simultaneously perform the sensing task and the communication task (the first level communication task), and perform different levels of communication tasks (the first level communication task and the second level communication task) in time sharing manner, so that the limited processing resources can be fully utilized to realize the fusion of communication and sensing, and the cost of realizing the fusion of communication and sensing of the terminal device is reduced.

[0010] In a possible implementation, the processing resources include one or more of an antenna, a processor, a memory, and a sensor. The processor may, for example, include a central processing unit (CPU), a graphics processing unit (GPU), etc., and the sensor may, for example, include a gyroscope, a distance sensor, an acceleration sensor, etc. In this way, the terminal device can fully utilize the limited processing resources (antenna, processor, memory, sensor, etc.) to realize the fusion of communication and sensing, and reduce the cost of realizing the fusion of communication and sensing of the terminal device.

[0011] In a possible implementation, the terminal capability information further includes information indicating that the terminal device can perform the first level communication task when performing the sensing task, and can perform the second level communication task when not performing the sensing task. In this way, the terminal device can simultaneously perform the sensing task and the communication task (the first level communication task), and perform different levels of communication tasks (the first level communication task and the second level communication task) in time sharing manner, so that the limited processing resources can be fully utilized to realize the fusion of communication and sensing, and the cost of realizing the fusion of communication and sensing of the terminal device is reduced.

[0012] In a possible implementation, the sensing task is triggered by the terminal device, or the sensing task is indicated by the network device. In this way, the terminal device can perform the first level communication task when performing the sensing task triggered by itself or the sensing task indicated by the network device, and can perform the second level communication task when not performing the sensing task triggered by itself or the sensing task indicated by the network device, so that the limited processing resources can be fully utilized to realize the fusion of communication and sensing, and the cost of realizing the fusion of communication and sensing of the terminal device is reduced.

[0013] In a possible implementation, the terminal capability information further includes information indicating whether the terminal device supports cooperative sensing with the network device. The cooperative sensing refers to that the UE can perform the sensing task indicated by the base station by means of its own processing resources.

[0014] In a possible implementation, in the case that the terminal device supports cooperative sensing with the network device and the sensing task is indicated by the network device, the method further includes: receiving, by the terminal device, indication information from the network device, the indication information being used to indicate a time and / or a frequency band for performing the sensing task. In this way, the terminal device can perform the sensing task in the time period and the frequency band indicated by the network device (for example, a base station). When performing the sensing task, a first level of communication task can be performed, and when not performing the sensing task, a second level of communication task can be performed, so that the limited processing resource can be fully utilized to implement the fusion of communication and sensing, and the cost of implementing the fusion of communication and sensing by the terminal device is reduced.

[0015] In a possible implementation, in the case that the sensing task is triggered by the terminal device, the method further includes: sending, by the terminal device, information about a time and / or a frequency band for performing the sensing task to the network device. In this way, the network device (for example, a base station) can learn the time and / or the frequency band for performing the sensing task by the terminal device. Therefore, the network device can perform different levels of communication task scheduling (which can also be referred to as data scheduling) according to whether the terminal device performs the sensing task (the sensing task triggered by the terminal device itself) in different time periods.

[0016] In a possible implementation, the terminal capability information further includes at least one of the following: a minimum time unit for performing the sensing task, a task type of the sensing task, and an accuracy of performing the sensing task. The minimum time unit for performing the sensing task refers to the shortest time required by the UE to perform one sensing task. The task type of the sensing task may, for example, include positioning, imaging, map construction, posture recognition, action recognition, gesture sensing, expression sensing, and environment sensing. The accuracy of performing the sensing task refers to the accuracy of the sensing result obtained by performing the sensing task. For example, in the case that the sensing task is positioning, the accuracy of the sensing result (that is, the positioning result) can reach the accuracy of meters (m) or centimeters (cm); for another example, in the case that the sensing task is posture recognition, the sensing result (that is, the posture recognition result) can identify dozens of postures, and the recognition accuracy is high.

[0017] In a possible implementation, the method further includes: reporting, by the terminal device, the sensing result after performing the sensing task. The UE can report the sensing result to the base station based on the time and the frequency band for reporting the sensing result indicated by the base station.

[0018] In a second aspect, a method for performing a communication task is provided, and applied to a network device, including: receiving, by the network device, terminal capability information from a terminal device, the terminal capability information including information indicating that the terminal device supports simultaneous performance of a sensing task and a communication task; scheduling, by the network device, the terminal device to perform a first level of the communication task in a first time period, the first time period being a time period in which the terminal device performs the sensing task; and scheduling, by the network device, the terminal device to perform a second level of the communication task in a second time period, the terminal device not performing the sensing task in the second time period, the first level of the communication task requiring less processing resource than the second level of the communication task.

[0019] Based on the method provided in the embodiments of the present application, the network device (for example, a base station) can perform different levels of communication task scheduling (also referred to as data scheduling) according to whether the terminal device performs a sensing task (a sensing task indicated by the network device or a sensing task triggered by the terminal device itself) in different time periods. In a time period (for example, a first time period) in which the terminal device performs the sensing task, the base station can schedule the terminal device to perform a first level of the communication task; in a time period (for example, a second time period) in which the terminal device does not perform the sensing task, the base station can schedule the terminal device to perform a second level of the communication task. The first level of the communication task requires less processing resource than the second level of the communication task. In this way, the terminal device can simultaneously perform the sensing task and the communication task (the first level of the communication task), and perform different levels of the communication task (the first level of the communication task and the second level of the communication task) in time, which can fully utilize limited processing resources to achieve communication and sensing fusion, and reduce the cost of the terminal device to achieve communication and sensing fusion.

[0020] In a possible implementation, the processing resource includes one or more of an antenna, a processor, a memory, and a sensor.

[0021] In a possible implementation, the terminal capability information further includes information indicating that the terminal device can perform the first level of the communication task when performing the sensing task, and can perform the second level of the communication task when not performing the sensing task.

[0022] In a possible implementation, the sensing task is triggered by the terminal device, or the sensing task is indicated by the network device.

[0023] In a possible implementation, the terminal capability information further includes information indicating whether the terminal device supports cooperative sensing with the network device.

[0024] In a possible implementation, in the case that the terminal device supports cooperative sensing with the network device and the sensing task is indicated by the network device, the method further includes: the network device sending indication information to the terminal device, the indication information being used to indicate a time and / or a frequency band for performing the sensing task.

[0025] In a possible implementation, in the case that the terminal device triggers the sensing task, the method further includes: the network device receiving, from the terminal device, information about a time and / or a frequency band for the terminal device to perform the sensing task.

[0026] In a possible implementation, the terminal capability information further includes at least one of: a minimum time unit for performing the sensing task, a task type of the sensing task, and a precision for performing the sensing task.

[0027] In a possible implementation, the method further includes: the network device receiving, from the terminal device, a sensing result of the sensing task.

[0028] In a third aspect, a computer readable storage medium is provided, which includes computer instructions. When the computer instructions are run on a terminal device, the terminal device performs the method in the first aspect and any possible implementation thereof. When the computer instructions are run on a network device, the network device performs the method in the second aspect and any possible implementation thereof.

[0029] In a fourth aspect, a computer program product is provided, which, when run on a computer, causes the computer to perform the method in the first aspect and any possible implementation thereof.

[0030] In a fifth aspect, an embodiment of the present application provides a communication apparatus, including a processor and a memory coupled with the processor, and the memory stores program instructions, which, when executed by the processor, cause the communication apparatus to implement the method in the first aspect or the second aspect and any possible implementation thereof. The communication apparatus can be a terminal device or a network device, or can be a component of the terminal device or the network device, such as a chip.

[0031] In a sixth aspect, an embodiment of the present application provides a communication apparatus, which can be a terminal device or a network device. The communication apparatus can be divided into different logical units or modules according to functions, and each unit or module performs different functions, so that the communication apparatus performs the method in the first aspect or the second aspect and any possible implementation thereof.

[0032] In a seventh aspect, an embodiment of the present application provides a communication system, comprising a terminal device and a network device, wherein the terminal device can perform the method in the first aspect and any possible design of the first aspect, and the network device can perform the method in the second aspect and any possible design of the second aspect.

[0033] In an eighth aspect, the present application provides a chip system, comprising one or more interface circuits and one or more processors. The interface circuit and the processor are connected through a line.

[0034] The chip system can be applied to a terminal device comprising a communication module and a memory. The interface circuit is configured to receive a signal from the memory of the terminal device and send the received signal to the processor, wherein the signal comprises computer instructions stored in the memory. When the processor executes the computer instructions, the terminal device can perform the method in the first aspect and any possible design of the first aspect; or the chip system can be applied to a network device comprising a communication module and a memory. The interface circuit is configured to receive a signal from the memory of the network device and send the received signal to the processor, wherein the signal comprises computer instructions stored in the memory. When the processor executes the computer instructions, the network device can perform the method in the first aspect and any possible design of the first aspect.

[0035] It can be understood that the beneficial effects of the computer readable storage medium in the third aspect, the computer program product in the fourth aspect, the communication apparatus in the fifth aspect and the sixth aspect, the communication system in the seventh aspect and the chip system in the eighth aspect can refer to the beneficial effects in the first aspect or the second aspect and any possible design of the first aspect or the second aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0036] FIG. 1 is a schematic diagram of a communication system architecture according to an embodiment of the present application;

[0037] FIG. 2 is a schematic diagram of a structure of a terminal device according to an embodiment of the present application;

[0038] FIG. 3 is a schematic diagram of a structure of a network device according to an embodiment of the present application;

[0039] FIG. 4 is a flowchart of a method of communication and perception fusion according to an embodiment of the present application;

[0040] FIG. 5 is a flowchart of another method of communication and perception fusion according to an embodiment of the present application;

[0041] FIG. 6 is a schematic diagram of an antenna array of a terminal device according to an embodiment of the present application;

[0042] FIG. 7 is a schematic diagram of a terminal device performing a communication task and a sensing task according to an embodiment of the present application;

[0043] FIG. 8 is a schematic diagram of another method of communication and sensing fusion according to an embodiment of the present application;

[0044] FIG. 9 is a schematic diagram of another method of communication and sensing fusion according to an embodiment of the present application;

[0045] FIG. 10 is a schematic diagram of another method of communication and sensing fusion according to an embodiment of the present application;

[0046] FIG. 11 is a schematic diagram of another method of communication and sensing fusion according to an embodiment of the present application;

[0047] FIG. 12 is a schematic diagram of a chip system according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] For the sake of clear and concise description of the following embodiments, first, a brief introduction of related concepts or technologies is given:

[0049] Direction of arrival (DOA): refers to the direction of arrival of a spatial signal (the direction angle of each signal arriving at the array reference element, referred to as the direction of arrival). DOA is an important parameter in spatial spectrum estimation, and spatial spectrum estimation can be used to estimate the spatial parameters or source positions of signals.

[0050] Multiple signal classification (MUSIC) algorithm: a subspace method in DOA estimation, which can be used to estimate the direction or frequency of a signal source in the field of array signal processing. The MUSIC algorithm can distinguish multiple signal sources from different directions in the received signal.

[0051] Integrated sensing and communication (ISAC): In an ISAC system, both communication and sensing performance criteria are included. On the one hand, the entire communication network can serve as a giant sensor, with network elements transmitting and receiving wireless signals, and using the transmission, reflection, and scattering of radio waves to better sense and understand the physical world. By obtaining distance, velocity, and angle information from wireless signals, high-precision positioning, gesture capture, motion recognition, detection and tracking of passive objects, imaging, and environmental reconstruction can be provided, and a "network as a sensor" can be achieved. On the other hand, the high-precision positioning, imaging, and environmental reconstruction capabilities provided by sensing can help improve communication performance, such as more accurate beamforming, faster beam failure recovery, lower overhead for terminal channel state information (CSI) tracking, and "sensing-aided communication". Sensing is also a "new channel" that connects the physical world and the biological world to the digital world. Therefore, real-time network sensing can create a parallel digital world for the physical world, which is extremely important for the realization of the concept of "digital twin" in the future.

[0052] Currently, the 3rd generation partnership project (3GPP) has initially started research on ISAC application scenarios and potential needs based on 5G-Advanced air interfaces. For future 6th generation mobile communication technology (6G) scenarios, communication and sensing are also important research directions and are a widely recognized key technology trend for 6G. In a 6G mobile communication system, higher frequency bands (millimeter waves and even terahertz), wider bandwidths, and larger-scale antenna arrays make high-precision, high-resolution sensing possible, thereby enabling integrated sensing and communication in a system, with communication and sensing functions complementing each other. Future 6G ISAC system application scenarios may include ultra-high-precision positioning and tracking, synchronous imaging, map construction, and human sensory enhancement.

[0053] However, current perception is mostly integrated at the base station side, and there is less research on the terminal device side. The prior art does not provide a related scheme for how the terminal performs communication and perception fusion. Moreover, it is a great challenge for the terminal device to support both communication and perception, and due to the size and cost of the terminal device, it is difficult to integrate more processing resources to support communication and perception. Therefore, how to realize communication and perception fusion under the condition of limited processing resources of the terminal device and reduce the cost of the terminal device is a problem to be solved.

[0054] The present application provides a method and device for communication and perception fusion, which can realize communication and perception fusion based on the limited processing resources of the terminal device and reduce the cost of the terminal device for realizing communication and perception fusion.

[0055] The technical scheme of the embodiments of the present application can be applied to various communication systems. For example: long term evolution (LTE) system, LTE frequency division duplex (FDD) system, LTE time division duplex (TDD), worldwide interoperability for microwave access (WiMAX) communication system, 5G mobile communication system or new radio (NR), etc. The 5G mobile communication system described in the present application includes non-standalone (NSA) 5G mobile communication system and / or standalone (SA) 5G mobile communication system. The technical scheme provided by the present application can also be applied to future communication systems, such as the sixth generation mobile communication system. The communication system can also be a future evolved public land mobile network (PLMN) network, device-to-device (D2D) network, machine to machine (M2M) network, internet of things (IoT) network or other network.

[0056] FIG. 1 shows a communication system according to an embodiment of the present application. The communication system can include a network device 200 and one or more terminal devices 100 (only one is shown in FIG. 1) connected to the network device 200. Data transmission can be performed between the network device (also referred to as network side device) and the terminal device (also referred to as terminal side device).

[0057] Network device 200 can be a device capable of communicating with terminal device 100. For example, network device 200 can be a base station, which can be an evolved Node B (eNB or eNodeB) in LTE, a base station in NR, a relay station or access point, or a base station in a future network, etc., and this application embodiment does not limit it. In NR, a base station can also be called a transmission reception point (TRP) or gNB. In this application embodiment, the network device can be a separately sold network device, such as a base station, or it can be a chip in the network device that implements the corresponding functions. In this application embodiment, the chip system can be composed of chips, or it can include chips and other discrete components. In the technical solutions provided in this application embodiment, the network device is used as an example to describe the technical solutions provided in this application embodiment.

[0058] In this application embodiment, the terminal device 100 can also be referred to as a terminal, which can be a device with wireless transceiver capabilities. The terminal can be deployed on land, including indoors or outdoors, handheld or vehicle-mounted; it can also be deployed on water (such as on ships); and it can also be deployed in the air (e.g., on airplanes, balloons, and satellites). The terminal device can be user equipment (UE). The UE includes handheld devices, vehicle-mounted devices, wearable devices, or computing devices with wireless communication capabilities. For example, the UE can be a mobile phone, tablet computer, or computer with wireless transceiver capabilities. The terminal device can also be a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in autonomous driving, a wireless terminal in telemedicine, a wireless terminal in a smart grid, a wireless terminal in a smart city, a wireless terminal in a smart home, etc. In this application embodiment, the terminal device can be a separately sold terminal or a chip within a terminal. In the technical solutions provided in the embodiments of this application, the terminal device used to implement the functions of the terminal is used as an example to describe the technical solutions provided in the embodiments of this application.

[0059] The network device 200 or the terminal device 100 in FIG. 1 of the embodiments of the present application can be implemented by one device, or can be one functional module in one device, and the embodiments of the present application do not make a specific limitation in this regard. It can be understood that the above functions can be network elements in a hardware device, or software functions running on a special hardware, or virtualized functions instantiated on a platform (for example, a cloud platform), or a chip system. In the embodiments of the present application, the chip system can be composed of a chip, or can include a chip and other discrete devices.

[0060] FIG. 2 is a structural schematic diagram of a terminal device (for example, the terminal device 100) provided by the embodiments of the present application. The terminal device 100 can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, an in-vehicle device, a smart home device, and / or a smart city device, and the embodiments of the present application do not make a special limitation on the specific type of the terminal device.

[0061] Referring to FIG. 2, the terminal device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a loudspeaker 170A, a receiver 170B, a microphone 170C, a headset interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0062] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0063] The processor 110 can include one or more processing units, such as: the processor 110 can include an application processor (AP), a coprocessor (CP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0064] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. Avoiding repeated access, reducing the waiting time of the processor 110, thus improving the efficiency of the system.

[0065] In some embodiments, the processor 110 can include one or more interfaces, such as an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0066] The wireless communication function of the terminal device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0067] The antenna 1 and the antenna 2 are used for transmitting and receiving electromagnetic wave signals.

[0068] The mobile communication module 150 can provide solutions including 2G / 3G / 4G / 5G wireless communication applied to the terminal device 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor, and convert the signals into electromagnetic waves radiated by the antenna 1.

[0069] The modem processor can include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is transmitted to the application processor. The application processor outputs the sound signal through the audio device (not limited to the loudspeaker 170A, the receiver 170B, etc.), or displays the image or video through the display screen 194.

[0070] The wireless communication module 160 can provide solutions for wireless communication including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc. applied on the terminal device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive signals to be sent from the processor 110, frequency-modulate them, amplify them, and radiate them as electromagnetic waves via the antenna 2.

[0071] In some embodiments, the antenna 1 and the mobile communication module 150 of the terminal device 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the terminal device 100 can communicate with the network and other devices through wireless communication technology.

[0072] The terminal device 100 can implement the photographing function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, etc. The camera 193 can also be referred to as a camera module.

[0073] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electric signal, and the camera photosensitive element transmits the electric signal to the ISP for processing to convert it into an image visible to the naked eye. The ISP can also optimize the algorithm of the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be arranged in the camera 193.

[0074] The camera 193 is configured to capture still images or videos. An object projects an optical image through a lens to a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to an ISP to be converted into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into a standard image signal in RGB, YUV, or the like. In some embodiments, the terminal device 100 can include one or N cameras 193, where N is an integer greater than 1.

[0075] The digital signal processor is configured to process digital signals, including digital image signals and other digital signals.

[0076] The video codec is configured to compress or decompress digital videos. The terminal device 100 can support one or more video codecs. In this way, the terminal device 100 can play or record videos in multiple encoding formats, such as moving picture experts group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, and the like.

[0077] The external memory interface 120 can be configured to connect an external memory card, such as a Micro SD card, to extend the storage capacity of the terminal device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement data storage functions. For example, music, video, and other files can be saved in the external memory card.

[0078] The internal memory 121 can be configured to store computer executable program codes including instructions. The processor 110 executes various functional applications and data processing of the terminal device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as an image playing function, etc.), and the like. The data storage area can store data (such as audio data, etc.) created by the terminal device 100 during use, and the like.

[0079] The methods in the following embodiments can be implemented in the terminal device 100 having the hardware structure described above.

[0080] In the embodiments of the present application, the terminal device 100 can include at least two sets of processing resources (the processing resources can also be referred to as functional modules), and each set of processing resources can include one or more of an antenna, a processor, a memory, and a sensor. In a first time period, one set of processing resources of the at least two sets of processing resources can perform a sensing task, and another set of processing resources can perform a communication task (which can be a first-level communication task, for example, a call, an online chess game, webpage browsing, etc.). In a second time period (different from the first time period), the terminal device does not need to perform a sensing task, and can perform a communication task (which can be a second-level communication task, for example, high-speed downloading, online video watching, online gaming, etc.) based on part or all of the at least two sets of processing resources. That is, the terminal device 100 can simultaneously perform a sensing task and a communication task (a first-level communication task), and perform different-level communication tasks (a first-level communication task and a second-level communication task) in time division, so that the limited processing resources can be fully utilized to realize communication and sensing fusion, and the cost of realizing communication and sensing fusion of the terminal device is reduced.

[0081] It can be understood that the structure illustrated in the embodiments does not constitute a specific limitation on the terminal device 100. In other embodiments, the terminal device 100 can include more or fewer components than illustrated, or combine certain components, or split certain components, or different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware. For example, the terminal device 100 can also include auxiliary devices such as a mouse, a keyboard, a drawing board, etc., for the process of making, delivering, receiving, and customizing a target expression.

[0082] The terminal device 100 described above can be a general-purpose device or a special-purpose device. In specific implementations, the terminal device 100 can be a desktop computer, a laptop computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, an embedded device, or a device having a structure similar to that in FIG. 2. The embodiments of the present application do not limit the type of terminal device 100.

[0083] For example, the apparatus for implementing the functions of the network device provided in the embodiments of the present application can be implemented by the apparatus 300 in FIG. 3. FIG. 3 shows a hardware structure schematic diagram of the apparatus 300 provided in the embodiments of the present application. The apparatus 300 includes at least one processor 301 for implementing the functions of the network device provided in the embodiments of the present application. The apparatus 300 can also include a bus 302 and at least one communication interface 304. The apparatus 300 can also include a memory 303.

[0084] The bus 302 can be used to transmit information between the above-mentioned components.

[0085] The communication interface 304 is configured to communicate with other devices or communication networks, such as an Ethernet, a RAN, a WLAN, and the like. The communication interface 304 can be an interface, a circuit, a transceiver, or other device capable of enabling communication, and the present application is not limited thereto. The communication interface 304 can be coupled to the processor 301.

[0086] The memory 303 is configured to store program instructions, and the processor 301 can control execution of the program instructions to implement the methods provided by the embodiments described below. For example, the processor 301 is configured to invoke and execute the program instructions stored in the memory 303 to implement the methods provided by the embodiments described below.

[0087] Optionally, the memory 303 can be included in the processor 301.

[0088] In specific implementations, as an example, the processor 301 can include one or more CPUs, such as CPU0 and CPU1 in FIG. 3.

[0089] In specific implementations, as an example, the apparatus 300 can include multiple processors, such as the processor 301 and the processor 305 in FIG. 3. Each of the processors can be a single-core processor or a multi-core processor. The processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0090] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the present application, unless otherwise specified, “at least one” refers to one or more, and “multiple” refers to two or more than two. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the same items or similar items with basically the same functions and roles are distinguished by using “first”, “second”, and the like. Those skilled in the art can understand that “first”, “second”, and the like do not limit the quantity and execution order, and “first”, “second”, and the like do not necessarily mean different.

[0091] In order to facilitate understanding, the method for communication and perception fusion provided by the embodiments of the present application will be specifically introduced below in conjunction with the drawings.

[0092] As shown in FIG. 4, the embodiments of the present application provide a method for communication and perception fusion, comprising:

[0093] 401. The terminal device reports terminal capability information to the network device.

[0094] The terminal capability information includes information indicating that the terminal device supports simultaneous execution of a sensing task (may also be referred to as a sensing service) and a communication task (may also be referred to as a communication service).

[0095] The simultaneous execution of the sensing task and the communication task refers to that the sensing task and the communication task are respectively executed based on different processing resources of the terminal device in the same time period (may also be referred to as a time interval or a time interval). For example, the terminal device can execute the sensing task based on a first processing resource and execute the communication task based on a second processing resource in the same time period (for example, a first time period). The first processing resource and the second processing resource are different.

[0096] The processing resource of the terminal device can include one or more of an antenna (antenna array), a processor (for example, CPU, GPU, etc.), a memory, a sensor (for example, a gyroscope, a distance sensor, an acceleration sensor).

[0097] For example, in the case where the terminal device includes three sets of antenna arrays (for example, antenna array 1, antenna array 2 and antenna array 3), the first processing resource can include the antenna array 1, and the second processing resource can include the antenna array 2 and the antenna array 3.

[0098] The sensing task can be triggered by the terminal device (for example, an APP installed on the terminal device can call the sensing function of the terminal device, and the terminal device can execute the sensing task triggered by the APP), or the sensing task can be indicated by a network device (for example, a base station) (that is, the terminal device can execute the sensing task issued by the network device).

[0099] The terminal capability information can also include information indicating that the terminal device can execute a first level of communication task when executing the sensing task, and can execute a second level of communication task when not executing the sensing task. So that the base station can schedule the terminal device to execute different levels of communication tasks in different cases.

[0100] In some embodiments, the terminal device can actively report the terminal capability information to the network device (for example, a base station) through an RRC message. The terminal capability information can be carried in the RRC message.

[0101] In some embodiments, the terminal device can actively report the terminal capability information to the network device (for example, a base station) through an RRC message. The terminal capability information can be carried in the RRC message.

[0102] 402、The network device receives terminal capability information from the terminal device.

[0103] After receiving the terminal capability information from the terminal device, the network device determines, according to the terminal capability information, that the terminal device supports simultaneous execution of the sensing task and the communication task. Further, the network device determines, according to the terminal capability information, that the terminal device is capable of executing the first level of the communication task when executing the sensing task, and is capable of executing the second level of the communication task when not executing the sensing task.

[0104] 403、The network device schedules the terminal device to execute the first level of the communication task in a first time period, the first time period being a time period in which the terminal device executes the sensing task.

[0105] That is, the network device can schedule the terminal device to execute the first level of the communication task in a time period (e.g., the first time period) in which the terminal device executes the sensing task.

[0106] 404a、The terminal device executes the sensing task in the first time period, and executes the first level of the communication task in the first time period.

[0107] That is, the terminal device simultaneously executes the sensing task and the first level of the communication task in the first time period.

[0108] The frequency band in which the terminal device executes the sensing task and the frequency band in which the terminal device executes the first level of the communication task can be the same, not completely the same (e.g., can partially overlap), or different.

[0109] 404b、The terminal device ends execution of the sensing task.

[0110] The terminal device can end execution of the sensing task after the first time period ends.

[0111] 405、The network device schedules the terminal device to execute the second level of the communication task in a second time period, wherein the first time period is different from the second time period, and the second level of the communication task requires more processing resources than the first level of the communication task.

[0112] That is, the network device can schedule the terminal device to execute the second level of the communication task in any time period (e.g., the second time period) in which the terminal device does not execute the sensing task.

[0113] The first level of the communication task requires less processing resources than the second level of the communication task. That is, the second level of the communication task requires more processing resources than the first level of the communication task.

[0114] For example, the first level of communication tasks can include a call, an online chess game, browsing a webpage, etc. The second level of communication tasks can include a high-speed download, online video watching, online network gaming, etc.

[0115] 406. The terminal device performs a second level of communication tasks in a second time period.

[0116] The terminal device can perform the second level of communication tasks in the second time period according to scheduling of the network device (e.g., a base station). The frequency band in which the terminal device performs the second level of communication tasks can be the same as, not completely the same as (e.g., can partially overlap), or different from the frequency band in which the terminal device performs the perception task.

[0117] In addition, there is no necessary execution sequence between steps 403-406, and the embodiment does not specifically limit the execution sequence between steps.

[0118] Based on the method provided in the embodiment of the present application, the network device (e.g., a base station) can perform different levels of communication task scheduling (also referred to as data scheduling) according to whether the terminal device performs a perception task (a perception task indicated by the network device or a perception task triggered by the terminal device itself) in different time periods. In a time period (e.g., a first time period) in which the terminal device performs a perception task, the base station can schedule the terminal device to perform a first level of communication tasks; in a time period (e.g., a second time period) in which the terminal device does not perform a perception task, the base station can schedule the terminal device to perform a second level of communication tasks. The first level of communication tasks requires less processing resources than the second level of communication tasks. In this way, the terminal device can simultaneously perform a perception task and a communication task (a first level of communication task), and perform different levels of communication tasks (a first level of communication task and a second level of communication task) in time, which can fully utilize limited processing resources to realize communication and perception fusion and reduce the cost of realizing communication and perception fusion of the terminal device.

[0119] As shown in FIG. 5, the embodiment of the present application provides a method for communication and perception fusion, taking a base station as a network device and a UE as a terminal device, and taking a perception task indicated by the base station as an example, which includes the following steps:

[0120] 501. The UE reports UE capability information to the base station.

[0121] The UE capability information (i.e., terminal capability information) can refer to the related description of step 401, which is not repeated here.

[0122] Optionally, the UE capability information further includes information indicating whether the UE supports cooperative perception with the network device. Cooperative perception means that the UE can perform a perception task indicated by the base station by means of its own processing resources.

[0123] Optionally, the UE capability information further comprises at least one of a minimum time unit for performing a sensing task, a task type of the sensing task, and an accuracy for performing the sensing task.

[0124] The minimum time unit for performing a sensing task refers to the shortest time required for the UE to perform a sensing task. The task type of the sensing task may, for example, include positioning, imaging, map construction, posture recognition, action recognition, gesture sensing, expression sensing, environment sensing, and the like. The accuracy for performing the sensing task refers to the accuracy of the sensing result obtained by performing the sensing task. For example, when the sensing task is positioning, the accuracy of the sensing result (i.e., the positioning result) can reach the accuracy of meters (m) or centimeters (cm); for another example, when the sensing task is posture recognition, the sensing result (i.e., the posture recognition result) can identify dozens of postures, and the recognition accuracy is high.

[0125] 502. The base station receives the UE capability information sent by the UE.

[0126] In the case that the UE supports cooperative sensing with the network device, after step 502, steps 503 and 504 can also be performed.

[0127] 503. The base station indicates to the UE a time and / or a frequency band for performing a sensing task.

[0128] The base station can send indication information to the UE, the indication information being used to indicate the time and / or the frequency band for performing the sensing task. That is, the base station can instruct the UE to perform cooperative sensing (for example, the base station can indicate to the UE information such as the task type of the sensing task (e.g., positioning, posture recognition, etc.), the accuracy for performing the sensing task, etc.), and indicate to the UE the time and / or the frequency band for performing the sensing task. The frequency band can also be replaced by a frequency point.

[0129] For example, the time for the base station to instruct the UE to perform the sensing task can be a first time period, which may, for example, be a time period [a, b], a being the start time of the first time period and b being the end time of the first time period. Alternatively, the first time period may, for example, be a time interval [c]. c is the duration of the first time period, and the start time of the first time period can be a preset time. The frequency band for the base station to instruct the UE to perform the sensing task may, for example, be a frequency band [e, f]. e is the lowest frequency point of the frequency band, and f is the highest frequency point of the frequency band.

[0130] Further, the base station can instruct the UE to report the time and the frequency band for reporting the sensing result.

[0131] 504. The UE receives the time and / or the frequency band for performing the sensing task indicated by the base station from the network device.

[0132] The UE can receive indication information from the network device, so as to know the time period and frequency band indicated by the base station to perform the sensing task.

[0133] 505. The UE performs the sensing task in the time period and frequency band indicated by the base station.

[0134] The UE performs the sensing task, that is, the UE performs the transmission and reception of sensing signals. The sensing signals can include orthogonal frequency division multiplexing (OFDM), single-carrier frequency-division multiple access (SC-FDMA), orthogonal time frequency & space (OTFS), frequency modulated continuous wave (FMCW), pulse signal, etc., which are not limited in the present application.

[0135] 506. The base station schedules the UE to perform a first level of communication task in a first time period, wherein the first time period is the time period indicated by the base station for the UE to perform the sensing task.

[0136] The base station schedules the UE to perform a first level of communication task in the time period (i.e. the first time period) for the UE to perform the sensing task. The scheduling of the communication task refers to the scheduling of the data transmission and scheduling of the UE by the base station in terms of communication.

[0137] 507a. The UE performs the first level of communication task in the first time period.

[0138] That is, in the time period indicated by the base station for the UE to perform the sensing task, the UE at most performs the first level of communication task.

[0139] For example, the first level of communication task can include a call, an online chess game, browsing a webpage, etc.

[0140] For example, as shown in FIG. 6, the UE can include an antenna array 1 (the antenna array can also be referred to as an antenna group), an antenna array 2, and an antenna array 3. Each group of antenna arrays can include a plurality of antennas, for example, 1 transmitting antenna and 2 receiving antennas (abbreviated as 1T2R), or 2 transmitting antennas and 4 receiving antennas (abbreviated as 2T4R). Different antenna arrays can correspond to different frequency points. For example, frequency point 1 can correspond to antenna array 1, frequency point 2 can correspond to antenna array 2, and frequency point 3 can correspond to antenna array 3.

[0141] If the frequency point used by the current UE to communicate with the base station is frequency point 1, and the antenna array corresponding to frequency point 1 is antenna array 1. In this case, as shown in FIG. 7, in the first time period (the time period during which the UE performs the sensing task), the antenna array 1 can be used to perform the communication task (the first level of communication task), and the remaining antennas (for example, the antenna array 2 and the antenna array 3) on the UE other than the antenna array 1 can be used to perform / execute the sensing task. In the second time period (any time period other than the time period during which the UE performs the sensing task), the antenna array 1-antenna array 3 can be used to perform the communication task (the second level of communication task).

[0142] The steps of performing the sensing task by the UE are described below taking the antenna array 2 and the antenna array 3 performing the sensing task as an example, including:

[0143] S1. Collect the signal data received by the antenna array (for example, the antenna array 2 and the antenna array 3).

[0144] For example, assuming that there are M (for example, 8) antennas, the collected signal data can be represented as an (M x N) matrix X, where N is the number of sampling points.

[0145] S2. Calculate the covariance matrix (R) of the received signal data.

[0146] R = \frac{1}{N} X (X^H);

[0147] Where (X^H) is the conjugate transpose of (X).

[0148] S3. Perform eigenvalue decomposition on the covariance matrix (R) to obtain the eigenvalues and eigenvectors.

[0149] R = E \Lambda E^H;

[0150] Where (\Lambda) is a diagonal matrix containing the eigenvalues, and (E) is an eigenvector matrix.

[0151] S4. According to the size of the eigenvalues, the eigenvectors are divided into a signal subspace and a noise subspace.

[0152] For example, assuming that there are (d) signal sources, then the eigenvectors corresponding to the largest (d) eigenvalues constitute the signal subspace, and the remaining eigenvectors constitute the noise subspace.

[0153] S5. Construct the projection matrix (P_n) of the noise subspace.

[0154] P_n = E_n E_n^H;

[0155] Where (E_n) is the eigenvector matrix of the noise subspace.

[0156] S6. For each assumed signal direction (θ), calculate the array manifold vector (a(θ)) according to the projection matrix (P_n) of the noise subspace, and then calculate the MUSIC spectrum.

[0157] P_{MUSIC}(\theta)=\frac{1}{a(\theta)^H P_n a(\theta)};

[0158] wherein P_{MUSIC} represents the MUSIC spectrum.

[0159] S7. Determine the peak value in the MUSIC spectrum, and determine the direction of arrival according to the peak value.

[0160] According to the direction of arrival, the position of the signal source can be estimated (i.e., the positioning of the signal source is achieved). It can be understood that when the MUSIC algorithm is used for DOA estimation, the spatial spectrum can obtain higher gain, so that high-precision (e.g., cm level) positioning can be achieved.

[0161] 507b. The UE ends the execution of the sensing task.

[0162] The UE can end the execution of the sensing task after the first time period ends.

[0163] 508. The base station schedules the UE to perform a second level of communication task in a second time period, wherein the second time period is any time period other than the time period in which the base station instructs the UE to perform the sensing task, and the processing resource required by the first level of communication task is less than the processing resource required by the second level of communication task.

[0164] The base station schedules the UE to perform a second level of communication task in a second time period.

[0165] 509. The UE performs the second level of communication task in the second time period.

[0166] That is, in any time period other than the time period in which the base station instructs the UE to perform the sensing task, the UE performs at most the second level of communication task. Exemplarily, the second level of communication task can include high-speed download, online video watching, online network game, etc.

[0167] 510. The UE reports the sensing result.

[0168] After the UE performs the sensing task instructed by the base station, the UE can report the sensing result to the base station based on the time and frequency band instructed by the base station for reporting the sensing result.

[0169] There is no certain execution sequence between steps 501 and 510, and the execution sequence between steps is not specifically limited in this embodiment.

[0170] Based on the method provided in the embodiment of the present application, the base station can perform different levels of communication task scheduling (which can also be referred to as data scheduling) according to whether the UE performs cooperative sensing (i.e., performs the sensing task indicated by the base station) at different time periods. In the time period (e.g., the first time period) in which the UE performs the sensing task, the base station can schedule the UE to perform a first level of communication task; in the time period (e.g., the second time period) in which the UE does not perform the sensing task, the base station can schedule the UE to perform a second level of communication task. The processing resources required by the first level of communication task are less than the processing resources required by the second level of communication task. In this way, the UE can simultaneously perform the sensing task and the communication task (the first level of communication task), and perform different levels of communication tasks (the first level of communication task and the second level of communication task) in time, which can fully utilize the limited processing resources to realize the fusion of communication and sensing, and reduce the cost of realizing the fusion of communication and sensing by the UE.

[0171] As shown in FIG. 8, the embodiment of the present application provides a method for the fusion of communication and sensing, taking the network device as a base station, the terminal device as a UE, and the sensing task as a UE self-triggered example, which includes the following steps:

[0172] 801. The UE reports UE capability information to the base station.

[0173] 802. The base station receives the UE capability information sent by the UE.

[0174] Steps 801-802 can refer to the related description of steps 501-502, which will not be repeated here.

[0175] 803. The UE sends information about the time and / or frequency band in which the UE performs the sensing task to the base station.

[0176] In some embodiments, when the UE triggers the sensing task itself, or before or when the UE performs the self-triggered sensing task, the UE can send information about the time (e.g., the first time period) and / or the frequency band (e.g., the first frequency band) in which the UE performs the sensing task to the base station.

[0177] The time period in which the UE performs the sensing task can be the first time period, and the frequency band in which the UE performs the sensing task can be the first frequency band.

[0178] 804. The base station receives the information about the time and / or frequency band in which the UE performs the sensing task from the UE.

[0179] 805. The UE performs the self-triggered sensing task.

[0180] The UE can perform the sensing task in the first time period and the first frequency band.

[0181] 806 The base station schedules the UE to perform a first level of communication task in the first time period, wherein the first time period is a time period in which the UE performs the self-triggered sensing task.

[0182] The base station schedules the UE to perform a first level of communication task in the first time period, wherein the first time period is a time period in which the UE performs the self-triggered sensing task.

[0183] 807a The UE performs the first level of communication task in the first time period.

[0184] The related description can refer to step 507a, which will not be repeated here.

[0185] 807b The terminal device ends the execution of the sensing task.

[0186] The terminal device can end the execution of the sensing task after the first time period ends.

[0187] 808 The base station schedules the UE to perform a second level of communication task in the second time period, wherein the second time period is any time period other than the time period in which the UE performs the self-triggered sensing task, and the processing resources required for the first level of communication task are less than the processing resources required for the second level of communication task.

[0188] The base station schedules the UE to perform a second level of communication task in the second time period.

[0189] 809 The UE performs the second level of communication task in the second time period.

[0190] The related description can refer to step 509, which will not be repeated here.

[0191] There is no certain execution order between steps 801-809, and the embodiment does not specifically limit the execution order between steps.

[0192] Based on the method provided in the embodiments of the present application, the network device (for example, a base station) can perform different levels of communication task scheduling (which can also be referred to as data scheduling) according to whether the terminal device (for example, a UE) performs a sensing task (a sensing task triggered by the terminal device itself) in different time periods. In a time period (for example, a first time period) in which the terminal device performs a sensing task, the base station can schedule the terminal device to perform a first level of communication task; in a time period (for example, a second time period) in which the terminal device does not perform a sensing task, the base station can schedule the terminal device to perform a second level of communication task. The processing resources required by the first level of communication task are less than the processing resources required by the second level of communication task. In this way, the terminal device can perform a sensing task and a communication task (a first level of communication task) at the same time, and perform different levels of communication tasks (a first level of communication task and a second level of communication task) at different times, so that the limited processing resources can be fully utilized to realize the fusion of communication and sensing, and the cost of realizing the fusion of communication and sensing of the terminal device is reduced.

[0193] As shown in FIG. 9, the embodiments of the present application provide a method for the fusion of communication and sensing, taking a base station as the network device and a UE as the terminal device for example. The method comprises the following steps:

[0194] 901. The UE reports UE capability information to the base station.

[0195] The UE capability information (i.e., terminal capability information) comprises information indicating that the UE supports performing a sensing task and a communication task at different times.

[0196] The performing of the sensing task and the communication task at different times means that the processing resources of the UE are used to perform the sensing task and the communication task at different times (i.e., the processing resources of the UE are used to perform the sensing task and the communication task at different times). For example, the UE can perform the sensing task and the communication task at different times based on the same processing resource (for example, a third processing resource). For example, the UE can perform the sensing task based on the third processing resource in a third time period, and perform the communication task based on the third processing resource in a fourth time period. The third time period is different from the fourth time period.

[0197] If the UE capability information does not comprise the information indicating that the UE supports performing the sensing task and the communication task at different times, i.e., the UE does not support performing the sensing task and the communication task at different times, there is no subsequent step.

[0198] Optionally, the UE capability information can further comprise information indicating whether the terminal device supports cooperative sensing with the network device, at least one of a minimum time unit for performing a sensing task, a task type of the sensing task, and an accuracy of performing the sensing task. The related description can be referred to in step 501, and will not be described here.

[0199] 902、The base station receives the UE capability information from the UE.

[0200] 903、The UE performs the sensing task in a first time period.

[0201] The sensing task can be triggered by the UE, or the sensing task can be instructed by the base station (e.g., the base station).

[0202] 904、The base station does not schedule the UE to perform a communication task in the first time period.

[0203] The first time period is a time period in which the UE performs the sensing task. That is, the base station can not schedule the UE to perform any communication task in the time period (e.g., the first time period) in which the UE performs the sensing task, i.e., the UE is not scheduled for data transmission and scheduling in terms of communication.

[0204] 905a、The UE ends the execution of the sensing task.

[0205] The UE can end the execution of the sensing task at the end of the first time period.

[0206] 905b、The base station schedules the UE to perform a communication task in a second time period.

[0207] The second time period is any time period in which the UE does not perform the sensing task. That is, the base station can schedule the UE to perform a corresponding communication task in any time period (e.g., the second time period) in which the UE does not perform the sensing task, i.e., the UE can be scheduled for data transmission and scheduling in terms of communication.

[0208] 906、The UE performs the communication task in the second time period.

[0209] For example, the communication task can include a call, an online chess game, browsing a webpage, high-speed downloading, online video watching, online network gaming, etc.

[0210] There is no certain execution order between steps 903-906, and the embodiment does not specifically limit the execution order between steps.

[0211] In some embodiments, as shown in FIG. 10, when the sensing task is instructed by the base station, before step 903, steps 903a and 903b are further included:

[0212] 903a、The base station sends information for instructing the time and / or frequency band for performing the sensing task to the UE.

[0213] That is, the base station can instruct the UE to perform cooperative sensing, and instruct the UE of the time (e.g., the first time period) and / or the frequency band (e.g., the first frequency band) for performing the sensing task. So that the UE performs the sensing task based on the time and / or the frequency band instructed by the base station.

[0214] Further, the base station can indicate the time and frequency band for the UE to report the sensing result. So that the UE reports the sensing result based on the time and / or frequency band indicated by the base station.

[0215] 903b, the UE receives information for indicating the time and / or frequency band for performing the sensing task from the base station.

[0216] In this way, the UE can perform the sensing task indicated by the base station based on the time and / or frequency band indicated by the base station.

[0217] In some other embodiments, as shown in FIG. 11, in the case that the sensing task is triggered by the UE itself, before step 903, further comprising steps 903c and 903d:

[0218] 903c, the UE sends information of the time and / or frequency band for performing the sensing task to the base station.

[0219] 903d, the base station receives the information of the time and / or frequency band for performing the sensing task from the UE.

[0220] In this way, the base station can know the time (e.g., the first time period) and / or frequency band (e.g., the first frequency band) for the UE to perform the sensing task, so that the base station does not schedule / avoids scheduling the UE to perform the communication task at the time (e.g., the first time period) for the UE to perform the sensing task.

[0221] Based on the method provided in the embodiments of the present application, the base station can perform the sensing task (the sensing task indicated by the base station or the sensing task triggered by the terminal device itself) and the communication task at different time periods according to the reuse of its own processing resources. In the time period (e.g., the first time period) for the terminal device to perform the sensing task, the base station does not schedule / avoids scheduling the terminal device to perform the communication task; in the time period (e.g., the second time period) for the terminal device not to perform the sensing task, the base station can schedule the terminal device to perform the communication task. In this way, the terminal device can reuse its own processing resources to perform the sensing task and the communication task at different time periods, which can fully utilize the limited processing resources to realize the fusion of communication and sensing, and reduce the cost of realizing the fusion of communication and sensing by the terminal device.

[0222] The embodiments of the present application also provide a chip system, as shown in FIG. 12, which comprises at least one processor 1201 and at least one interface circuit 1202. The processor 1201 and the interface circuit 1202 can be interconnected through a line. For example, the interface circuit 1202 can be used to receive signals from other devices (e.g., the memory of the terminal device). For another example, the interface circuit 1202 can be used to send signals to other devices (e.g., the processor 1201).

[0223] For example, the interface circuit 1202 can read instructions stored in the memory of the terminal device and send the instructions to the processor 1201. When the instructions are executed by the processor 1201, the terminal device (such as the terminal device 100 shown in FIG. 2) or the network device (such as the network device shown in FIG. 3) can perform various steps in the above embodiments.

[0224] Of course, the chip system can also include other discrete devices, and the embodiments of the present application do not make specific limitations.

[0225] The embodiments of the present application also provide a computer readable storage medium, which includes computer instructions, when the computer instructions are run on a terminal device (such as the terminal device 100 shown in FIG. 2) or a network device (such as the network device shown in FIG. 3), the terminal device 100 performs various functions or steps performed by the terminal device in the above method embodiments, and the network device performs various functions or steps performed by the network device in the above method embodiments.

[0226] The embodiments of the present application also provide a computer program product, when the computer program product is run on a computer, the computer performs various functions or steps performed by the terminal device in the above method embodiments.

[0227] The embodiments of the present application also provide a processing device, which can be divided into different logical units or modules according to functions, each unit or module performs different functions, so that the processing device performs various functions or steps performed by the terminal device or the network device in the above method embodiments.

[0228] Through the above description of the embodiments, those skilled in the art can clearly understand that the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0229] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutual units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0230] The units described as separate components may or may not be physically separate, and the components displayed as units may be a physical unit or multiple physical units, that is, may be located in one place, or also can be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0231] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0232] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical scheme of the embodiments of the present application essentially or the part that contributes to the prior art or the whole or part of the technical scheme can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing an apparatus (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0233] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any change or replacement within the technical scope disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of communication and sensing fusion, the method comprising: The method is applied to a terminal device, and comprises the following steps: The terminal device reports terminal capability information to a network device, wherein the terminal capability information comprises information indicating that the terminal device supports simultaneous execution of a sensing task and a communication task; The terminal device executes the sensing task in a first time period and executes a first level of the communication task in the first time period; The terminal device executes a second level of the communication task in a second time period, and does not execute the sensing task in the second time period, wherein the first level of the communication task requires less processing resource than the second level of the communication task.

2. The method of claim 1, wherein The processing resource comprises one or more of an antenna, a processor, a memory, and a sensor.

3. The method of claim 1 or 2, wherein The terminal capability information further comprises information indicating that the terminal device can execute the first level of the communication task when executing the sensing task, and can execute the second level of the communication task when not executing the sensing task.

4. The method of claim 1 or 2, wherein The sensing task is triggered by the terminal device, or the sensing task is indicated by the network device.

5. The method of claim 4, wherein The terminal capability information further comprises information indicating whether the terminal device supports cooperative sensing with the network device.

6. The method of claim 5, wherein, In a case where the terminal device supports cooperative sensing with the network device and the sensing task is indicated by the network device, the method further comprises: The terminal device receives indication information from the network device, wherein the indication information is used to indicate a time and / or a frequency band for executing the sensing task.

7. The method of claim 4, wherein, In a case where the sensing task is triggered by the terminal device, the method further comprises: The terminal device sends, to the network device, information about a time and / or a frequency band for executing the sensing task by the terminal device.

8. The method according to claim 5 or 6, characterized in that, The terminal capability information further comprises at least one of the following: A minimum time unit for executing the sensing task, a task type of the sensing task, and an accuracy for executing the sensing task.

9. The method according to claim 5 or 6, characterized in that, The method further comprises: The terminal device reports a sensing result after executing the sensing task.

10. A method of communication and sensing fusion, the method comprising: The method is applied to a network device, and comprises the following steps: The network device receives terminal capability information from a terminal device, wherein the terminal capability information comprises information indicating that the terminal device supports simultaneous execution of a sensing task and a communication task; The network device schedules the terminal device to execute a first level of the communication task in a first time period, wherein the first time period is a time period for the terminal device to execute a sensing task; The network device schedules the terminal device to execute a second level of the communication task in a second time period, wherein the terminal device does not execute the sensing task in the second time period, and the first level of the communication task requires less processing resource than the second level of the communication task.

11. The method of claim 10, wherein The processing resource includes one or more of an antenna, a processor, a memory, a sensor.

12. The method of claim 10 or 11, wherein, The terminal capability information further comprises information indicating that the terminal device is capable of performing the first level of communication task when performing the sensing task, and is capable of performing the second level of communication task when not performing the sensing task.

13. The method of claim 10 or 11, wherein, The sensing task is triggered by the terminal device, or the sensing task is indicated by the network device.

14. The method of claim 13, wherein, The terminal capability information further comprises information indicating whether the terminal device supports cooperative sensing with the network device.

15. The method of claim 14, wherein, In a case where the terminal device supports cooperative sensing with the network device, and the sensing task is indicated by the network device, the method further comprises: The network device sends indication information to the terminal device, the indication information indicating a time and / or a frequency band for performing the sensing task.

16. The method of claim 13, wherein, In a case where the sensing task is triggered by the terminal device, the method further comprises: The network device receives, from the terminal device, information of a time and / or a frequency band for performing the sensing task by the terminal device.

17. The method of claim 14 or 15, wherein, The terminal capability information further comprises at least one of: A minimum time unit for performing the sensing task, a task type of the sensing task, and an accuracy for performing the sensing task.

18. The method of claim 14 or 15, wherein, The method further comprises: The network device receives, from the terminal device, a sensing result of the sensing task.

19. A communication system, characterized by A terminal device and a network device, the terminal device performing the method of any one of claims 1-9, and the network device performing the method of any one of claims 10-18.

20. A communications device, characterized by The communication device is a terminal device or a network device, and the communication device comprises a wireless communication module, a memory, and one or more processors; the wireless communication module, the memory, and the processor are coupled; The memory is configured to store computer program code, the computer program code comprising computer instructions; when the computer instructions are executed by the processor, the communication device performs the method of any one of claims 1-9, or performs the method of any one of claims 10-18.

21. A computer-readable storage medium, characterized in that, The computer instructions comprise: When the computer instructions are executed on a terminal device, the terminal device performs the method of any one of claims 1-9; or, when the computer instructions are executed on a network device, the network device performs the method of any one of claims 10-18.

Images