Sensing method, apparatus and system, and storage medium and program product

Abstract

Provided in the present application are a sensing method, apparatus and system, and a storage medium and a program product. In the method, a first apparatus represented by a terminal device or an access network device, etc., may determine first sensing measurement data on the basis of a first signal used for environment sensing; and the first apparatus sends sensing training data to a second apparatus represented by a sensing network element, etc., wherein the sensing training data comprises the first sensing measurement data and an environment sensing result, and is configured to train a sensing neural network. In this way, a method for collecting sensing training data for training a sensing neural network is provided. Moreover, since the sensing training data is collected on the basis of a sensing measurement process, the number of information exchanges in a system is not excessively increased.

Description

A perception method, apparatus, system, storage medium and program product

[0001] Cross-reference to Related Applications

[0002] This application claims priority to the Chinese Patent Application No. 202411241536.2, filed on September 4, 2024, entitled “A perception method, apparatus, system, storage medium and program product”, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0003] The present application relates to the field of communication technology, and in particular to a perception method, apparatus, system, storage medium and program product. BACKGROUND

[0004] Artificial intelligence (AI) refers to a technology that uses a general computer program to present human intelligence. A model is an application of AI. A model for perception can be referred to as a perception model. The perception model is introduced into the field of perception to improve the efficiency of perception. However, before the perception model is applied, a large amount of training data needs to be relied on to train the perception model. How to collect the training data of the perception model for perception, there is currently no corresponding solution. SUMMARY

[0005] The present application provides a perception method, apparatus, system, storage medium and program product for providing a mechanism for collecting perception training data. The perception method, apparatus, system, storage medium and program product can be referred to as a communication method, apparatus, system, storage medium and program product, or a communication and perception integrated method, apparatus, system, storage medium and program product, or a communication and perception method, apparatus, system, storage medium and program product.

[0006] In a first aspect, an embodiment of the present application provides a sensing method. Alternatively, an embodiment of the present application provides a communication method, or a communication-sensing integrated method, or a communication-sensing method. The method can be applied to a first device side. The first device side can be the first device itself, or a module in the first device, or a logical module or software capable of realizing part or all functions of the first device. The first device side can also be referred to as a sensing node, which has sensing capability. The first device side is, for example, a network device side or a terminal device side. The network device side can refer to the network device itself (e.g., a base station), or a module in the network device, or a logical module or software capable of realizing all or part of the functions of the network device. The module in the network device is, for example, a processor, a communication module, or a circuit, a chip or a central unit (CU), a distributed unit (DU) responsible for communication functions, etc. in the network device. The chip is, for example, a modem chip, or a system on chip (SoC) chip or a system in package (SIP) chip containing a modem core, etc. The terminal device side can refer to the terminal device itself (e.g., a mobile phone), or a module in the terminal device, or a logical module or software capable of realizing all or part of the functions. The module in the terminal device is, for example, a processor, a communication module, or a circuit or chip responsible for communication functions, etc. in the terminal device. The chip is, for example, a modem chip, or a SoC chip or a SIP chip containing a modem core, etc. For simplicity of description, the method is described below by taking the application of the method to the first device as an example.

[0007] The method comprises: determining first sensing measurement data based on a first signal, the first signal being used for sensing an environment, and sending sensing training data to a second device, wherein the sensing training data comprises the first sensing measurement data and a sensing result of the environment, and the sensing training data is used for training a sensing neural network.

[0008] The first signal can be a signal obtained after a sensing signal passes through the environment, for example, the first signal can be a signal obtained after the sensing signal is refracted, reflected or diffracted by a target in the environment. For example, the first signal can be a return signal in a single-station sensing mode, or the first signal can be a signal received by a receiving end in a double-station sensing mode. The first sensing measurement data is determined based on the first signal, for example, the first sensing measurement data can include intermediate data in the process of determining the sensing result through the first signal, and / or include the first signal, etc. The sensing result of the environment can be used as label information for training the sensing neural network, or a real label or a real result, etc.

[0009] In an embodiment of the present application, the first device can collect perception training data based on the first signal for perception, and report the perception training data to the second device, so that the second device can train a perception neural network for perception based on the training data. In this way, a collection mechanism of perception training data is provided. Moreover, the first device collects perception training data based on the first signal for perception, without increasing the number of information interactions between systems.

[0010] In a possible implementation, the method further includes: sending, to the second device, information about an acquisition manner of the perception result of the environment. The acquisition manner can also be referred to as a determination manner, an obtaining manner, or a source, and the name thereof is not limited.

[0011] In this way, the second device can clearly know the acquisition manner of the perception result of the environment, and the second device can filter out perception training data that is more suitable for its needs based on the acquisition manner.

[0012] In a possible implementation, the acquisition manner of the perception result of the environment includes: the perception result of the environment is determined based on first perception measurement data; the perception result of the environment is measured by a sensor of the first device, and the first device receives the first signal; or the perception result of the environment is measured by a sensor of a third device, and the third device sends the first signal to the first device.

[0013] The sensor can be, for example, a high-precision sensor, such as a laser radar, a high-definition camera, a depth camera, or a millimeter wave radar.

[0014] In this way, in the case of determining the perception result of the environment based on the first perception measurement data, the information interaction between systems can be relatively reduced without the aid of other devices or apparatuses. The sensor of the first device or the sensor of the third device is used to measure the perception result of the environment, the acquisition manner of the perception result is relatively simple and direct, the determination efficiency is relatively high, and the reliability of the acquired perception result is relatively high. The sensor of the third device is used to measure the perception result of the environment, the perception result can be acquired based on the perspective of the third device, the perception perspective range is expanded, the reliability of the perception result is increased, and the processing amount of the first device can be relatively reduced.

[0015] In a possible implementation, the method further includes: receiving a request message from the second device, the request message being used to request to report the perception training data.

[0016] In this way, the first device reports the perception training data after the request message of the second device triggers, and the timing of the reported perception training data is more suitable for the needs of the second device, so that unnecessary reporting of the first device can be reduced.

[0017] In a possible implementation, the at least one of the perception measurement data in the perception training data requested to be reported by the request message comprises at least one of: a first signal; channel data corresponding to the first signal; a feature spectrum of a channel corresponding to the first signal; or point cloud data obtained based on the first signal. The at least one can be collectively referred to as a content item of the perception measurement data.

[0018] In this way, the request message specifies the content item of the perception measurement data that needs to be reported, which makes the perception measurement data reported by the first device more in line with the needs of the second device, and helps to reasonably reduce the data amount of the perception measurement data reported by the first device.

[0019] In a possible implementation, the at least one of the first perception measurement data comprises the at least one of the perception measurement data in the perception training data requested to be reported by the request message.

[0020] In this way, the content of the perception measurement data reported by the first device belongs to the content item of the perception measurement data requested to be reported by the second device, which makes the first device more targeted in reporting the perception measurement data, and can reasonably reduce the data amount of the perception measurement data reported by the first device.

[0021] In a possible implementation, the at least one of the perception result in the perception training data requested to be reported by the request message comprises at least one of: whether a target exists in an environment; a position of a target existing in the environment; a size of the target existing in the environment; a number of the target existing in the environment; an outline of the target existing in the environment; or a category of the target existing in the environment. The at least one can be collectively referred to as a content item of the perception result.

[0022] In this way, the request message specifies the content item of the perception result that needs to be reported, which makes the perception result reported by the first device more in line with the needs of the second device, and helps to reasonably reduce the data amount of the perception result reported by the first device.

[0023] In a possible implementation, the at least one of the perception result of the environment comprises the at least one of the perception result in the perception training data requested to be reported by the request message.

[0024] In this way, the content of the perception result reported by the first device belongs to the content item of the perception result requested to be reported by the second device, which makes the first device more targeted in reporting the perception result, and can reasonably reduce the data amount of the perception result reported by the first device.

[0025] In a possible implementation, the method further includes: sending capability information to the second device, the capability information indicating that the first device is capable of obtaining the perception result including at least one of: whether the target exists in the environment; a position of the target existing in the environment; a size of the target existing in the environment; a number of the target existing in the environment; an outline of the target existing in the environment; or a category of the target existing in the environment.

[0026] In this way, the second device can determine the content items of the perception result supported by the first device, and request the content items of the perception result supported by the first device from the first device, thereby avoiding the case that the second device requests the content items of the perception result not supported by the first device from the first device, and thus the availability and success rate of the perception training data reported by the first device to the second device can be improved.

[0027] In a second aspect, an embodiment of the present application provides a perception method. Alternatively, an embodiment of the present application provides a communication method, or a communication-perception integrated method, or a communication-perception method. The method can be applied to the second device side. The second device side can be the second device itself, or a module in the second device, or a logical module or software capable of realizing part or all functions of the second device. The second device can also be referred to as a perception network element or a training network element, and the like, which has the functions of obtaining perception training data and training a perception neural network. The second device can be, for example, a sensing management function (SMF), a sensing function (SF), a location management function (LMF), or a sensing management control (SMC), and the like, and the implementation form thereof is not limited. The method includes: receiving perception training data from a first device, the perception training data including first perception measurement data and a perception result of an environment, the first perception measurement data being determined based on a first signal, and the first signal being used for perceiving the environment; and training a perception neural network based on the perception training data.

[0028] In a possible implementation, the method further includes: receiving information about an obtaining manner of the perception result of the environment from the first device.

[0029] In a possible implementation, the obtaining manner of the perception result of the environment includes: the perception result of the environment is determined based on the first perception measurement data; the perception result of the environment is measured by a sensor of the first device, and the first device receives the first signal; or the perception result of the environment is measured by a sensor of a third device, and the third device sends the first signal to the first device.

[0030] In a possible implementation, the method further includes: sending a request message to the first device, the request message being used to request to report the perception training data.

[0031] In a possible implementation, the perception measurement data in the perception training data requested to be reported by the request message includes at least one of the following: the first signal; channel data corresponding to the first signal; a feature spectrum of a channel corresponding to the first signal; or point cloud data obtained based on the first signal.

[0032] In a possible implementation, the first perception measurement data includes at least one of the following: the perception measurement data in the perception training data requested to be reported by the request message.

[0033] In a possible implementation, the perception result in the perception training data requested to be reported by the request message includes at least one of the following: whether there is a target in the environment; a position of a target existing in the environment; a size of the target existing in the environment; a number of targets existing in the environment; an outline of the target existing in the environment; or a category of the target existing in the environment.

[0034] In a possible implementation, the perception result of the environment includes at least one of the following: the perception result in the perception training data requested to be reported by the request message.

[0035] In a possible implementation, the method further includes: receiving capability information from the first device, the capability information indicating that the first device is capable of obtaining perception results including at least one of the following: whether there is a target in the environment; a position of a target existing in the environment; a size of the target existing in the environment; a number of targets existing in the environment; an outline of the target existing in the environment; or a category of the target existing in the environment.

[0036] In a third aspect, an embodiment of the present application provides a perception device. The perception device can also be referred to as a perception apparatus, a communication device, a communication apparatus, a communication-perception integrated device, or the like, and the name thereof is not limited. For example, the perception device includes a processing unit (also referred to as a processing module) and a communication unit (also referred to as a communication module). The communication unit is configured to perform a transceiving operation, such as functions related to sending and receiving; the communication unit can be referred to as a transceiving unit; optionally, the communication unit includes a receiving unit and a sending unit. The processing unit is configured to perform a processing operation. Alternatively, the communication unit can be a transmitter and a receiver, or the communication unit is a transmitter and a receiver. Optionally, the perception device further includes a storage unit (also referred to as a storage module).

[0037] The perception apparatus can be the first apparatus in the first aspect described above, for example, can be the first apparatus, or a module (for example, a chip system) configured in the first apparatus. The perception apparatus includes means or modules for performing the corresponding functions of the first aspect described above or any possible implementation. For example, the processing unit is configured to determine the first perception measurement data based on the first signal, and the communication unit is configured to send the perception training data.

[0038] The perception apparatus can also implement the content of any possible implementation of the first aspect described above, which will not be listed one by one here.

[0039] In a fourth aspect, the embodiments of the present application provide a perception apparatus. For example, the perception apparatus includes a processing unit (sometimes also referred to as a processing module), and a communication unit (sometimes also referred to as a communication module). The communication unit is configured to perform transceiving operations, such as functions related to sending and receiving; the communication unit can be referred to as a transceiving unit; optionally, the communication unit includes a receiving unit and a sending unit. The processing unit is configured to perform processing operations. Alternatively, the communication unit can be a transmitter and a receiver, or the communication unit is a transmitter and a receiver. Optionally, the perception apparatus further includes a storage unit (sometimes also referred to as a storage module).

[0040] The perception apparatus can be the second apparatus in the second aspect described above, for example, can be the second apparatus, or a module (for example, a chip system) configured in the second apparatus, or a module (for example, a chip system) configured in the second apparatus. The perception apparatus includes means or modules for performing the corresponding functions of the second aspect described above or any possible implementation. For example, the communication unit is configured to receive the perception training data, and the processing unit is configured to train the neural network based on the perception training data.

[0041] The perception apparatus can also implement the content of any possible implementation of the second aspect described above, which will not be listed one by one here.

[0042] In a possible design, the perception apparatus is a communication chip, the processing unit can be one or more processors or processor cores, and the communication unit can be an input / output circuit or an input / output interface of the communication chip.

[0043] In a fifth aspect, the embodiments of the present application provide a perception system. The perception system in this document can also be referred to as a communication system, a communication network, a communication and perception integrated system, or a communication and perception integrated network, etc., without limitation to its name. The system includes a first apparatus and a second apparatus, the first apparatus is, for example, the first apparatus in any of the third aspect and possible implementation, and the second apparatus is, for example, the second apparatus in any of the fourth aspect and possible implementation.

[0044] For example, the first device is configured to determine first perception measurement data based on a first signal, and send perception training data to the second device, the first signal is used for perceiving an environment, the perception training data comprises the first perception measurement data and a perception result of the environment, and the perception training data is used for training a perception neural network; and the second device is configured to train the perception neural network based on the perception training data.

[0045] In a possible implementation, the first device is further configured to determine the perception result of the environment based on the first perception measurement data, or measure the perception result of the environment by a sensor of the first device.

[0046] In a possible implementation, the system further comprises a third device, and the third device is configured to measure the perception result of the environment by a sensor of the third device, and send the perception result of the environment to the first device.

[0047] The first device can also implement the content of any possible implementation of the first aspect, and the second device can also implement the content of any possible implementation of the second aspect, which will not be listed one by one.

[0048] In a sixth aspect, an embodiment of the present application provides a perception device. The perception device comprises one or more processors. The one or more processors can execute computer programs or instructions in a memory, and when the computer programs or instructions are executed, the perception device implements the method in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0049] Optionally, the perception device can comprise a memory, in which case the memory can be coupled with the one or more processors, or the memory is relatively independent of the one or more processors. Alternatively, the memory exists independently of the perception device.

[0050] In a possible design, the perception device can further comprise an interface circuit, and the processor is configured to communicate with other devices or components through the interface circuit.

[0051] The perception device can be a terminal device, a communication module in the terminal device, or a chip responsible for communication function in the terminal, such as a Modem chip (also known as a baseband chip) or a SoC or SIP chip containing a modem module. Alternatively, the perception device can be an access network device or a module in the access network device.

[0052] In a seventh aspect, an embodiment of the present application provides a perception device. The perception device comprises a processor and an interface circuit. The interface circuit is configured to receive a signal from another perception device outside the perception device and transmit the signal to the processor or send a signal from the processor to another perception device outside the perception device. The processor is configured to implement the method in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect by means of a logic circuit or by executing code instructions. The number of processors can be one or more, which is not limited.

[0053] In a specific implementation process, the perception device can be a chip, and the processor can be a transistor, a gate circuit, a flip-flop, and various logic circuits, etc. The specific implementation of the processor is not limited in the embodiments of the present application.

[0054] In an implementation, the perception device can be a wireless perception device, i.e., a computer device supporting wireless communication function. Specifically, the wireless perception device can be a terminal device such as a smart phone, or a network device such as a wireless access network device (e.g., a base station).

[0055] In another implementation, the perception device can be a part of an integrated circuit product in the wireless perception device, such as a system chip or a communication chip. The system chip can also be referred to as a SoC or SoC chip. The communication chip can include a baseband processing chip and a radio frequency processing chip. The baseband processing chip is also sometimes referred to as a modem or a baseband chip. The radio frequency processing chip is also sometimes referred to as a radio frequency transceiver or a radio frequency chip. In physical implementation, part or all of the chips in the communication chip can be integrated inside the SoC chip. For example, the baseband processing chip is integrated in the SoC chip, and the radio frequency processing chip is not integrated with the SoC chip. The interface circuit can be a radio frequency processing chip in the wireless perception device, and the processor can be a baseband processing chip in the wireless perception device. The interface circuit can be an input / output interface, an interface circuit, an output circuit, an input circuit, a pin, or related circuit on the chip or chip system. The processor can also be embodied as a processing circuit or a logic circuit.

[0056] In yet another implementation, the perception device can be a chip system, which can be composed of a chip or can include a chip and other discrete devices. The chip system can include, for example, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a SoC, a CPU, a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chip, etc.

[0057] In an eighth aspect, an embodiment of the present application provides a chip system. The chip system includes a processor. Optionally, the chip system can further include an interface (such as a communication interface). The processor can be configured to implement any of the methods in the first aspect and possible implementation ways to the fourth aspect and possible implementation ways. Optionally, the chip system further includes a memory. The memory is configured to store a computer program (which can also be referred to as code or instructions). The processor is configured to call and run the computer program from the memory, so that a device installed with the chip system performs the method in the first aspect, any possible implementation way of the first aspect, the second aspect, or any possible implementation way of the second aspect. The implementation way of the chip system can refer to the content of the chip system mentioned above, which will not be listed here.

[0058] In a ninth aspect, an embodiment of the present application provides a computer readable storage medium. The computer readable storage medium is configured to store a computer program or instructions, which, when executed, implement the method in the first aspect, any possible implementation way of the first aspect, the second aspect, or any possible implementation way of the second aspect.

[0059] In a tenth aspect, an embodiment of the present application provides a computer program product. When the computer program product is executed, the processor executes the method in the first aspect, any possible implementation way of the first aspect, the second aspect, or any possible implementation way of the second aspect. The computer program product includes a computer program and / or instructions, etc.

[0060] The beneficial effects of any of the technical solutions in the second aspect to the tenth aspect described above can be discussed with reference to the beneficial effects of the corresponding technical solutions in the first aspect, which will not be listed here. BRIEF DESCRIPTION OF DRAWINGS

[0061] FIG. 1 is a schematic diagram of an architecture of a wireless communication system to which embodiments of the present application are applicable;

[0062] FIG. 2 is a schematic diagram of a sensing scenario to which embodiments of the present application are applicable;

[0063] FIG. 3 is a schematic diagram of a communication-sensing integrated scenario to which embodiments of the present application are applicable;

[0064] FIG. 4 is a schematic diagram of a sensing system provided by embodiments of the present application;

[0065] FIG. 5 and FIG. 6 are schematic diagrams of two sensing systems to which embodiments of the present application are applicable;

[0066] FIG. 7 is a schematic diagram of an architecture of an open radio access network system to which embodiments of the present application are applicable;

[0067] FIG. 8 is a schematic diagram of a sensing method provided by embodiments of the present application;

[0068] FIG. 9 is a schematic diagram of a sensing scenario provided by embodiments of the present application;

[0069] FIG. 10 is a schematic diagram of a range-angle spectrum provided by embodiments of the present application;

[0070] FIG. 11 is a schematic diagram of point cloud data provided by embodiments of the present application;

[0071] FIG. 12 is a schematic diagram of a process of obtaining sensing training data provided by embodiments of the present application;

[0072] FIG. 13 is a schematic diagram of a process of training and applying a sensing neural network provided by embodiments of the present application;

[0073] FIG. 14 is a schematic diagram of sensing interaction provided by embodiments of the present application;

[0074] FIG. 15 to FIG. 17 are schematic diagrams of three sensing devices provided by embodiments of the present application. DETAILED DESCRIPTION

[0075] The content related to embodiments of the present application is introduced below.

[0076] The technical solutions provided by the embodiments of the present application can be applied to various communication systems, for example, a 5th generation (5G) mobile communication system (such as a new radio (NR) system), a future communication system, a short-range wireless communication system (such as a side link, a wireless fidelity (Wi-Fi) system, a Bluetooth system, etc.), a long range radio (LoRa) communication system, a wired network, a vehicle to everything (V2X) communication system, a device-to-device (D2D) communication system, a satellite communication system, or a vehicle networking communication system, or a fusion system of at least two of the above communication systems, or other similar communication systems, etc., without limitation. The communication system in the embodiments of the present application can also be referred to as a communication network, a perception system (or a perception network), or a communication-perception integrated system (or a communication-perception integrated network), etc., without specific limitation on the name.

[0077] FIG. 1 is a schematic diagram of an architecture of a wireless communication system to which the embodiments of the present application are applicable. As shown in FIG. 1, the communication system 1000 includes an access network 100. Optionally, the communication system can also include a core network 200 and an Internet 300. The access network 100 can include at least one network device, such as 110a and 110b in FIG. 1, and can also include at least one terminal device, such as 120a to 120j in FIG. 1. Among them, 110a is a base station, 110b is a micro station, 120a, 120e, 120f and 120j are mobile phones, 120b is a car, 120c is a fuel dispenser, 120d is a home access point (HAP) arranged indoors or outdoors, 120g is a notebook computer, 120h is a printer, and 120i is a drone. Among them, the same terminal device or network device can provide different functions in different application scenarios. For example, the mobile phones in FIG. 1 are 120a, 120e, 120f and 120j, the mobile phone 120a can access the base station 110a, connect the car 120b, communicate directly with the mobile phone 120e and access the HAP, the car 120b can access the HAP and communicate directly with the mobile phone 120a, the mobile phone 120f can access the micro station 110b, connect the notebook computer 120g, and connect the printer 120h, and the mobile phone 120j can control the drone 120i.

[0078] 1. Network device

[0079] The network device is a network-side device with wireless transceiving function. The network device can be a device or module with corresponding communication function located at the network side of a communication system. The network device is usually provided with a communication module, circuit or chip for performing corresponding communication functions. The network device is also provided with program instructions for performing corresponding communication functions and corresponding program instructions. The network device can be a device in a radio access network (RAN) that provides wireless communication functions for terminal devices, referred to as a RAN device. The RAN can be an access network in the 3rd generation partnership project (3GPP), such as a 4G, 5G, or future-oriented 6G network. The RAN can also be an open access network (open RAN, O-RAN or ORAN), a cloud radio access network (CRAN), or a communication network of two or more of the above networks.

[0080] The RAN device can also be a base station, an evolved NodeB (eNodeB), a transmission reception point (TRP), a next generation NodeB (gNB) in a 5G mobile communication system, a base station in a 6G mobile communication system, a base station in a future mobile communication system, or an access node in a WiFi system, etc.

[0081] The RAN device can also be a module or unit that completes the functions of the base station part, for example, can be a central unit (CU), can also be a distributed unit (DU), and can also be a radio unit (RU). The CU here completes the functions of the radio resource control protocol and the packet data convergence layer protocol (PDCP) of the base station, and can also complete the function of the service data adaptation protocol (SDAP); the DU completes the functions of the radio link control layer and the medium access control (MAC) layer of the base station, and can also complete part of the physical layer or the entire physical layer. The specific description of the above-mentioned various protocol layers can refer to the related technical specifications of 3GPP. The CU and the DU can be separately arranged, or can also be included in the same network element, such as a baseband unit (BBU). The RU can be included in a radio frequency device or a radio frequency unit, such as a remote radio unit (RRU), an active antenna processing unit (AAU), or a remote radio head (RRH). In different systems, the CU, the DU, or the RU can also have different names, but those skilled in the art can understand their meanings. For example, in the ORAN system, the CU can also be referred to as an O-CU (open CU), the DU can also be referred to as an O-DU, and the RU can also be referred to as an O-RU. Any one of the CU (or CU-CP, CU-UP), the DU, and the RU in this application can be implemented by a software module, a hardware module, or a combination of a software module and a hardware module. The RAN device can be a macro base station (such as 110a in FIG. 1), can also be a micro base station or an indoor station (such as 110b in FIG. 1), and can also be a relay node or a donor node, etc. The embodiments of this application do not limit the specific technology and specific device form adopted by the network device.

[0082] In the embodiments of this application, the functions of the network device can also be executed by a module (such as a chip) in the network device, or can also be executed by a control subsystem containing the functions of the network device. The control subsystem containing the functions of the network device here can be a control center in the above-mentioned application scenarios such as smart grid, industrial control, intelligent transportation, and smart city.

[0083] 2. Terminal device

[0084] A terminal device is a user-side device with wireless transceiving function. The terminal device can also be referred to as a terminal, a user equipment (UE), a mobile station, a mobile terminal, etc. The terminal device can be widely applied in various scenarios, such as device-to-device (D2D), vehicle-to-everything (V2X) communication, machine-type communication (MTC), internet of things (IOT), virtual reality, augmented reality, industrial control, autonomous driving, remote medical treatment, smart grid, smart furniture, smart office, smart wear, smart transportation, smart city, etc. The terminal device can be a mobile phone, a tablet computer, a computer with wireless transceiving function, a wearable device, a vehicle, a drone, a helicopter, an airplane, a ship, a robot, a mechanical arm, a smart home device, etc. The terminal device is usually provided with a communication module, circuit or chip for performing corresponding communication functions. The terminal device is also configured with program instructions for performing corresponding communication functions. In the embodiments of the present application, the device for implementing the functions of the terminal device can be the terminal device, or a device capable of supporting the terminal device to implement the functions, such as a chip system or a combination device or component capable of implementing the functions of the terminal device, which can be installed in the terminal device. The embodiments of the present application do not limit the specific technology and specific device form adopted by the terminal device.

[0085] In the embodiments of the present application, the functions of the terminal device can also be performed by a module (such as a chip or a modem) in the terminal device, or by a device containing the functions of the terminal device.

[0086] The network device and the terminal device can be fixed in position or movable. The network device and the terminal device can be deployed on land, including indoors or outdoors, handheld or vehicle-mounted; can also be deployed on water surface; can also be deployed on airplanes, balloons and artificial satellites in the air. The embodiments of the present application do not limit the application scenarios of the network device and the terminal device.

[0087] The roles of the network device and the terminal device can be relative. For example, the helicopter or the drone 120i in FIG. 1 can be configured as a mobile network device, and for those terminal devices 120j that access the wireless access network 100 through 120i, the terminal device 120i is a network device; but for the network device 110a, 120i is a terminal device, that is, 110a communicates with 120i through a wireless air interface protocol. Of course, 110a and 120i can also communicate through an interface protocol between network devices and network devices, and at this time, 120i is also a network device relative to 110a. Therefore, the network device and the terminal device can be collectively referred to as a communication apparatus, and 110a and 110b in FIG. 1 can be referred to as a communication apparatus with a network device function, and 120a-120j in FIG. 1 can be referred to as a communication apparatus with a terminal device function.

[0088] Currently, it is also proposed to introduce sensing into a communication system (such as a wireless communication system). For ease of understanding, some contents related to sensing are introduced below.

[0089] Sensing can also be referred to as detection, and its name is not limited. Sensing can be used to detect parameters of a target (or target object) existing in a physical environment, such as the position of the target and / or the speed of the target. In the sensing process, the transmitter can detect the target by transmitting an electric wave (i.e., a sensing signal) and analyzing the electric wave (i.e., a return signal) reflected, scattered, refracted, or diffracted by the target.

[0090] The sensing signal is a signal with a sensing function, or in other words, the sensing signal is used for sensing. The sensing signal is also referred to as a detection signal, a linear frequency modulation signal, a radar signal, a radar sensing signal, a radar detection signal, or an environmental sensing signal, and its name is not limited. In addition to the sensing function, the sensing signal can also have other functions, such as a communication function, in which case the sensing signal can also be referred to as a communication-sensing fusion signal. For example, the sensing signal can be a pulse signal, or a signal in wireless communication, for example, the sensing signal can be a reference signal for sensing.

[0091] The target can be referred to as a sensed target, a detected target, a sensed object, a detected object, or a sensed device, and the like, and is not limited. The target can be various objects in the environment that can reflect electromagnetic waves, such as mountains, forests, or buildings, and can also include movable objects such as vehicles, drones, people, and terminal devices. According to the processing type of the sensing signal by the target, the target can also be divided into different types of targets. For example, the target scatters the sensing signal, and the target can be regarded as a scatterer. For another example, the target reflects the sensing signal, and the target can be regarded as a reflector.

[0092] For sensing, according to the sender and receiver of the sensing signal, the sensing mode can be divided into two modes: single station sensing and double station sensing, which are introduced as follows.

[0093] Single station sensing mode is also called self-sending and self-receiving mode or A-sending and A-receiving mode, which means that the device sending the sensing signal and the device receiving the echo signal reflected by the target are the same device.

[0094] For example, please refer to FIG. 2 for a schematic diagram of a sensing scenario. FIG. 2 (1) and FIG. 2 (2) illustrate single station sensing mode.

[0095] As shown in FIG. 2 (1), the device sending the sensing signal and the device receiving the echo signal are the same base station. For example, the base station sends a sensing signal, which is reflected by a target (such as a vehicle), and then the base station receives the echo signal of the sensing signal. The echo signal and the sensing signal can reflect the parameters of the target, for example, the time delay of the echo signal relative to the sensing signal can reflect the distance of the target relative to the transmitter, and the Doppler shift of the echo signal relative to the sensing signal can reflect the speed of the target.

[0096] As shown in FIG. 2 (2), the device sending the sensing signal and the device receiving the echo signal are the same UE. For example, the UE sends a sensing signal, which is reflected by a target (such as a vehicle), and then the UE receives the echo signal of the sensing signal.

[0097] Double station sensing mode is also called A-sending and B-receiving mode or self-sending and other-receiving mode, which means that the device sending the sensing signal and the device receiving the signal reflected by the target are different devices. In this case, the signal received by the receiving end can also be called sensing signal, echo signal, reflected signal or other names, which are not limited to the names. For ease of description, the echo signal is still taken as an example for illustration.

[0098] Continuing to refer to FIG. 2, FIG. 2 (3) to FIG. 2 (6) illustrate double station sensing mode.

[0099] As shown in FIG. 2 (3), the device sending the sensing signal is base station 1, and the device receiving the echo signal is base station 2. For example, the base station 1 sends a sensing signal, which is reflected by a target (such as a vehicle), and then the base station 2 receives the echo signal of the sensing signal.

[0100] As shown in FIG. 2 (4), the device sending the sensing signal is UE1, and the device receiving the echo signal is UE2. For example, the UE1 sends a sensing signal, which is reflected by a target (such as a vehicle), and then the UE2 receives the echo signal of the sensing signal.

[0101] As shown in (5) of FIG. 2, the device sending the sensing signal is a base station, and the device receiving the echo signal is a UE. For example, the sensing signal sent by the base station is reflected by a target (e.g., a vehicle), and the UE receives the echo signal of the sensing signal.

[0102] As shown in (6) of FIG. 2, the device sending the sensing signal is a UE, and the device receiving the echo signal is a base station. For example, the sensing signal sent by the UE is reflected by a target (e.g., a vehicle), and the base station receives the echo signal of the sensing signal.

[0103] The communication scenario in which sensing is introduced can be referred to as a communication-sensing integrated scenario. The following describes the communication-sensing integrated scenario in combination with a communication-sensing integrated scenario shown in FIG. 3. FIG. 3 includes a network device and multiple UEs (e.g., UE1, UE2, and UE3).

[0104] As shown in FIG. 3, UE1 and the network device adopt a two-station sensing mode, in which UE1 sends a sensing signal, and the network device receives an echo signal of the sensing signal reflected by target 1. UE3 and the network device also adopt a two-station sensing mode, in which the network device sends a sensing signal, and UE3 receives an echo signal of the sensing signal reflected by target 2. The network device and UE2 perform communication (e.g., wireless communication) and can transmit a communication signal.

[0105] FIG. 3 also shows a single-station sensing mode. The network device performs sensing on targets 3 to 5 in the single-station sensing mode. Specifically, the network device sends a sensing signal and receives an echo signal of the sensing signal reflected by a target (e.g., target 3, target 4, or target 5).

[0106] FIG. 3 takes target 1 and target 3 as drones, target 2 and target 4 as vehicles, and target 5 as a person as examples, and does not limit the implementation form of the target.

[0107] The following describes a schematic diagram of a sensing system provided by an embodiment of the present application in combination with FIG. 4. Alternatively, FIG. 4 is a schematic diagram of a communication system, or a schematic diagram of a communication-sensing integrated system. FIG. 4 shows a first device, a second device, and a third device. The first device can communicate with the second device and the third device. The device involved in each embodiment of the present application can be understood as a device, a software module or a hardware module in the device, or a logical module, and the like, and is not specifically limited. In addition, the device can be replaced by a network element, a device, an entity, or a node, and the like.

[0108] Exemplarily, the first device can collect the training data based on the sensing. For example, in a single-station sensing mode, the first device can send a sensing signal, the sensing signal passes through reflection, scattering, refraction or diffraction of a target, and then the first device receives a return signal of the sensing signal, and further can obtain the training data based on the return signal. For another example, in a double-station sensing mode, the third device can send a sensing signal, the sensing signal passes through reflection, scattering, refraction or diffraction of a target, and then the first device receives a return signal of the sensing signal, and further the first device can obtain the training data based on the return signal. The first device can report the training data to the second device, and the second device can train the model based on the training data.

[0109] In a possible implementation, the first device is, for example, a network device, or a component in the network device, such as a chip or a chip system arranged in the network device, and the third device is, for example, a terminal device, or a component in the terminal device, such as a chip or a chip system arranged in the terminal device.

[0110] In another possible implementation, the first device and the third device are both network devices, or components in the network devices, such as chips or chip systems arranged in the network devices, for example, the first device is a base station, and the third device is a micro station.

[0111] In another possible implementation, the first device is, for example, a terminal device, or a component in the terminal device, such as a chip or a chip system arranged in the terminal device, and the third device is, for example, a network device, or a component in the network device, such as a chip or a chip system arranged in the network device.

[0112] The second device can have at least one function of sensing calculation, sensing management or sensing neural network model training. The second device can be, for example, a sensing management function (SMF), a sensing function (SF), a location management function (LMF) (or a positioning management device, a positioning server, a positioning center, a positioning network element, a positioning function network element, or a positioning management function, etc.), or a sensing management control (SMC) (which can also be referred to as a control network element, an edge sensing function network element, an edge control network element, or an edge control node, and the name is not limited), or a module in these network elements, etc.

[0113] Please refer to FIG. 5, which is a schematic diagram of a sensing system applicable to the embodiments of the present application. Alternatively, FIG. 5 is a schematic diagram of a communication system, or alternatively, FIG. 5 is a schematic diagram of a communication-sensing integrated system. FIG. 5 illustrates a terminal device, a RAN (e.g., including a first access network device and a second access network device), and some core network elements. The core network elements illustrated in FIG. 5 include an AMF and an LMF. FIG. 5 also illustrates an SMF or an SF. Optionally, the SMF or the SF can also be deployed in the core network, i.e., belong to the core network elements.

[0114] One of the terminal device and the access network device (e.g., the first access network device or the second access network device) illustrated in FIG. 5 can serve as an example of the first apparatus, and the other of the terminal device and the access network device (e.g., the first access network device or the second access network device) can serve as an example of the third apparatus. The SMF / SF involved in FIG. 5 can serve as an example of the second apparatus.

[0115] The first access network device and the second access network device can be the same type of access network device, e.g., both are gNBs or next generation (NG)-eNBs (i.e., ng-eNBs). An ng-eNB is a base station of LTE, and the ng-eNB can include one or more transmission points (TPs). A gNB is a base station of NR, and the gNB can include one or more TRPs. The ng-eNB and the gNB can communicate with each other through an Xn interface. Alternatively, the first access network device and the second access network device are different types of access network devices. For example, one of the first access network device and the second access network device includes an ng-eNB, and the other is a gNB.

[0116] The terminal device communicates with the access network through a Uu link, e.g., the terminal device can communicate with an ng-eNB through an LTE-Uu, and communicate with a gNB through an NR-Uu link. The access network communicates with the AMF through an NG-C interface, the AMF is equivalent to the access network communicating with the LMF, and the access network communicating with the router of the SMF / SF. The AMF communicates with the LMF through an NLs (e.g., NL1) interface.

[0117] Optionally, the SMF / SF and the LMF in FIG. 5 can be the same network element, or alternatively, the SMF / SF and the LMF can be integrated together. In this case, the integration result of the LMF and the SMF can serve as an example of the second apparatus.

[0118] Optionally, the SMF can be an architecture without separation of user plane and control plane. In actual application, the user plane and the control plane of the SMF can also be separated, and the SMF includes a sensing function-control (SF-C) and a sensing function-user (SF-U).

[0119] FIG. 6 is a schematic diagram of a sensing system to which embodiments of the present application can be applied. Alternatively, FIG. 6 is a schematic diagram of a communication system, or alternatively, FIG. 6 is a schematic diagram of a communication-sensing integrated system. As shown in FIG. 6, the communication system includes a terminal device, a RAN (e.g., including a first access network device and a second access network device), an SMC / access and mobility management function (AMF) and an SMF. The first access network device and the second access network device can be respectively the same as the first access network device and the second access network device discussed with reference to FIG. 5. The first access network device and the second access network device can communicate with each other through an Xn interface. The SMC / AMF is connected to the first access network device and the second access network device through an interface. The first access network device and the second access network device can also be connected to different SMCs respectively. Optionally, the SMF can communicate with the SMC. Optionally, the SMC belongs to a network element in the access network.

[0120] One of the terminal device and the access network device (e.g., the first access network device or the second access network device) shown in FIG. 6 can be taken as an example of the first apparatus, and the other of the terminal device and the access network device (e.g., the first access network device or the second access network device) can be taken as an example of the third apparatus. The SMF or the SMC involved in FIG. 6 can be taken as an example of the second apparatus.

[0121] The SMF shown in FIG. 6 is taken as an example of an implementation manner without separation of user plane and control plane. In actual application, the user plane and the control plane of the SMF can also be separated, and the SMF includes an SF-C and an SF-U. Optionally, the SMC includes an SC-C and an SC-U. The SC-C communicates with the SF-C, and the SC-U communicates with the SF-U; or only the SC-C and the SF-C communicate, which is not limited. Optionally, the SMC includes or can be replaced by a sensing control function (SCF).

[0122] In the communication system shown in FIG. 6, the SMC can directly communicate with the SMF, or the SMC can communicate with the SMF through a UPF and an AMF, which is not limited.

[0123] Optionally, when the first access network device adopts the separated architecture of CU and DU, the SMC can be deployed or integrated on the CU or the DU, and no limitation is made in this regard.

[0124] FIG. 7 shows a schematic diagram of an architecture of an O-RAN system provided by an embodiment of the present application. The O-RAN defines the architectural connection and interface standardization between various modules in the RAN. Thus, a RAN can be disassembled into multiple modules, and the modules can be spliced together by modules provided by different device manufacturers because of the interface standardization.

[0125] As shown in FIG. 7, the O-RAN can include an O-CU, an O-DU, and an O-RU. The O-CU includes an O-CU-CP and an O-CU-UP. The system architecture can further include an open cloud (O-cloud), a service management and orchestration framework (SMO), an open eNB (O-eNB), and a RAN intelligent controller (RIC), including a near-real time (RT) RIC (which can be abbreviated as Near-RT RIC) and a non-real time (RT) RIC (which can be abbreviated as non-RT RIC). The O-RAN system shown in FIG. 7 or one or more modules included in the O-RAN system can serve as an example of the first device or the second device.

[0126] The SMO functions like a network management, and performs operations, maintenance, and management on a cloud infrastructure.

[0127] The non-RT RIC is used to implement non-real-time intelligent management of RAN functions, for example, can implement an AI / ML workflow including model training and model updating, and guide applications / functions in the Near-RT RIC based on a policy. The non-RT RIC can be located in the SMO.

[0128] The Near-RT RIC is used to implement near-real-time intelligent management of the RAN. Through data collection and related operations on the E2 interface, near-real-time control and optimization of modules and resources of the O-RAN are implemented.

[0129] O-CU, to implement radio resource control (RRC) layer, packet data convergence protocol (PDCP) layer, and service data adaptation protocol (SDAP) layer and other control functions in 3GPP standards. The O-CU includes an O-CU-CP and an O-CU-UP.

[0130] O-CU-CP, similar to the CU-CP in the NR system, to implement the functions of the RRC layer, and the control plane functions of the PDCP layer.

[0131] O-CU-UP, similar to the CU-UP in the NR system, to implement the functions of the SDAP layer, and the user plane functions of the PDCP layer.

[0132] O-DU, based on low-layer function split, to implement the radio link control (RLC) layer, media access control (MAC) layer, and higher physical layer (Higher PHY) in 3GPP standards. The higher physical layer functions include one or more of the following: forward error correction (FEC) encoding / decoding, scrambling / descrambling, or modulation / demodulation.

[0133] O-RU, based on low-layer function split, to implement the lower physical layer (Lower PHY) functions and radio frequency functions in 3GPP standards. The lower physical layer functions include one or more of the following: fast Fourier transform (FFT) transform / inverse fast Fourier transformation (iFFT) transform, digital beamforming, or extraction and filtering of the physical random access channel (PRACH). Similar to the transmission reception point (TRP) or remote radio head (RRH) in 3GPP, but including the lower physical layer functions, such as FFT / iFFT or extraction of the PRACH.

[0134] As a cloud computing platform, O-Cloud includes physical infrastructure nodes for hosting O-RAN functions such as RIC, O-DU, etc., as well as supporting software components (such as operating system, container runtime), management and orchestration functions.

[0135] The interfaces involved in FIG. 7 are introduced as follows.

[0136] The A1 interface is an interface between the Non-RT RIC and the Near-RT RIC, and is used for intelligent and dynamic control of O-RAN internal wireless resources. The Non-RT RIC provides policies, rich information, and ML model updates to the Near-RT RIC through the A1 interface, and the Near-RT RIC provides policy feedback to the Non-RT RIC through the A1 interface.

[0137] The E2 interface is an open interface between two endpoints, and is used to connect the Near-RT RIC and the RAN node, including, for example, the CU in 5G, the DU, the O-RAN compatible eNB in 4G, the O-CU (O-CU-CP and / or O-CU-UP) and / or the O-DU in O-RAN, etc. The RIC can obtain RAN node data collection and feedback through the E2 node, and the RAN node can obtain control feedback of the Near-RT RIC through the E2 node.

[0138] The O1 interface is an interface between the management entity in the SMO and the O-RAN module, and is used for operation management. Through the interface, FCAPS management, software management, and file management are implemented. The O2 interface is an interface between the SMO and the infrastructure management framework supporting the O-RAN virtual network function.

[0139] The open front haul control, user and synchronization (FHCUS)-Plane interface includes control-plane (C-Plane), user-plane (U-Plane), and synchronization-plane (S-Plane) interfaces. The control plane is used for real-time control between the O-DU and the O-RU, for example, for the O-DU to transmit the weight value for beamforming to the O-RU, or for the O-DU to perform power control on the O-RU, etc. The user plane is used for transmitting communication data between the access network device and the terminal between the DU and the RU. The synchronization plane is used for the O-DU to provide clock synchronization to the O-RU.

[0140] The NG interface is an interface between an NR RAN device (such as a base station, a CU, a CU-CP, or a CU-UP) and an NR core network. Among them, NG-u is a user plane NG interface, and NG-c is a control plane NG interface. The Xn interface is an interface between NR RAN devices (such as base stations, CUs, CU-CPs, or CU-UPs). Xn-u is a user plane Xn interface, and Xn-c is a control plane Xn interface.

[0141] The X2 interface is an interface between LTE RAN devices. X2-u is a user plane X2 interface, and X2-c is a control plane X2 interface. In NR, the X2 interface is mainly used in the evolved universal terrestrial radio access dual connectivity (EN-DC) scenario, and the master station is an LTE RAN device connected to an LTE core network through the X2 interface. The E1 interface is an interface between a CU-CP and a CU-UP. The F1-C interface is an interface between a CU-CP and a DU. The F1-U interface is an interface between a CU-UP and a DU.

[0142] Under the O-RAN architecture, the module that receives the difference reporting of the twin channel and the measurement channel can be a CU, an RT RIC, or a Non-RT RIC, etc. The DU is responsible for receiving signals, signal processing, multipath measurement, channel difference calculation, etc.

[0143] The names of the various interfaces and the connection modes of the various units shown in FIG. 7 are an example. In actual applications, the O-RAN system can include more or fewer interfaces, or more or fewer units.

[0144] The network architecture and service scenarios described in the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, as the network architecture evolves and new service scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0145] With the introduction of sensing into a cellular communication system, in order to improve the efficiency of sensing, it can be considered to use a sensing neural network for sensing. However, there is currently no corresponding solution for how to collect training data for training the sensing neural network.

[0146] Therefore, the embodiment of the present application provides a perception scheme, which mainly utilizes a device (such as a first device, specifically a terminal device or an access network device) that already has a perception function to collect perception training data based on perception, and reports the perception training data to a second device, so that the second device can train a perception neural network for perception based on the perception training data. In this way, a mechanism for collecting perception training data is provided, and the scheme does not excessively increase the signaling overhead of the system.

[0147] Here, some nouns related to the embodiments of the present application are first explained. When not specifically stated, these explanations are to support the meanings of some nouns and make the embodiments of the present application easier to understand, and should not be regarded as strict limitations on the terms in the scope of protection claimed by the present application.

[0148] 1. Artificial intelligence (AI) refers to giving machines human intelligence, using the software and hardware of a computer to simulate some intelligent behaviors of humans, including machine learning and many other methods.

[0149] 2. Machine learning (ML) refers to learning models or rules from raw data, and there are many different machine learning methods, such as neural networks, decision trees, support vector machines, etc.

[0150] 3. Model can include or replace AI, ML model, AI model, algorithm, feature, function, AI function, or machine model, etc., and the name is not limited. The model refers to a function model that maps a certain dimension of input to a certain dimension of output. Or it can be understood that the model is a specific implementation of one or more functions, representing the mapping relationship between the input and output of the model.

[0151] In the field of AI or ML, a model can be understood as an algorithm or system that can make predictions or perform tasks after being trained and learned from input data. The AI model can be at least one of a linear regression model, a logistic regression model, a decision tree model, a support vector machine (SVM), a neural network (may also be referred to as a neural network model or network, etc.), a clustering model, a Bayesian network, a Q-learning model, a generative adversarial network, or other machine learning models, without limitation. The neural network is a mathematical model that simulates the behavior characteristics of animal neural networks for distributed parallel information processing. The neural network model may, for example, be one or more of a deep neural network (DNN), a feed forward neural network (FNN), a convolutional neural network (CNN), and a recurrent neural network (RNN), without specific limitation. According to the universal approximation theorem, a neural network can theoretically approximate any continuous function, so that the neural network has the ability to learn any mapping.

[0152] The model can include one or more parameters. The parameters of the model can be obtained through model training. A substructure (or, a sub-module) of the model can include one or more parameters. For example, f(x) = ax 2 +b can be a model, x can be the input of the model, f(x) can be the output of the model, a and b correspond to the parameters of the model, and the parameters of the model can be obtained through learning and training. The process of model training can be regarded as the process of optimizing the parameters of the model, for example, model training can be training the model parameters by selecting a suitable loss function and using an optimization algorithm to minimize the loss function value. The loss function refers to a measure of the difference between the predicted value of the model and the true value. During or after training the model, model testing can also be performed, i.e., evaluating the performance of the model using test data.

[0153] The model used to implement perception in the embodiments of the present application is referred to as a perception neural network. The perception neural network can also be referred to as an AI, ML model, AI model, algorithm, feature, function, AI function, machine model, perception model, perception algorithm, perception feature, perception AI model, perception ML model, or perception function, without limitation on its name.

[0154] 4、Dataset, refers to the data used for model training, validation and testing in machine learning, the quantity and quality of the data will affect the effect of machine learning. The data used for training the model can be called training data, and the data used for testing the model can be called test data. The data used for training the perception neural network in the embodiment of the application is called perception training data, and the data used for testing the perception neural network is called perception test data.

[0155] 5、Model application, also known as model inference, refers to using the trained model to solve practical problems. For example, the application of the perception neural network is to apply the perception neural network to perception.

[0156] 6、Reference signal (RS), also known as pilot signal or pilot, for example, can be a signal provided by the sending end to the receiving end for channel estimation, channel sounding or data demodulation, etc.

[0157] Reference signal, for example, demodulation reference signal (DMRS), sounding reference signal (SRS), phase tracking reference signal (PTRS), channel state information-reference signal (CSI-RS), cell-specific reference signal (C-RS / CRS), or positioning reference signal (P-RS / PRS) and the like.

[0158] Among them, SRS can be SRS for beam management, or SRS for codebook-based uplink transmission, or SRS for non-codebook-based uplink transmission, or SRS for antenna selection, which is not specifically limited. DMRS, for example, can include DMRS for physical uplink control channel (PUCCH) demodulation (which can be referred to as DMRS for PUCCH) and DMRS for physical uplink share channel (PUSCH) demodulation (which can be referred to as DMRS for PUCCH).

[0159] With the continuous evolution of the standard, the names of the above-mentioned reference signals may change, and more types of reference signals may also appear, which are not specifically limited.

[0160] In a sidelink (SL) communication scenario, a reference signal such as SL-PRS, PSSCH DMRS, PSCCH DMRS, or PSBCH DMRS, etc.

[0161] 7、Point cloud data, which can also be referred to as point cloud information, is not limited by its name. Point cloud data refers to data of a plurality of points on one or more target surfaces. Data of a point includes, for example, coordinates and / or intensity information of the point, etc. Correspondingly, for example, point cloud data can include coordinates and / or intensity information of a plurality of points on one or more target surfaces. For example, data of a point can be represented as (x, y, z, p), where (x, y, z) represents the coordinates of the point, and p represents the intensity information of the point. The coordinates of a point can be the coordinates of the point in a Cartesian coordinate system or a world coordinate system. For example, the point cloud data is three-dimensional (3D) point cloud data. The coordinates of the points included in the three-dimensional point cloud data can be three-dimensional coordinates, for example, including information of the X-axis, the Y-axis, and the Z-axis. The intensity of a point refers to the intensity presented by the point determined by detection. For example, in the point cloud data collected by a laser radar system, the intensity of a point is, for example, the intensity of the radar signal reflected by the point determined by detection; for another example, in the point cloud data collected by a synthetic aperture radar (SAR) technology, the intensity of a point can be the scattering coefficient of the point determined by analyzing the echo signal of the point. Optionally, the point cloud data can also include color parameters of a plurality of points, etc. The color parameter is, for example, color data in red, green, and blue (RGB) format. The density of point cloud data can be understood as the number of points included in a unit area. The higher the density of point cloud data, the larger the data amount of point cloud data, or the lower the density of point cloud data, the smaller the data amount of point cloud data.

[0162] In the embodiments of this application, the number of nouns, unless otherwise specified, represents "a singular noun or a plural noun", that is, "one or more". "At least one" means one or more, and "multiple" means two or more. The association relationship of the associated objects is described, which means that there can be three relationships, for example, A and / or B, which can represent the following cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and rear associated objects. For example, A / B represents A or B. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c means a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0163] In the embodiments of this application, the words "exemplarily", "such as", "for example" and the like are used to represent examples, illustrations or descriptions. Any embodiment or design scheme described as "example" in this application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "example" is used to present the concept in a specific way. In the embodiments of this application, "of", "corresponding" and "corresponding" can be used interchangeably at times, and it should be pointed out that when their differences are not emphasized, their meanings expressed are consistent.

[0164] In the embodiments of this application, "indication" can include direct indication, indirect indication, display indication, and implicit indication. When describing that certain indication information is used to indicate A, it can be understood that the indication information carries A, directly indicates A, or indirectly indicates A. In this application, the information indicated by the indication information is referred to as the to-be-indicated information. There are many ways to indicate the to-be-indicated information in the specific implementation process, for example, but not limited to, the to-be-indicated information can be directly indicated, such as the to-be-indicated information itself or the index of the to-be-indicated information. The to-be-indicated information can also be indirectly indicated by indicating other information, where the other information and the to-be-indicated information have an association relationship. The to-be-indicated information can also be indicated only by a part, and the other part of the to-be-indicated information is known or agreed in advance. For example, the indication of a specific information can also be achieved by means of the arrangement order of each information agreed in advance (for example, a protocol), thereby reducing the indication overhead to a certain extent. In addition, the to-be-indicated information can be sent as a whole, or can be sent separately in multiple sub-information, and the sending period and / or sending time of these sub-information can be the same or different.

[0165] In the embodiments of the present application, "sending" and "receiving" represent the direction of signal transmission. For example, "sending information to XX" can be understood as that the destination of the information is XX, which can include direct sending through the air interface, and also includes indirect sending through the air interface by other units or modules. "Receiving information from YY" can be understood as that the source of the information is YY, which can include direct receiving from YY through the air interface, and also includes indirect receiving from YY through the air interface by other units or modules. "Sending" can also be understood as the "output" of the chip interface, and "receiving" can also be understood as the "input" of the chip interface. In other words, sending and receiving can be carried out between devices, for example, between network devices and terminal devices, or can be carried out within a device, for example, between components, between modules, between chips, between software modules or hardware modules in a device through a bus, a wire or an interface.

[0166] The sensing method provided by the embodiments of the present application will be described below with reference to the accompanying drawings. In the corresponding drawings of the various embodiments of the present application, the steps represented by dashed lines are optional steps.

[0167] The first device involved in the various embodiments of the present application is, for example, any one of the network devices (such as a base station) involved in FIG. 1, any one of the base stations involved in FIG. 2, the network device involved in FIG. 3, the first device involved in FIG. 4, the access network device involved in FIG. 5, or the access network device involved in FIG. 6, or the O-RAN system or the module (such as at least one of CU or DU) in the O-RAN system involved in FIG. 7, etc., and the third device involved in the various embodiments of the present application is, for example, any one of the terminal devices involved in FIG. 1, any one of the UEs involved in FIG. 2, etc., the UE involved in FIG. 3, the third device involved in FIG. 4, the terminal device involved in FIG. 5, or the terminal device involved in FIG. 6, etc. Alternatively, the first device involved in the various embodiments of the present application is, for example, any one of the terminal devices involved in FIG. 1, any one of the UEs involved in FIG. 2, etc., the UE involved in FIG. 3, the first device involved in FIG. 4, the terminal device involved in FIG. 5, or the terminal device involved in FIG. 6, etc., and the third device involved in the various embodiments of the present application is, for example, any one of the network devices (such as a base station) involved in FIG. 1, any one of the base stations involved in FIG. 2, etc., the network device involved in FIG. 3, the third device involved in FIG. 4, the access network device involved in FIG. 5, or the access network device involved in FIG. 6, or the O-RAN system or the module (such as at least one of CU, DU, or RIC (specifically, Near-RT RIC and / or non-RT RIC)) in the O-RAN system involved in FIG. 7, etc.

[0168] In addition, the second device involved in various embodiments of the present application is, for example, the second device involved in FIG. 4, the SMF or SF involved in FIG. 5, or the SMC or SMF involved in FIG. 6, etc. With the continuous evolution of standards, the name and / or function of the device may change, which is not limited.

[0169] Please refer to FIG. 8 for a schematic diagram of a sensing method provided by an embodiment of the present application. The following introduces each step involved in FIG. 8.

[0170] S801, the first device determines first sensing measurement data based on a first signal.

[0171] The first signal can be a signal of the sensing signal after passing through the environment, such as a signal of the sensing signal after at least one of reflection, scattering, refraction, or diffraction of the sensing signal by a target (or obstacle or obstruction, etc.) in the environment. The environment here can be the environment in which the first device is located, such as the communication environment between the first device and other devices (such as the third device). The target in the environment refers to at least one of an object, an animal, or a person in the environment, such as a vehicle and / or a building, etc.

[0172] The content of the sensing signal can refer to the content of the sensing signal discussed above, and the repeated parts are not listed. In addition to the function of sensing, the sensing signal can also have a communication or other function, and the function of the sensing signal is not limited. For example, the sensing signal can be used for sensing and communication, in which case the sensing signal can be referred to as a sensing-communication fusion signal. For example, the sensing signal is a reference signal, in which case the reference signal has the function of estimating the channel and the function of sensing. The content of the reference signal can refer to the content of the reference signal discussed above, which is not listed here. In different sensing modes, the way the first device obtains the first signal is also different, which is introduced below in two cases of A1 and A2.

[0173] A1, single station sensing mode.

[0174] For example, the first device transmits the sensing signal (or the first signal). Correspondingly, the first device receives the sensing signal reflected by the environment, i.e., receives the first signal. The first signal is the echo signal of the sensing signal. In this case, it can be considered that the first device transmits the first signal, and the first device receives the first signal.

[0175] For example, the first device is a base station, and the base station transmits the sensing signal. After the sensing signal is reflected by the target in the environment, the base station receives the first signal. The base station here can also be replaced by a CU, a DU, or a RIC, etc.

[0176] For example, the first device is a terminal device, and the terminal device transmits a sensing signal. The sensing signal is reflected by a target in the environment, and then the first device receives the first signal.

[0177] In short, in the single-station sensing mode, the first signal can be considered as being transmitted by the first device, i.e., the first device spontaneously transmits and receives the first signal.

[0178] A2, double-station sensing mode.

[0179] For example, the third device transmits a sensing signal (or the first signal) to the first device. Correspondingly, the first device receives the first signal from the third device. In this case, the third device can be considered as transmitting the first signal, and the first device receives the first signal.

[0180] For example, the first device is a base station, and the third device is a terminal device. The terminal device can transmit a sensing signal to the base station. The sensing signal is reflected by a target in the environment, and then the base station receives the first signal. The base station can be replaced by a CU, a DU, or a RIC, etc.

[0181] For example, the first device is a terminal device, and the third device is a base station. The base station can transmit a sensing signal to the terminal device. The sensing signal is reflected by a target in the environment, and then the terminal device receives the first signal. The base station can be replaced by a CU, a DU, or a RIC, etc.

[0182] In short, in the double-station sensing mode, the first signal can be considered as being transmitted by the third device to the first device, i.e., the third device transmits the first signal, and the first device receives the first signal.

[0183] Please refer to FIG. 9, which is a sensing scenario provided by an embodiment of the present application. In FIG. 9, the first device is a base station, and the third device is a terminal device. The specific targets in the environment include a building, a tree, and a vehicle running on a road. The terminal device is located at point a, the vehicle is located at point b, the tree is located at point c, the base station is located at point d, and the building is located at point f.

[0184] As shown in FIG. 9, the terminal device transmits a sensing signal to the base station. The sensing signal is reflected by the building, the tree, and the vehicle, and then is received by the base station. That is, the transmission path of the sensing signal includes three paths: path afd, path acd, and path abd. In other words, the base station receives the first signal reflected by different targets.

[0185] The first sensing measurement data can include the first signal, and / or data in the process of determining the sensing result based on the first signal. The first sensing measurement data can include at least one of the following B1 to B4. Or, at least one of the following B1 to B4 is included in the first sensing measurement data.

[0186] B1, a first signal.

[0187] B2, channel data corresponding to the first signal. The channel data corresponding to the first signal includes, for example, a complex result of a channel response corresponding to the first signal, an amplitude of the channel response, a phase of the channel response, an in-phase (I) path corresponding to the first signal, a quadrature (Q) path corresponding to the first signal, or a result of a correlation operation of at least two of the above.

[0188] The first device can perform channel estimation based on the first signal, and can obtain the channel data corresponding to the first signal. Alternatively, the first device can perform conversion processing on the first signal to obtain the channel data corresponding to the first signal. Alternatively, the first device can obtain the channel data corresponding to the first signal through another device.

[0189] Optionally, B1 and B2 can also be collectively referred to as perception raw data. Alternatively, B1 and B2 can be regarded as further subdivision of the perception raw data.

[0190] B3, a feature spectrum of a channel corresponding to the first signal, which can also be referred to as perception preliminary data or perception feature information, perception feature data or spectrum signal, and the like, and the name thereof is not limited. The feature spectrum of the channel corresponding to the first signal represents feature data corresponding to the first signal.

[0191] The feature spectrum of the channel corresponding to the first signal reflects at least one feature of a time delay, a Doppler, an angle, or a signal strength of the signal. The feature spectrum of the channel corresponding to the first signal includes, for example, at least one of a time delay spread spectrum, a Doppler spectrum, an angle spectrum (specifically, a distance-angle spectrum for reflecting a relationship between a distance and an angle), or a signal strength spectrum corresponding to the first signal. The above spectrum information includes information of multiple paths of transmitting the first signal or multiple motion modes, and each path or each motion mode can be reflected by an independent spectrum line or parameter. The spectrum information is a one-dimensional or two-dimensional or three-dimensional information matrix. For example, the time delay spectrum can be represented as a one-dimensional matrix [x1, x2, …, xn, …], wherein the index or position of the value represents time delay information, for example, the time delay corresponding to [x1, x2, …, xn, …] is [t1, t2, …, tn, …], and xn represents the strength information of the signal / channel at the time delay.

[0192] The first device can perform feature variation on the channel (or channel data) corresponding to the first signal to obtain the feature spectrum of the channel.

[0193] For example, assuming that the channel (such as a space-frequency channel) corresponding to the first signal is H, the first device can transform H multiplied by an angle steering vector a to an angle spectrum, transform the angle spectrum multiplied by a time delay steering vector b to a time delay spectrum, transform the time delay spectrum multiplied by an angle-time delay steering vector to an angle-time delay spectrum, and so on.

[0194] For example, a possible angle steering vector can refer to the following formula (1).

[0195] wherein, indicates an angle of arrival (AOA) steering vector, indicates the AOA, and M indicates the number of antenna elements. The dimension of the channel H can be represented as [M*N], N indicates the number of corresponding channels on the subcarrier, and d indicates the element spacing.

[0196] A possible time delay steering vector can refer to the following formula (2).

[0197] b(τ i ) indicates a time delay steering vector in the frequency domain, τ i indicates a time delay.

[0198] Assuming that the angle steering vector and the time delay steering vector are shown in the above formula (1) and formula (2) respectively, the value range of the AOA is 0-30 degrees, the first device can substitute different candidate AOA angles into the steering vector of the AOA, so that the angle steering vector within 0-30 degrees can be obtained as a=[a(0),a(1),…,a(29)] T , and the dimension is [30*M].

[0199] Similarly, assuming that the time delay search range is 400-500 nanoseconds (ns) and the search step is 1 ns, the time delay steering vector is b=[b(400),b(401),…,b(499)], and the dimension is [N*100], so the angle-time delay spectrum can be represented as the following formula (3).

[0200] wherein, “*” in formula (3) indicates matrix multiplication, the dimension of P is [30,100], and P indicates the angle-time delay spectrum information constructed at 0-30 degrees and 400-500 ns.

[0201] For example, continuing with the scenario of the perception shown in FIG. 9, the first device can perform a two Fourier transform (FFT) on the first signal, and can obtain a distance-angle spectrum as shown in FIG. 10. The horizontal axis of the angle spectrum shown in FIG. 10 represents the detection distance of the radio frequency signal, the vertical axis represents the detection angle, and the value represents the signal strength. Therefore, when the signal passes through the target surface and the target surface reflects the signal to the base station within the detection distance and angle range, the target exists at the position of the bright spot on the distance-angle spectrum signal. For example, there are three bright spots in FIG. 10, which correspond to the building, the tree and the vehicle shown in FIG. 9, respectively.

[0202] B4, point cloud data, which can also be referred to as perception intermediate data, is not specifically limited by its name. The point cloud data herein can be point cloud data corresponding to a target in the environment. The point cloud data can be determined based on the first signal, such as information of the target in the environment obtained by the first device measuring the first signal, such as information of at least one of a reflection point, a scattering point, or a diffraction point in the environment.

[0203] Taking the first device as the base station and the first device adopting a single-station perception mode as an example, the base station estimates the angle of the target echo, the zenith of arrival (ZOA) = θ, The time delay (or echo time delay) corresponding to the first signal is τ, and the position coordinates of the base station are (x0, y0, z0). Therefore, the distance from the obstacle (such as a reflection point) to the base station is c*τ / 2, and c is the speed of light.

[0204] Therefore, the base station can determine the coordinate data of the obstacle based on the position coordinates of the base station, the distance from the base station to the scattering point, and the direction from the base station to the scattering point (including the AOA and the ZOA). The x, y, and z in the coordinate data of the obstacle can be represented as the contents of the following formulas (4) to (6), respectively.

[0205] Continuing with the scenario of the perception shown in FIG. 9, the first device can measure the first signal to calculate the point cloud data corresponding to the environment. For example, a point cloud data diagram is shown in FIG. 11. FIG. 11 shows the points on the surfaces of the road, the vehicle, the tree, and the building.

[0206] The first device can analyze and calculate the first signal to obtain the point cloud data. Alternatively, the first device can also obtain the point cloud data from other devices.

[0207] In a possible design, any one of the data in B1 to B4 can be a type of data included in the first perception measurement data, i.e., the first perception measurement data can include at least one type of data in B1 to B4. In this case, B1 to B4 can be regarded as four types of data included in the first perception measurement data. For example, the data in B1 to B4 can correspond to Type 1, Type 2, Type 3, and Type 4, respectively.

[0208] In addition, the data included in the first perception measurement data or the type of the data included can be various, and is not specifically limited. In addition, the division manner of the type of the data included in the first perception measurement data can also be various, and is not specifically limited.

[0209] Optionally, the first device can determine the granularity of the first perception measurement data according to a remaining situation of a processing resource of the first device. The processing resource can be at least one of a computing resource, a storage resource, or a display memory resource, for example. The display memory resource can be a graphics processing unit (GPU).

[0210] For example, if a ratio of the remaining processing resource of the first device to all processing resources of the first device is less than or equal to a first threshold value, it indicates that the first device is currently busy, and the first device can determine relatively coarse-grained first perception measurement data. If the ratio of the remaining processing resource of the first device to all processing resources of the first device is greater than the first threshold value, it indicates that the first device is currently idle, and the first device can determine relatively fine-grained first perception measurement data. The coarse-grained first perception measurement data includes less data than the fine-grained first perception measurement data. Optionally, the difficulty of obtaining the coarse-grained first perception measurement data is lower than the complexity of obtaining the fine-grained first perception measurement data. For example, the first perception measurement data includes point cloud data, the coarse-grained first perception measurement data can include a number of points in the point cloud data less than or equal to a first number, i.e., the density of the point cloud data is smaller, and the fine-grained first perception measurement data can include a number of points in the point cloud data greater than the first number, i.e., the density of the point cloud data is greater.

[0211] Since the first device can flexibly adjust the granularity of the first perception measurement data based on the remaining situation of the processing resource of the first device, the comprehensiveness of the determined first perception measurement data can be ensured as much as possible without overloading the first device.

[0212] The perception measurement data determined by the first device based on the first signal can be different at different moments, and the first measurement perception data can include perception measurement data corresponding to multiple moments, optionally. The multiple moments can be multiple moments in a time period or a period, or can belong to a set of at least one moment in each period.

[0213] For example, the first device or the third device periodically sends the perception signal, the first device periodically receives the first signal, and based on the first signal corresponding to each of the plurality of periods, obtains the perception measurement data corresponding to each of the plurality of periods, that is, the first perception measurement data. In this way, the environment can be perceived relatively continuously to reduce the influence of the deviation of a certain perception and increase the accuracy of the first measurement perception data.

[0214] For example, please refer to FIG. 12, which is a process diagram for obtaining perception training data provided by an embodiment of the present application. As shown in FIG. 12, the first device receives the first signal at t1, t2, t3 and t4 respectively, and obtains the corresponding measurement perception data based on the first signal at these four time points. These perception measurement data can be used as an example of the first perception measurement data.

[0215] S802, the first device sends the perception training data to the second device. Correspondingly, the second device receives the perception training data from the first device.

[0216] The perception training data can also be referred to as training data, which refers to the training data for training the perception neural network. The content of the perception neural network can refer to the content of the perception neural network discussed in the foregoing, which will not be listed here. The perception training data includes the first perception measurement data and the perception result of the environment. In order to simplify the description, the perception result of the environment is referred to as the first perception result in the embodiment of the present application. In other words, the perception training data includes the first perception measurement data and the first perception result corresponding (or associated) to the first perception measurement data. The first perception result can be used as the label information or perception label information of the perception neural network.

[0217] The first perception result refers to the perception result obtained based on the perception. The first perception result can include at least one of the following C1 to C6. Or, at least one of the following C1 to C6 is the content item included in the first perception result.

[0218] C1, whether there is a target in the environment. For example, there is a target or no target in the perception detection range (i.e. in the environment) of the first device.

[0219] C2, the position of the target existing in the environment. For example, there are two targets in the environment, and the center positions of the two targets are (x1, y1, z1) and (x2, y2, z2) respectively.

[0220] C3, the size (or size) of the target existing in the environment. For example, the sizes (such as length, width and height) of the two targets in the environment are [3m, 2m, 1.5] and [1m, 2m, 2m] respectively.

[0221] C4, the number of targets existing in the environment. That is, the number of targets existing in the environment, such as 3 targets existing.

[0222] C5, the contour of the target existing in the environment. The shape / contour of the target. For example, the first device can report the coordinates of part or all of the contour points of the target, such as reporting 10 point coordinates on the contour of the target, and the 10 points are sequentially connected to represent the contour of the target.

[0223] C6, the category of the target existing in the environment. For example, the target is a vehicle, a pedestrian, a bicycle, a garbage can, or a building, etc.

[0224] In the case that the first device can determine the category of the target existing in the environment, the number of targets existing in the environment can specifically include the number of targets of each category.

[0225] In a possible design, any one of the data in C1 to C6 above can be included as a type of data in the first perception result, that is, the first perception result can include at least one type of data in C1 to C6 above. In this case, C1 to C6 above can be regarded as six types of data included in the first perception measurement data. For example, the data shown in C1 to C6 above can correspond to Type 1, Type 2, Type 3, Type 4, Type 5, and Type 6, respectively.

[0226] In addition, the data included in the first perception result or the type of the data included can be various, and no specific limitation is made in this regard. In addition, the division manner of the type of the data included in the first perception result can also be various, and no specific limitation is made in this regard.

[0227] The following illustrates the manner in which the first device obtains (or determines, or acquires) the first perception result.

[0228] D1, the first device determines the first perception result based on the first perception measurement data. For example, the first device can identify the relevant information of the target in the environment according to the point cloud data, channel data, or characteristic spectrum of the channel corresponding to the first signal, to obtain the first perception result.

[0229] D2, the first device obtains the first perception result by measuring the first perception result through the sensor of the first device. The sensor of the first device can be carried on or integrated in the first device. The sensor of the first device can be a high-precision sensor, specifically a laser radar, a high-definition camera, a depth camera, or a millimeter wave radar, etc. Optionally, the manner shown in D2 can be applicable to the case of single-station perception mode. In this way, the first device can coordinate the time of measuring the first signal and detecting the perception result by the sensor, so as to maintain the synchronization of the first perception measurement data and the first perception result as much as possible, which is conducive to improving the accuracy of the obtained perception training data.

[0230] D3, the first device obtains the first sensing result through the sensor of the third device. The content of the sensor of the third device can refer to the content of the sensor of the first device, which is not listed here. Optionally, the mode shown in D3 can be applicable to the case of the two-station sensing mode. In this way, the third device can coordinate the time of measuring the first signal and the sensor detection sensing result, try to keep the synchronization of the first sensing measurement data and the first sensing result, and help to improve the accuracy of the obtained sensing training data.

[0231] The way in which different first devices determine the first sensing result can be different, or the way in which the first device determines the first sensing result at different times can be different. Optionally, the first device can also send information about the acquisition mode of the first sensing result to the second device in the case of sending the sensing training data. The information about the acquisition mode of the first sensing result is, for example, information such as the index (or identifier, or serial number, or number) of the acquisition mode of the first sensing result. In this way, the second device can determine the acquisition mode of the first sensing result.

[0232] Optionally, the first device can also send information about the confidence of the sensing training data to the second device. The confidence can be, for example, the error between the sensing result obtained in the manner shown in D2 or D3 and the sensing result obtained in the manner shown in D1. If the error is smaller, it means that the confidence of the first sensing result is higher. If the error is larger, it means that the confidence of the first sensing result is lower. The second device can also filter the required sensing training data based on the confidence.

[0233] In the case where the first sensing measurement data includes sensing measurement data at multiple times, the first device can also determine the sensing result corresponding to the sensing measurement data at multiple times respectively to obtain the first sensing result. In this case, the first sensing result is equivalent to including the sensing result corresponding to the sensing measurement data at multiple times respectively. One sensing measurement data and the sensing result corresponding to the sensing measurement data can be regarded as a group of sensing training data. In this way, the sensing training data reported by the first device includes multiple groups of sensing training data, which helps to reduce the number of times of reporting the sensing training data by the first device.

[0234] For example, please continue to refer to the schematic diagram shown in FIG. 12, the first device can obtain the sensing results corresponding to t1, t2, t3 and t4 respectively, that is, the first sensing result. The sensing measurement data corresponding to t1 and the sensing result corresponding to t1 can be regarded as a set of sensing training data, the sensing measurement data corresponding to t2 and the sensing result corresponding to t2 can be regarded as a set of sensing training data, the sensing measurement data corresponding to t3 and the sensing result corresponding to t3 can be regarded as a set of sensing training data, and the sensing measurement data corresponding to t4 and the sensing result corresponding to t4 can be regarded as a set of sensing training data.

[0235] After the first device obtains the first sensing measurement data and the first sensing measurement result, the first device can report the sensing training data to the second device. The first device can send the sensing training data in the form of a bit map or a table, and the form of the sensing training data is not limited in the embodiments of the present application. The first device can directly send the sensing training data to the second device, or the first device can send the sensing training data to the second device through other devices (such as a third device, etc.). The implementation of the first device is different, and the path of the first device for sending the sensing training data is also different. The following examples are introduced in combination with E1 to E3.

[0236] E1, the first device is a terminal device, and the first device can send the sensing training data to the second device through an access network device (such as a base station). Optionally, the access network device can transparently forward the sensing training data. For example, the sensing training data can be carried in a non-access stratum (NAS) message.

[0237] E2, the first device is an access network device, and the access network device can directly send the sensing training data to the second device, or forward the sensing training data to the second device through an AMF, etc.

[0238] E3, the first device is a CU, a DU or a RIC, and the CU, the DU or the RIC can directly send the sensing training data to the second device, or forward the sensing training data to the second device through an AMF, etc.

[0239] The first device can send the sensing training data to the second device under the condition that a first condition is met. The first condition can be, for example, that a reporting period of the sensing training data is reached, and / or the data volume of the sensing training data reaches a first data volume, and the specific content of the first condition is not limited. In this way, the second device does not need to be triggered, and the number of interactions between the first device and the second device is relatively reduced.

[0240] Alternatively, the first device can send the perception training data to the second device after receiving the request message from the second device. The request message is used to request reporting the perception training data. That is, the first device reports the perception training data in the case of triggering by the third device, which can reduce unnecessary reporting of the first device.

[0241] Optionally, the perception measurement data in the perception training data requested to be reported in the request message includes at least one of B1 to B4, i.e., at least one of the first signal, the channel data corresponding to the first signal, the characteristic spectrum of the channel corresponding to the first signal, or the point cloud data. That is, the request message specifies which content items of the perception measurement data need to be reported by the first device, i.e., which perception measurement data needs to be reported by the first device. In the case of the above-mentioned four types of perception measurement data, the optional mode can be regarded as the request message specifying the type of perception measurement data that needs to be reported by the first device.

[0242] For example, the request message can include an index of at least one of the data in B1 to B4. Optionally, the index of the at least one data is used to request reporting the perception training data. Alternatively, the request message can include an identification of at least one type of B1 to B4. Optionally, the identification of the at least one type is used to request reporting the perception training data.

[0243] Taking 00, 01, 10, and 11 as examples of the identification of at least one type of B1 to B4. If the request message includes 00, 01, 10, and 11, it means that the second device requests the first device to report the perception measurement data including the types shown in B1 to B4.

[0244] In this optional mode, the first device reports the content items of the perception measurement data requested to be reported in the request message, i.e., the content included in the first perception measurement data belongs to the content items of the perception measurement data requested to be reported in the request message. In other words, at least one of the first perception measurement data includes at least one of the perception measurement data in the perception training data requested to be reported in the request message. Alternatively, the first device reports the type of the perception measurement data requested to be reported in the request message, i.e., the type included in the first perception measurement data belongs to the type of the perception measurement data requested to be reported in the request message. In other words, the type included in the first perception measurement data is the type of the perception measurement data in the perception training data requested to be reported in the request message.

[0245] Optionally, the perception result in the perception training data reported in response to the request message includes at least one of C1 to C6, i.e., at least one of whether there is a target in the environment, the position of the target in the environment, the size of the target in the environment, the number of the target in the environment, the contour of the target in the environment, or the category of the target in the environment. In other words, the request message specifies the content items included in the perception result reported by the first device, i.e., which perception result needs to be reported by the first device. In the case where C1 to C6 are regarded as four types of perception result, the optional manner can be regarded as the request message specifying the type of the perception result reported by the first device.

[0246] For example, the request message can include an index of at least one of C1 to C6. Optionally, the index of the at least one of C1 to C6 is used to request the perception training data to be reported. Alternatively, the request message can include an identification of at least one type of C1 to C6. Optionally, the identification of the at least one type of C1 to C6 is used to request the perception training data to be reported.

[0247] For example, the identification of at least one type of C1 to C6 is 000, 001, 010, 110, 101, and 111. If the request message includes 000, 001, 010, 110, 101, and 111, it means that the second device requests the first device to report the perception result including the type shown in C1 to C6.

[0248] In the optional manner, the first device reports the content item of the perception result requested by the request message, i.e., the first perception result includes the content item of the perception result requested by the request message. In other words, the at least one item included in the first perception result is the at least one item included in the perception result in the perception training data requested by the request message to be reported. Alternatively, the first device reports the type of the perception result requested by the request message, i.e., the type included in the first perception result is the type of the perception result requested by the request message to be reported.

[0249] In addition, the request message can request the perception measurement data in the perception training data to be reported to include at least one of B1 to B4, and can request the perception result in the perception training data to be reported to include at least one of C1 to C6, which is not specifically limited.

[0250] Since the capabilities of different first devices can be different, in order to ensure that the perception result requested by the third device is a perception result that the first device can determine, the first device can optionally send capability information to the second device. The capability information indicates that the perception result that the first device can obtain includes at least one of the above C1 to C6, in other words, the capability information indicates which content items the perception result that the first device can obtain includes. Optionally, the perception result in the perception training data reported in response to the request message includes at least one of the above C1 to C6, which belongs to the perception result that the first device can obtain, ensuring that the perception result requested by the second device is a perception result that the first device can obtain.

[0251] S803, the second device trains the perception neural network based on the perception training data.

[0252] The second device can train the perception neural network based on the perception training data after obtaining the perception training data, until the second condition is met, stop training, and obtain the trained perception neural network. The second condition may, for example, be that the loss function of the perception neural network converges, or the number of times of training of the perception neural network reaches a preset number, or the learning rate of the perception neural network reaches a preset learning rate, etc. The content of the second condition is not limited.

[0253] Optionally, the second device can receive a large amount of perception training data from the first device, and can also receive perception training data from other devices to obtain multiple perception training data. The second device can screen the multiple perception training data to obtain at least one screened perception training data, and train the perception neural network based on the at least one screened perception training data. Optionally, before screening the multiple perception training data, the second device can determine that the data amount of the multiple perception training data is greater than or equal to a preset data amount. In this way, it is ensured that the amount of data used to train the neural network will not be too small.

[0254] In one possible manner, the second device can screen the multiple perception training data based on the respective acquisition manners of the multiple perception training data. For example, the second device can screen out the perception training data acquired by using a specific acquisition manner, which may, for example, be the perception training data acquired by using the acquisition manner shown in D1 above. This is equivalent to retaining the perception training data acquired by using the acquisition manners shown in D2 and D3 above. Since the perception training data acquired by using the acquisition manners shown in D2 and D3 above is more accurate, the perception training data acquired by using the acquisition manners shown in D2 and D3 above by the second device can ensure the accuracy of training, improve the accuracy of the perception neural network, and further improve the accuracy of subsequent perception using the perception neural network.

[0255] In another possible approach, the second device can filter multiple perceptual training data sets based on their respective confidence levels. For example, the second device can filter out perceptual neural networks with confidence levels less than or equal to a preset confidence level, which is equivalent to retaining perceptual neural networks with confidence levels greater than the preset confidence level. Because perceptual training data with higher confidence levels is retained, the second device can use this perceptual training data to train the perceptual neural network, ensuring the accuracy of the training, improving the accuracy of the perceptual neural network, and further improving the accuracy of subsequent perception using the perceptual neural network.

[0256] After obtaining the trained perceptual neural network, optionally, the second device can use perceptual test data to test the trained perceptual neural network to determine its qualification. Once the trained perceptual neural network is determined to be qualified, it is applied. Optionally, the second device can provide the trained perceptual neural network to other devices, such as to network elements like SF or SMF, or the second device can use the trained perceptual neural network itself.

[0257] In the application phase of the perceptual neural network, the second device can input the measured target perception measurement data into the trained perceptual neural network, which then outputs the target perception result. The second device can obtain the target perception measurement data from other devices (such as the first or third device), or it can obtain the target perception measurement data itself; the method of obtaining the target perception measurement data is not specifically limited. The content of the target perception measurement data and the target perception result can be referred to the content of the first perception measurement data and the first perception result discussed above, and will not be listed here. Optionally, the second device can also send the target perception result to other devices (such as the first or third device). Thus, by using a perceptual neural network to achieve perception, the efficiency of perception can be improved.

[0258] For example, please refer to Figure 13, which is a schematic diagram of a process for training and applying a perceptual neural network according to an embodiment of this application. As shown in Figure 13, during the training phase, the second device can filter multiple perceptual training data to obtain filtered perceptual training data. The second device trains the perceptual neural network based on the filtered perceptual training data. The second device can also test the perceptual neural network. During the application phase, the second device can input target perceptual measurement data into the perceptual neural network to obtain the output target perceptual result.

[0259] The interaction between the first and second devices differs depending on the first device and its corresponding sensing mode. Examples of these differences will be provided below, using the sensing interaction diagram in Figure 14 as an example. Figure 14 uses SMF as an example for the second device, but the actual implementation of the second device is not limited.

[0260] In a first example, the first device is a UE, and the first device adopts a single station sensing mode.

[0261] As shown in the sensing schematic diagram of (1) in FIG. 14, in this case, the UE can send a sensing signal, the sensing signal is processed through target reflection, refraction or diffraction, etc., and then the UE receives a first signal (i.e., a back echo signal) corresponding to the sensing signal, and the UE determines first sensing measurement data based on the first signal. The content of the first signal, the content of the first sensing measurement data, and the content of determining the first sensing measurement data can be referred to the content of the first signal, the content of the first sensing measurement data, and the content of determining the first sensing measurement data discussed in FIG. 8 above, which will not be listed here.

[0262] The UE can also obtain the sensing result of the environment based on the first sensing measurement data, or measure the sensing result of the environment through the sensor of the UE, and then report the sensing training data to the SMF, the sensing training data including the first sensing measurement data and the sensing result of the environment. The content of the sensing result of the environment, the content of the sensing training data, and the content of determining the sensing result of the environment can be referred to the content of the sensing result of the environment, the content of the sensing training data, and the content of determining the sensing result of the environment discussed in FIG. 8 above, which will not be listed here. In this way, the SMF can obtain the sensing training data, and train the sensing neural network based on the sensing training data.

[0263] In a second example, the first device is a UE, and the first device adopts a double sensing mode.

[0264] As shown in the sensing schematic diagram of (2) in FIG. 14, in this case, the base station can send a sensing signal, the sensing signal is processed through target reflection, refraction or diffraction, etc., and then the UE receives a first signal corresponding to the sensing signal, and the UE determines first sensing measurement data based on the first signal. The content of the first signal, the content of the first sensing measurement data, and the content of determining the first sensing measurement data can be referred to the content of the first signal, the content of the first sensing measurement data, and the content of determining the first sensing measurement data discussed in FIG. 8 above, which will not be listed here.

[0265] The UE can also obtain the perception result of the environment based on the first perception measurement data, or measure the perception result of the environment through a sensor of the UE, and thus report the perception training data to the SMF, where the perception training data includes the first perception measurement data and the perception result of the environment. The content of the perception result of the environment, the content of the perception training data, and the content of the determination of the perception result of the environment can be respectively referred to the content of the perception result of the environment, the content of the perception training data, and the content of the determination of the perception result of the environment discussed above with reference to FIG. 8, and will not be listed here. In this way, the SMF can obtain the perception training data, and train the perception neural network based on the perception training data.

[0266] The base station shown in (2) of FIG. 14 can be an example of the third apparatus.

[0267] In a third example, the first apparatus is a base station, and the first apparatus adopts a single-station perception mode.

[0268] As shown in the perception schematic diagram in (3) of FIG. 14, in this case, the base station can send a perception signal, the perception signal is processed through target reflection, refraction, or diffraction, etc., the base station receives a first signal (i.e., a backwave signal) corresponding to the perception signal, and the base station determines first perception measurement data based on the first signal. The content of the first signal, the content of the first perception measurement data, and the content of the determination of the first perception measurement data can be respectively referred to the content of the first signal, the content of the first perception measurement data, and the content of the determination of the first perception measurement data discussed above with reference to FIG. 8, and will not be listed here.

[0269] The base station can also obtain the perception result of the environment based on the first perception measurement data, or measure the perception result of the environment through a sensor of the base station, and thus report the perception training data to the SMF, where the perception training data includes the first perception measurement data and the perception result of the environment. The content of the perception result of the environment, the content of the perception training data, and the content of the determination of the perception result of the environment can be respectively referred to the content of the perception result of the environment, the content of the perception training data, and the content of the determination of the perception result of the environment discussed above with reference to FIG. 8, and will not be listed here. In this way, the SMF can obtain the perception training data, and train the perception neural network based on the perception training data.

[0270] In a fourth example, the first apparatus is a base station, and the first apparatus adopts a double-perception mode.

[0271] As shown in the sensing schematic diagram in (4) of FIG. 14, in this case, the UE can send a sensing signal, the base station receives a first signal corresponding to the sensing signal after the target reflection, refraction, or diffraction, etc. processing of the sensing signal, and the base station determines first sensing measurement data based on the first signal. The content of the first signal, the content of the first sensing measurement data, and the content of determining the first sensing measurement data can be referred to the content of the first signal, the content of the first sensing measurement data, and the content of determining the first sensing measurement data discussed in the foregoing FIG. 8, which will not be listed here.

[0272] The base station can also obtain the sensing result of the environment based on the first sensing measurement data, or measure the sensing result of the environment through the sensor of the base station, so as to report the sensing training data to the SMF, the sensing training data including the first sensing measurement data and the sensing result of the environment. The content of the sensing result of the environment, the content of the sensing training data, and the content of determining the sensing result of the environment can be referred to the content of the sensing result of the environment, the content of the sensing training data, and the content of determining the sensing result of the environment discussed in the foregoing FIG. 8, which will not be listed here. In this way, the SMF can obtain the sensing training data, so as to train the sensing neural network based on the sensing training data.

[0273] The UE shown in (4) of FIG. 14 can be an example of the third device.

[0274] The above is an example of the first device and the interaction process in different sensing modes, and the actual implementation form of the first device and the specific interaction process between devices are not limited.

[0275] Based on the same inventive concept, an embodiment of the present application provides a sensing device. Any of the sensing devices shown in FIGS. 15 to 17 is introduced below. The sensing device is, for example, the first device or the second device discussed above, or can be a module thereof, which is not limited here.

[0276] As shown in FIG. 15, the sensing device 1500 can include modules or units for implementing the above-mentioned method embodiments. In one possible design, the sensing device 1500 includes a processing unit 1510 and a communication unit 1520. The communication unit 1520 is configured to perform the functions related to transmitting and receiving, such as the functions related to transmitting and receiving; the communication unit 1520 can be referred to as a transceiver unit; optionally, the communication unit 1520 includes a receiving unit and a transmitting unit. The processing unit 1510 is configured to perform processing operations. Alternatively, the communication unit 1520 can be a transmitter and a receiver, or the communication unit 1520 is a transmitter and a receiver. Optionally, the sensing device 1500 further includes a storage unit 1530. The storage unit 1530 is configured to store the program code or data of the device. The storage unit 1530 is an optional unit as shown by the dashed box in FIG. 15.

[0277] In a first embodiment, the perception apparatus 1500 can be the first device, the communication module in the first device, the circuitry or chip responsible for communication function in the first device, or the like, in the method embodiments of Figure 8 or Figure 14, or implement the functions of the first device in the method embodiments of Figure 8 or Figure 14. For example, the perception apparatus 1500 is a communication module in a terminal device or a network device, or a circuitry or chip responsible for communication function in a terminal device or a network device.

[0278] In the above embodiment, the processing unit 1510 is configured to determine the first perception measurement data, and the communication unit 1520 is configured to transmit the perception training data.

[0279] The perception apparatus 1500 can further implement other steps performed by the first device in the method embodiments of Figure 8, or implement any of the perception interaction processes in Figure 14, which are not listed one by one here.

[0280] In a second embodiment, the perception apparatus 1500 can be the second device, the communication module in the second device, the circuitry or chip responsible for communication function in the second device, or the like, in the method embodiments of Figure 8 or Figure 14, or implement the functions of the second device in the method embodiments of Figure 8. For example, the perception apparatus 1500 is a communication module in an SF or an SMF, or a circuitry or chip responsible for communication function in an SF or an SMF.

[0281] In the above embodiment, the communication unit 1520 is configured to receive the perception training data, and the processing unit 1510 is configured to train the perception neural network based on the perception training data.

[0282] The perception apparatus 1500 can further implement other steps performed by the second device in the method embodiments of Figure 8, or implement any of the perception interaction processes in Figure 14, which are not listed one by one here.

[0283] In a possible design, when the perception apparatus 1500 is a terminal device, a communication module in a terminal device, an access network device, or a communication module in an access network device, the function of the processing unit 1510 can be implemented by one or more processors. Specifically, the processor can include a Modem chip, or a System on Chip (SoC) chip or a SIP chip including a Modem core. The function of the communication unit 1520 can be implemented by a transceiver circuit.

[0284] In a possible design, when the perception apparatus 1500 is a circuit or chip responsible for communication functions in a terminal device, or a circuit or chip responsible for communication functions in an access network device, such as a Modem chip or a System on Chip (SoC) chip or a SIP chip containing a Modem core, the function of the processing unit 1510 can be implemented by circuitry including one or more processors or processor cores in the chip. The function of the communication unit 1520 can be implemented by interface circuitry or data transceiver circuitry on the chip.

[0285] It can be understood that the division of units in the above apparatus is only a logical division of functions, one function unit can be provided for each function, or two or more functions can be integrated in one function unit. In actual implementation, all or part of the units can be integrated on one physical entity, or distributed on different physical entities. In addition, the above function units can be implemented in the form of hardware, or in the form of software, or in the form of hardware combined with software. Whether a certain function is implemented in the form of hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0286] In one example, the function units in any of the above apparatuses can be one or more integrated circuits configured to implement the above methods, for example: one or more application specific integrated circuits (ASICs), or one or more central processing units (CPUs), one or more microcontroller units (MCUs), one or more DSPs, or one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0287] In one example, the storage unit 1530 can include random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, and / or registers, etc.

[0288] The sensing device shown in FIG. 16 is described as follows. As shown in FIG. 16, the sensing device 1600 includes a processor 1610. Optionally, the sensing device 1600 further includes an interface circuit 1620 and a memory 1630. The processor 1610 and the interface circuit 1620 are coupled to each other. It can be understood that the interface circuit 1620 can be a transceiver or an input / output interface. The memory 1630 is configured to store instructions executed by the processor 1610 or store input data required by the processor 1610 to execute instructions or store data generated after the processor 1610 executes instructions. The interface circuit 1620 and the memory 1630 are optional modules, which are shown in a dashed box in FIG. 16. In addition, one processor 1610 and one memory 1630 are taken as an example in FIG. 16, and the number of the processor 1610 and the memory 1630 is not limited in practice.

[0289] The sensing device 1600 is configured to implement the method embodiment shown in FIG. 8 or implement any sensing interaction process related to FIG. 14. Optionally, the processor 1610 is configured to implement the functions of the processing unit 1510, and the interface circuit 1620 is configured to implement the functions of the communication unit 1520.

[0290] For example, the sensing device 1600 can be configured to implement the functions of the first device or the second device related to the method embodiment shown in FIG. 8 or implement the functions of the first device or the second device related to FIG. 14.

[0291] When the sensing device 1600 is a chip applied to a certain device (such as the terminal device or the network device described above), the device chip implements the functions of the device in the above method embodiment. The device chip receives information from other modules (such as a radio frequency module or an antenna) in the device, and the information is sent by other devices to the device; or the device chip sends information to other modules (such as a radio frequency module or an antenna) in the device, and the information is sent by the device to other devices. The sensing device 1600 can be a baseband chip of a certain device, or a DU or other modules, and the DU can be a DU under the open radio access network (O-RAN) architecture.

[0292] The processor 1610 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor can be a microprocessor, or any conventional processor. In addition, the memory involved in various embodiments of the present application can include volatile memory (such as random access memory (RAM)), and can also include non-volatile memory (such as read-only memory (ROM), flash memory, a hard disk drive (HDD), or a solid state drive (SSD)).

[0293] The sensing device shown in FIG. 17 is described below. As shown in FIG. 17, the sensing device 1700 includes a processor 1710 and a transceiver 1730. The processor 1710 can also be referred to as a processing unit, a processing board, a processing module, a processing device, etc. The implementation of the processor 1710 can refer to the content of the processor 1610 in FIG. 16. The transceiver 1730 can also be referred to as a transceiving unit, a transceiver, a transceiving device, etc. The transceiver 1730 includes a transmitter 1731, a receiver 1732, and an antenna 1733. Optionally, the transceiver 1730 can also include radio frequency circuitry, input / output devices, etc., which are not limited herein.

[0294] Optionally, the devices in the transceiver 1730 for implementing the receiving function are regarded as a receiving module, and the devices in the transceiver 1730 for implementing the sending function are regarded as a sending module, that is, the transceiver 1730 includes a receiver and a transmitter. The transceiver can also be referred to as a transceiver, a transceiving module, or a transceiving circuit, etc. The receiver can also be referred to as a receiver, a receiving module, or a receiving circuit, etc. The transmitter can also be referred to as a transmitter, a transmitting module, or a transmitting circuit, etc.

[0295] Optionally, the sensing device 1700 can also include a memory 1720, which can store computer program codes and / or data.

[0296] The processor 1710 is mainly used for processing communication protocol and communication data, controlling the sensing device 1700, executing software programs, processing data of the software programs, etc. The memory 1720 is mainly used for storing software programs and data. The radio frequency circuit is mainly used for conversion between baseband signals and radio frequency signals and processing of the radio frequency signals. The antenna 1733 is mainly used for receiving and transmitting radio frequency signals in the form of electromagnetic waves. The input and output device, such as a touch screen, a display screen, a keyboard, etc. is mainly used for receiving data input by a user and outputting data to the user.

[0297] When data needs to be transmitted, the processor 1710 performs baseband processing on the data to be transmitted, and outputs the baseband signal to the radio frequency circuit. The radio frequency circuit performs radio frequency processing on the baseband signal, and transmits the radio frequency signal in the form of electromagnetic waves through the antenna. When data is transmitted to the sensing device 1700, the radio frequency circuit receives the radio frequency signal through the antenna, converts the radio frequency signal into a baseband signal, and outputs the baseband signal to the processor. The processor 1710 converts the baseband signal into data and processes the data. For the convenience of description, only one memory 1720, one processor 1710 and one transceiver 1730 are shown in FIG. 17. In actual terminal products, there can be one or more processors 1710 and one or more memories 1720. The memory 1720 can also be referred to as a storage medium or a storage device, etc. The memory 1720 can be arranged independently of the processor 1710, or can be integrated with the processor 1710, which is not limited.

[0298] In the embodiments of the present application, the antenna and the radio frequency circuit with the transceiving function are regarded as the communication unit of the sensing device 1700, and the processor with the processing function is regarded as the processing unit of the sensing device 1700. The processor 1710 is used to execute the processing actions of the first device or the second device in the above-mentioned method embodiments for realizing the method shown in FIG. 8, and the transceiver 1730 is used to execute the transceiving actions of the first device or the second device in the above-mentioned embodiments. Alternatively, the processor 1710 is used to execute the actions of the first device or the second device involved in any sensing interaction process involved in FIG. 14.

[0299] When the sensing device 1700 is a chip, the chip includes a processor and a transceiver. The transceiver can be an input and output circuit or a communication interface; the processor can be a processing module integrated on the chip or a microprocessor or an integrated circuit. Optionally, the chip can further include a memory. The transmission operation of the terminal device or the network device in the above-mentioned method embodiments can be understood as the output of the chip, and the reception operation of the terminal device or the network device in the above-mentioned method embodiments can be understood as the input of the chip.

[0300] The embodiments of the present application provide a communication system. The communication system includes a first device and a second device. Optionally, the system further includes a third device.

[0301] The first device can implement the functions of the first device shown in the method embodiments of FIG. 8, or implement the functions of the first device involved in any of the perceptual interaction processes involved in FIG. 14. The second device can implement the functions of the second device in the method embodiments of FIG. 8, or implement the functions of the second device involved in any of the perceptual interaction processes involved in FIG. 14. The third device can implement the functions of the third device in the method embodiments of FIG. 8, or implement the functions of the third device involved in any of the perceptual interaction processes involved in FIG. 14.

[0302] The embodiments of the present application provide a chip system, which comprises a processor and an interface. Wherein, the processor is configured to call and run instructions from the interface, and when the processor executes the instructions, the method embodiments shown in FIG. 8 are implemented, or any of the perceptual interaction processes involved in FIG. 14 are implemented.

[0303] The embodiments of the present application provide a computer readable storage medium for storing computer programs or instructions, when the computer programs or instructions are run, the method embodiments shown in FIG. 8 are implemented, or any of the perceptual interaction processes involved in FIG. 14 are implemented.

[0304] The embodiments of the present application provide a program product, when the program product is executed, the processor implements the method embodiments shown in FIG. 8. The program product is, for example, a computer program product, and specifically, for example, a computer program and / or instructions, etc. The processor is, for example, a processor running in a computer.

[0305] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer programs or instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are performed. The computer can be a general purpose computer, a special purpose computer, a computer network, a network device, a user equipment or other programmable apparatus. The computer programs or instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer programs or instructions can be transferred from one website site, computer, server or data center to another website site, computer, server or data center through wired or wireless manner. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center and the like integrated with one or more available media. The available media can be a magnetic medium, such as a floppy disk, a hard disk, a magnetic tape; an optical medium, such as a digital video disc; a semiconductor medium, such as a solid state disk. The computer readable storage medium can be a volatile or non-volatile storage medium, or can include both volatile and non-volatile storage media.

[0306] In various embodiments of the present application, the terms and / or descriptions of different embodiments are consistent and can be referred to each other if there is no special description and logical conflict. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0307] The various digital numbers involved in the various embodiments of the present application are only used for differentiation for convenience of description, and are not used to limit the scope of the embodiments of the present application. The size of the serial number of the above processes does not mean the execution order, and the execution order of the processes should be based on its function and inherent logic.

Claims

1. A perception method, comprising: The method is applied to a first device; the method comprises: determining first perception measurement data based on a first signal, the first signal being used for perceiving an environment; sending perception training data to a second device, wherein the perception training data comprises the first perception measurement data and a perception result of the environment, and the perception training data is used for training a perception neural network.

2. The method of claim 1, wherein, The method further comprises: sending information about a manner of obtaining the perception result of the environment to the second device.

3. The method of claim 2, wherein, The manner of obtaining the perception result of the environment comprises: the perception result of the environment is determined based on the first perception measurement data; the perception result of the environment is measured by a sensor of the first device, and the first device receives the first signal; or the perception result of the environment is measured by a sensor of a third device, and the third device sends the first signal to the first device.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: receiving a request message from the second device, the request message being used for requesting to report perception training data.

5. The method of claim 4, wherein, The perception measurement data in the perception training data requested to be reported in the request message comprises at least one of: the first signal; channel data corresponding to the first signal; a feature spectrum of a channel corresponding to the first signal; or point cloud data obtained based on the first signal.

6. The method of claim 5, wherein, The first perception measurement data comprises at least one of: The perception measurement data in the perception training data requested to be reported in the request message comprises at least one of:

7. The method according to any one of claims 4-6, characterized in that, whether a target exists in the environment; a position of a target existing in the environment; a size of a target existing in the environment; a number of targets existing in the environment; an outline of a target existing in the environment; or a category of a target existing in the environment. The perception result of the environment comprises at least one of:

8. The method of claim 7, wherein, The perception result in the perception training data requested to be reported in the request message comprises at least one of: whether a target exists in the environment; 9. The method according to any one of claims 1 to 8, characterized in that, a position of a target existing in the environment; a size of a target existing in the environment; a number of targets existing in the environment; an outline of a target existing in the environment; or a category of a target existing in the environment. The method further comprises: sending capability information to the second device, the capability information indicating that the perception result capable of being obtained by the first device comprises at least one of: whether a target exists in the environment; 10. A perception method comprising: a position of a target existing in the environment; a size of a target existing in the environment; a number of targets existing in the environment; 11. The method of claim 10, wherein, an outline of a target existing in the environment; or a category of a target existing in the environment.

12. The method of claim 11, wherein, The method is applied to a second device; the method comprises: receiving perception training data from a first device, the perception training data comprising first perception measurement data and a perception result of an environment, the first perception measurement data being determined based on a first signal, and the first signal being used for perceiving the environment; training a perception neural network based on the perception training data. The method further comprises: receiving information about a manner of obtaining the perception result of the environment from the first device. The manner of obtaining the perception result of the environment comprises: the perception result of the environment is determined based on the first perception measurement data; The perception result of the environment is measured by a sensor of the first device, and the first device receives the first signal; or The perception result of the environment is measured by a sensor of a third device, and the third device sends the first signal to the first device.

13. The method according to any one of claims 10-12, characterized in that, The method further comprises: sending a request message to the first device, the request message being used to request reporting of perception training data.

14. The method of claim 13, wherein, The perception measurement data in the perception training data requested to be reported by the request message comprises at least one of: the first signal; channel data corresponding to the first signal; a feature spectrum of a channel corresponding to the first signal; or point cloud data obtained based on the first signal.

15. The method of claim 14, wherein, The at least one of the first perception measurement data comprises: The at least one of the perception measurement data in the perception training data requested to be reported by the request message.

16. The method according to any one of claims 13-15, characterized in that, The perception result in the perception training data requested to be reported by the request message comprises at least one of: whether a target exists in the environment; a position of a target existing in the environment; a size of a target existing in the environment; a number of targets existing in the environment; an outline of a target existing in the environment; or a category of a target existing in the environment.

17. The method of claim 16, wherein, The at least one of the perception result comprises: The at least one of the perception result in the perception training data requested to be reported by the request message.

18. The method according to any one of claims 10-17, characterized in that, The method further comprises: receiving capability information from the first device, the capability information indicating that the first device can obtain a perception result comprising at least one of: whether a target exists in the environment; a position of a target existing in the environment; a size of a target existing in the environment; a number of targets existing in the environment; an outline of a target existing in the environment; or a category of a target existing in the environment.

19. A perception system, comprising: The system comprises a first device and a second device, wherein: the first device is configured to determine first perception measurement data based on a first signal, and send perception training data to the second device, the first signal being used to perceive an environment, and the perception training data comprising the first perception measurement data and a perception result of the environment, the perception training data being used to train a perception neural network; the second device is configured to train a perception neural network based on the perception training data.

20. The system of claim 19, wherein, The first device is further configured to: determine the perception result of the environment based on the first perception measurement data; or measure the perception result of the environment by a sensor of the first device.

21. The system of claim 19, wherein, The system further comprises a third device, and the third device is configured to: measure the perception result of the environment by a sensor of the third device; send the perception result of the environment to the first device.

22. A sensing device, characterized by The computer program product comprises one or more processors configured to implement the method of any one of claims 1-9, or implement the method of any one of claims 10-18.

23. A computer program product, characterised in that, When the computer program product is executed, the processor is caused to perform the method of any one of claims 1-9, or the method of any one of claims 10-18.

24. A computer-readable storage medium, characterized in that, The storage medium stores a computer program or instructions, which, when executed by the communication device, implement the method of any one of claims 1-9 or the method of any one of claims 10-18.

Images