Perception processing method and device, equipment and medium

Abstract

The invention discloses a perception processing method and device, equipment and a medium, and belongs to the field of artificial intelligence, and the perception processing method comprises the steps that a first node processes first data through a first AI model to obtain second data; wherein the first data or the second data are data related to a sensing service, and the first data comprises at least one of the following items: at least two different types of input data; at least two different types of input data are processed by an AI model to obtain intermediate data; one input data in the at least two different types of input data and other input data in the at least two different types of input data are processed by the AI model to obtain intermediate data.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to a perception processing method and device, equipment and a medium. BACKGROUND

[0002] Future mobile communication systems, such as Beyond 5th Generation (B5G) mobile communication systems or 6th Generation (6G) mobile communication systems, will have perception capabilities in addition to communication capabilities. One or more devices with perception capabilities can perceive the position, distance, speed, etc. of a target object through the transmission and reception of wireless signals, or detect, track, identify, image, etc. target objects, events or environments, etc. At present, how to perform AI-based fusion perception is a problem to be solved. SUMMARY

[0003] The embodiments of the application provide a perception processing method, device, equipment and medium, which can solve the problem of how to perform AI-based fusion perception.

[0004] In a first aspect, a perception processing method is provided, comprising:

[0005] The first node processes the first data through the first AI model to obtain second data;

[0006] The first data or the second data is related to a perception service, and the first data includes at least one of the following: at least two types of input data; intermediate data obtained by processing at least two types of input data through an AI model; and intermediate data obtained by processing one of at least two types of input data and other input data through an AI model.

[0007] In a second aspect, a perception processing device is provided, applied to a first node, comprising a first transceiver and a first processing unit.

[0008] The first processing unit is configured to process the first data through the first AI model to obtain second data;

[0009] The first data or the second data is related to a perception service, and the first data includes at least one of the following: at least two types of input data; intermediate data obtained by processing at least two types of input data through an AI model; and intermediate data obtained by processing one of at least two types of input data and other input data through an AI model.

[0010] In a third aspect, a terminal is provided, which comprises a processor and a memory, the memory storing programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method according to the first aspect.

[0011] In a fourth aspect, a terminal is provided, which comprises a processor and a communication interface, wherein the processor is configured to process first data by a first AI model to obtain second data; wherein the first data or the second data is related to a perception service, and the first data comprises at least one of the following: at least two types of input data; intermediate data obtained by processing the at least two types of input data by the AI model respectively; and intermediate data obtained by processing one of the at least two types of input data and other input data by the AI model.

[0012] In a fifth aspect, a network-side device is provided, which comprises a processor and a memory, the memory storing programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method of the perception processing according to the first aspect.

[0013] In a sixth aspect, a network-side device is provided, which comprises a processor and a communication interface, wherein the processor is configured to process first data by a first AI model to obtain second data; wherein the first data or the second data is related to a perception service, and the first data comprises at least one of the following: at least two types of input data; intermediate data obtained by processing the at least two types of input data by the AI model respectively; and intermediate data obtained by processing one of the at least two types of input data and other input data by the AI model.

[0014] In a seventh aspect, a readable storage medium is provided, which stores programs or instructions, and the programs or instructions, when executed by a processor, implement the steps of the method of the perception processing according to the first aspect.

[0015] In an eighth aspect, a wireless communication system is provided, which comprises a terminal and a network-side device, the terminal being configured to implement the steps of the method of the perception processing according to the first aspect, or the network-side device being configured to implement the steps of the method of the perception processing according to the first aspect.

[0016] In a ninth aspect, a chip is provided, which comprises a processor and a communication interface, the communication interface being coupled to the processor, and the processor being configured to execute programs or instructions to implement the steps of the method of the perception processing according to the first aspect.

[0017] In a tenth aspect, a computer program / program product is provided, which is stored in a storage medium, and which is executed by at least one processor to implement the steps of the method of the perception processing according to the first aspect.

[0018] In the embodiments of the present application, the first node processes the first data by the first AI model to obtain the second data; wherein the first data or the second data is data related to a perception service, and the first data includes at least one of the following: at least two types of input data; intermediate data obtained by processing at least two types of input data by an AI model respectively; intermediate data obtained by processing one input data of at least two types of input data and other input data of at least two types of input data by an AI model, which realizes AI-based fusion perception. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a schematic diagram of different perception modes of communication perception integration;

[0020] Figure 2 is a structural schematic diagram of a neural network;

[0021] Figure 3 is a schematic diagram of a neuron;

[0022] Figure 4 is an architectural schematic diagram of a wireless communication system provided by an embodiment of the present application;

[0023] Figure 5 is a flowchart of a method of perception processing provided by an embodiment of the present application;

[0024] Figures 6a to 6d is a schematic diagram of AI-based fusion perception provided by an embodiment of the present application;

[0025] Figure 7 is a schematic diagram of time delay spectrum information subset interception provided by an embodiment of the present application;

[0026] Figure 8 is a schematic diagram of time delay-Doppler spectrum information subset interception provided by an embodiment of the present application;

[0027] Figure 9 is a schematic diagram of multipath in a first dimension of a channel response;

[0028] Figure 10 is a schematic diagram of a device for perception processing provided by an embodiment of the present application;

[0029] Figure 11 is a schematic diagram of a terminal provided by an embodiment of the present application;

[0030] Figure 12 is a schematic diagram of a network-side device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of them. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0032] The terms "first", "second", and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second" are generally a category and do not limit the number of objects, for example, the first object can be one or more. In addition, "or" in the present application means at least one of the connected objects. For example, the protection scope of "A or B" at least covers three schemes, namely, scheme one: including A and not including B; scheme two: including B and not including A; scheme three: including A and including B. In addition, the terms "A and / or B", "at least one of A and B", "at least one of A or B" also at least cover the above three schemes, respectively. The character " / " generally represents that the objects before and after are in an "or" relationship.

[0033] The term "indication" in the present application can be a direct indication (or explicit indication) or an indirect indication (or implicit indication). Among them, the direct indication can be understood as that the sender explicitly informs the receiver of specific information, operations to be performed or requested results, etc. in the indication sent by the sender; the indirect indication can be understood as that the receiver determines the corresponding information according to the indication sent by the sender, or judges and determines the operation to be performed or the requested result according to the judgment result.

[0034] It is worth noting that the technology described in the embodiments of the present application is not limited to Long Term Evolution (LTE) / LTE-Advanced (LTE-A) systems, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA) or other systems. The terms "system" and "network" in the embodiments of the present application are often used interchangeably, and the described technology can be used in the above-mentioned systems and radio technologies, as well as in other systems and radio technologies. The following description describes a New Radio (NR) system for example purposes, and NR terminology is used in most of the following description, but these technologies can also be applied to systems other than NR systems, such as 6th Generation (6G) communication systems. th

[0035] In order to facilitate understanding of the embodiments of the present application, the following technical points are introduced first:

[0036] I. On the integration of communication and perception.

[0037] Future mobile communication systems, such as Beyond 5th Generation (B5G) mobile communication systems or 6th Generation (6G) mobile communication systems, will have not only communication capabilities but also perception capabilities. The perception capability, i.e., one or more devices with perception capability, can perceive the position, distance, speed, etc. of a target object through the transmission and reception of wireless signals, or detect, track, identify, image, etc. target objects, events or environments, etc. In the future, with the deployment of small base stations with high-frequency large-bandwidth capabilities such as millimeter waves and terahertz in 6G networks, the resolution of perception will be significantly improved compared to centimeter waves, so that 6G networks can provide more refined perception services. Typical perception functions and application scenarios are shown in Table 1.

[0038]

[0039] Table 1: Typical perception functions and application scenarios.​

[0040] Communication and sensing integration (referred to as wireless sensing integration) refers to the integrated design of communication and sensing functions within the same system through spectrum sharing and hardware sharing. While transmitting information, the system can sense information such as location, distance, and speed, and detect, track, and identify target devices or events. The communication system and the sensing system complement each other, thereby improving overall performance and bringing a better service experience.

[0041] The integration of communication and radar is a typical application of communication-sensing integration (communication-sensing fusion). In the past, radar systems and communication systems were strictly distinguished due to different research objects and focuses, and in most scenarios, the two systems were studied independently. In fact, radar and communication systems are both typical methods of information transmission, acquisition, processing, and exchange, and they share many similarities in terms of working principles, system architecture, and frequency bands. The design of integrated communication and radar systems is highly feasible, mainly in the following aspects: First, both communication and sensing systems are based on electromagnetic wave theory, using the transmission and reception of electromagnetic waves to complete information acquisition and transmission; second, both communication and sensing systems have structures such as antennas, transmitters, receivers, and signal processors, resulting in significant overlap in hardware resources; with technological advancements, their operating frequency bands also increasingly overlap; furthermore, they share similarities in key technologies such as signal modulation and reception detection, and waveform design. The integration of communication and radar systems can bring many advantages, such as cost savings, size reduction, power consumption reduction, improved spectral efficiency, and reduced mutual interference, thereby improving the overall system performance.

[0042] Based on the different target signal transmitting and receiving nodes, there are six basic sensing modes, such as... Figure 1 As shown, it specifically includes:

[0043] (1) Base station self-transmitting and self-receiving sensing. In this sensing mode, base station A transmits a target signal and performs sensing measurements by receiving the echo of the target signal.

[0044] The aforementioned target signals include at least one of reference signals, synchronization signals, data signals, and dedicated signals. Receiving or transmitting target signals can support sensing services. For example, by receiving or transmitting target signals, sensing measurements or sensing results can be obtained. Sensing results refer to those that meet sensing requirements, such as: the shape of the sensing target, 2D or 3D environment reconstruction, spatial position, orientation, displacement, speed, and acceleration; radar-based sensing for velocity, distance, angle measurement, or imaging of target objects; the presence of people or objects; and sensing of targets such as human movements, gestures, respiratory rate, heart rate, and sleep quality.

[0045] (2) Inter-BS air interface sensing. Base station B receives the target signal sent by base station A and performs sensing measurement.

[0046] (3) Uplink air interface sensing. Base station A receives the target signal sent by terminal A and performs sensing measurement.

[0047] (4) Downlink air interface sensing. Terminal B receives the target signal sent by base station B and performs sensing measurement.

[0048] (5) Terminal self-sensing. Terminal A sends a first signal and performs sensing measurement by receiving the echo of the target signal.

[0049] (6) Inter-terminal Sidelink (SL) sensing. Terminal B receives the target signal sent by terminal A and performs sensing measurement.

[0050] It is worth noting that, Figure 1 Each sensing mode in the above table takes one target signal sending node and one target signal receiving node as an example. In actual systems, one or more different sensing modes can be selected according to different sensing use cases and sensing requirements, and there can be one or more sending nodes and receiving nodes for each sensing mode. Figure 1 The sensing targets in the above table take people and vehicles as examples, and it is assumed that neither people nor vehicles carry or install signal receiving or sending devices. The sensing targets in actual scenarios are more diverse.

[0051] Optionally, the configuration information of the sensing signal includes at least one of the following:

[0052] 1) Signal resource ID (ID), used to distinguish different signal resource configurations;

[0053] 2) Signal purpose, indicating whether the signal is a signal for communication (such as channel measurement, channel estimation, synchronization, carrying data information, etc.), a signal for sensing, or a signal for both communication and sensing. Specifically, it can also be a signal for which sensing service or a signal for which type of sensing service.

[0054] 3) Waveform, such as Orthogonal Frequency Division Multiplexing (OFDM), Single-Carrier Frequency Division Multiple Access (SC-FDMA), Orthogonal Time Frequency Space (OTFS), Frequency Modulated Continuous Wave (FMCW), pulse signal, etc.

[0055] 4) Subcarrier spacing, e.g. 30 KHz for OFDM system.

[0056] 5) Guard interval, which is the time interval from the end of signal transmission to the time when the latest echo signal of the signal is received; this parameter is proportional to the maximum sensing distance; it can be calculated by c / (2R max ), where R max is the maximum sensing distance (belongs to sensing requirement information), e.g. for self-generated and self-received sensing signal, R max represents the maximum distance from the sensing signal transceiver point to the signal transmission point; in some cases, the OFDM signal cyclic prefix (CP) can play the role of minimum guard interval; c is the speed of light.

[0057] 6) Starting frequency domain position, i.e. starting frequency point, which can also be starting resource element (RE) or resource block (RB) index;

[0058] 7) Starting time domain position, i.e. starting time point, which can also be starting symbol index, time slot index, frame index;

[0059] 8) Ending frequency domain position, i.e. ending frequency point, which can be represented by ending RE or RB index;

[0060] 9) Ending time domain position, i.e. ending time point, which can be represented by ending RE or RB index;

[0061] 10) Frequency domain resource length, i.e. frequency domain bandwidth, which is inversely proportional to distance resolution, and the frequency domain bandwidth B of each signal satisfies B≥c / (2ΔR), where c is the speed of light and ΔR is the distance resolution;

[0062] 11) Time domain resource length, also known as burst duration, which is inversely proportional to Doppler resolution.

[0063] 12) Frequency domain resource interval, which represents the interval of adjacent signal frequency domain resource units and can be represented by RE number or RB number, or by density value Density, e.g. Density = 1 means that one RE in each RB is used to carry signals. The frequency domain resource interval is inversely proportional to the maximum unambiguous distance / delay, and for OFDM system, when the subcarriers are continuously mapped, the frequency domain interval is equal to the subcarrier spacing;

[0064] 13) Time domain resource interval, which is the time interval between two adjacent signal resource units, and is associated with maximum unambiguous Doppler shift or maximum unambiguous velocity.

[0065] 14) Time domain resource property, periodic transmission, semi-persistent transmission, aperiodic transmission.

[0066] 15) Signal power, for example, from -20dBm to 23dBm with a value every 2dBm.

[0067] 16) Sequence information, including sequence type information (ZC (Zadoff-Chu) sequence, pseudo-random sequence (Pseudo-Noise Sequence, PN sequence), etc.), sequence generation method, sequence length, etc.

[0068] 17) Signal direction, angle information or beam information of signal transmission.

[0069] 18) Quasi Co-Located (QCL) relationship, for example, the sensing signal includes multiple resources, each resource is associated with a Synchronization Signal and PBCH block (SSB) QCL, QCL includes Type A, B, C or D.

[0070] 19) Antenna port information, for example, the maximum number of antenna ports, antenna port index.

[0071] 20) Cyclic Prefix (CP) information, including CP type (for example, normal cyclic prefix (NCP), extended cyclic prefix (ECP) or newly designed CP dedicated for sensing measurement, etc.), CP length, etc.

[0072] 2. Introduction to artificial intelligence.

[0073] Artificial intelligence (AI) has been widely applied in various fields. AI models have various implementation methods, such as neural networks, decision trees, support vector machines, and Bayesian classifiers. The present application takes neural networks as an example for illustration, but does not limit the specific type of AI model. The structure of the neural network is as shown in Figure 2 .

[0074] Among them, the neural network is composed of neurons, and the schematic diagram of the neuron is as shown in Figure 3 . Among them, a1, a2, … a KFor input, w is weight (multiplicative coefficient), b is bias (additive coefficient), σ(.) is activation function, z = a1w1+…+a k w k +…+a K w K +b. Common activation functions include Sigmoid function, tanh function, Rectified Linear Unit (ReLU), etc.

[0075] The parameters of neural networks can be optimized by optimization algorithms. Optimization algorithms are a class of algorithms that can minimize or maximize an objective function (sometimes also called loss function). And the objective function is often a mathematical combination of model parameters and data. For example, given data X and its corresponding label Y, construct a neural network model f(.), after having the model, according to the input x, the predicted output f(x) can be obtained, and the gap between the predicted value and the true value (f(x)-Y) can be calculated, which is the loss function. If the appropriate W, b is found to make the value of the above loss function minimum, the smaller the loss value, the closer the model is to the true situation.

[0076] The common optimization algorithm at present is basically based on error Back Propagation (BP) algorithm. The basic idea of BP algorithm is that the learning process consists of two processes of forward propagation of signals and backward propagation of errors. When forward propagation, the input sample is transmitted from the input layer, processed layer by layer through each hidden layer, and transmitted to the output layer. If the actual output of the output layer does not match the expected output, the backward propagation of error is entered. Error back propagation is to transmit the output error to the input layer through the hidden layer in a certain form, and allocate the error to all units of each layer, so as to obtain the error signal of each layer unit, which is used as the basis for correcting the weight of each unit. The process of adjusting the weight of each layer through forward propagation of signals and backward propagation of errors is repeated. The process of continuously adjusting the weight is the learning and training process of the network. This process continues until the error of the network output is reduced to an acceptable level, or until the pre-set learning times are reached.

[0077] Common optimization algorithms include Gradient Descent, Stochastic Gradient Descent (SGD), mini-batch gradient descent, Momentum, Stochastic Gradient Descent with Momentum, ADAptive GRADient descent (Adagrad), Adadelta (an optimization algorithm that adjusts the learning rate for each parameter individually), root meansquare prop (RMSprop), Adaptive Moment Estimation (Adam), and the like.

[0078] These optimization algorithms, when error backpropagation, are based on the error or loss obtained from the loss function, the derivative or partial derivative of the current neuron, the learning rate, the previous gradient, the derivative or partial derivative, and the like, to obtain the gradient, and pass the gradient to the previous layer.

[0079] The AI model in the present application can also be referred to as an AI unit, a machine learning (ML) model, an ML unit, an AI structure, an AI function, an AI feature, a machine learning model, a neural network, a neural network function, a neural network function, or the like. Alternatively, the AI model can refer to a processing unit capable of implementing a specific algorithm, formula, processing flow, capability, or the like related to AI, or the AI model can be a processing method, algorithm, function, module or unit for a specific data set, or the AI model can be a processing method, algorithm, function, module or unit running on an AI or ML related hardware such as a graphics processing unit (GPU), neural processing unit (NPU), tensor processing unit (TPU), application specific integrated circuit (ASIC), or the like. The present application does not make specific limitations.

[0080] Alternatively, the identification of the AI model can also be referred to as the identification of the AI unit, the identification of the AI structure, the identification of the AI algorithm, or the identification of the specific data set associated with the AI model, or the identification of the specific scene, environment, channel feature, device related to AI or ML, or the identification of the function, feature, capability or module related to AI or ML. The present application does not make specific limitations.

[0081] The above AI functionality can be understood as an AI algorithm functionality, which can contain one or more AI models for a terminal (e.g., a user equipment (UE)).

[0082] Figure 4A block diagram of a wireless communication system to which embodiments of the present application can be applied is shown. The wireless communication system includes a terminal 41 and a network-side device 42. The terminal 41 can be a terminal-side device such as a mobile phone, a Tablet Personal Computer, a Laptop Computer, a notebook, a Personal Digital Assistant, a palm PC, a netbook, an Ultra-mobile Personal Computer (UMPC), a Mobile Internet Device (MID), an Augmented Reality (AR) device, a Virtual Reality (VR) device, a robot, a wearable device, a flight vehicle, a Vehicle User Equipment (VUE), a shipboard device, a Pedestrian User Equipment (PUE), a smart home (a home device with a wireless communication function such as a refrigerator, a television, a washing machine, or furniture), a game console, a Personal Computer (PC), a kiosk, or a self-service machine. The wearable device includes a smart watch, a smart bracelet, a smart earphone, smart glasses, smart jewelry (a smart bracelet, a smart necklace, a smart ring, a smart necklace, a smart anklet, a smart necklace, etc.), a smart wristband, smart clothes, etc. The vehicle-mounted device can also be referred to as a vehicle-mounted terminal, a vehicle-mounted controller, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip, or a vehicle-mounted unit, etc. It should be noted that the specific type of the terminal 41 is not limited in the embodiments of the present application. The network-side device 42 can include an access network device or a core network device. The access network device can also be referred to as a Radio Access Network (RAN) device, a radio access network function, or a radio access network unit. The access network device can include a base station, a Wireless Local Area Network (WLAN) Access Point (AP), or a Wireless Fidelity (WiFi) node, etc.The base station can be referred to as a Node B (NB), an evolved Node B (eNB), a next generation Node B (gNB), a New Radio Node B (NR Node B), an access point, a relay station (RBS), a serving base station (SBS), a base transceiver station (BTS), a radio base station, a radio transceiver, a basic service set (BSS), an extended service set (ESS), a home Node B (HNB), a home evolved Node B, a transmit / receive point (TRP), or some other suitable terminology in the art, and is not limited to a particular technical terminology, provided that the same technical effect is achieved. It should be noted that in the embodiments of the present application, only the base station in the NR system is taken as an example for introduction, and the specific type of the base station is not limited.

[0083] The core network device can also be referred to as a core network node, a core network function, or a core network network element, etc., which includes but is not limited to at least one of the following: a mobility management entity (MME), an access and mobility management function (AMF), a session management function (SMF), a user plane function (UPF), a policy control function (PCF), a policy and charging rules function (PCRF), an edge application server discovery function (EASDF), a unified data management (UDM), a unified data repository (UDR), a home subscriber server (HSS), a centralized network configuration (CNC), a network repository function (NRF), a network exposure function (NEF), a local NEF (L-NEF), a binding support function (BSF), an application function (AF), a location management function (LMF), a gateway mobile location center (GMLC), a network data analytics function (NWDAF), etc. It should be noted that only the core network device in the NR system is taken as an example for introduction in the embodiments of the present application, and the specific type of the core network device is not limited. If the name of the core network device mentioned in the embodiments of the present application changes in the subsequent protocol version (for example, 6G), it is also within the protection scope of the present application.

[0084] Optionally, the core network device can be implemented by one or more function modules in one device, or can be implemented by multiple devices together, and the embodiments of the present application do not make specific limitations thereto. It can be understood that the above function modules can be network elements in a hardware device, can be software function modules running on a special hardware, or can be virtualized function modules instantiated on a platform (for example, a cloud platform).

[0085] The method, device, equipment and medium for perception processing provided by the embodiments of the present application will be described in detail below in combination with the accompanying drawings and some embodiments and application scenarios thereof.

[0086] Referring to Figure 5 , the embodiments of the present application provide a method for perception processing, and the specific steps include:

[0087] Step 51: The first node processes the first data through the first AI model to obtain second data;

[0088] Wherein, the first node processing the first data through the first AI model includes but is not limited to reasoning. That is, the first node processes the first data through the first AI model to obtain the second data, that is, the reasoning result.

[0089] Wherein, the first data or the second data is data related to perception service, and the first data includes at least one of the following:

[0090] 1) at least two different types of input data;

[0091] That is, the first node can process at least two different types of input data through the first AI model to obtain the second data, so as to realize AI-based fusion perception processing.

[0092] Optionally, the at least two different types of input data include first input data and second input data, referring to Figure 6a .

[0093] Optionally, the first input data and the second input data have at least the following differences:

[0094] 1) different devices for perception data;

[0095] 2) different ways of perception data;

[0096] 3) different RATs;

[0097] 4) different perception modes;

[0098] For example, single-base perception or double-base perception.

[0099] 5) different frequency bands.

[0100] For example, the frequency bands include, but are not limited to, sub-6GHz frequency bands, millimeter wave frequency bands.

[0101] Optionally, the sensor includes at least one of the following: a visible light camera, an infrared camera, a global navigation satellite system (GNSS), a laser radar, a millimeter wave radar, a thermometer, a hygrometer, a barometer, a gyroscope, an accelerometer, a magnetometer, a gravity sensor, a sonar, and a rain gauge.

[0102] Optionally, the data sensed by the sensor includes sensing data generated by different sensors described above; wherein the sensing data includes at least one of the following: received signals, channel information, spectrum information calculated based on the channel information or the received signals, measurement quantities, properties of sensing targets, states of sensing targets, and sensing results.

[0103] 2) intermediate data obtained by processing at least two different types of input data through AI models respectively;

[0104] That is, the first node can process at least two different types of intermediate data through the first AI model to obtain second data, so as to realize AI-based fusion sensing processing, wherein the at least two different types of intermediate data are obtained by processing at least two different types of input data through AI models respectively.

[0105] Optionally, the intermediate data obtained by processing at least two different types of input data through AI models respectively includes: first intermediate data and second intermediate data, the first intermediate data is obtained by processing the first input data through a second AI model, and the second intermediate data is obtained by processing the second input data through a third AI model, see Figure 6b .

[0106] It can be understood that the second AI model or the third AI model can be deployed on the first node together with the first AI model, or the second AI model or the third AI model is deployed on a node other than the first node.

[0107] 3) intermediate data obtained by processing one of the at least two different types of input data and other input data through AI models.

[0108] Optionally, the intermediate data obtained by processing one of the at least two different types of input data and other input data through AI models includes at least one of the following:

[0109] 1) the first input data and third intermediate data, the third intermediate data being obtained by processing the second input data by a fourth AI model, see Figure 6c ;

[0110] 2) the second input data and fourth intermediate data, the fourth intermediate data being obtained by processing the first input data by a fifth AI model, see Figure 6d .

[0111] It can be understood that the fourth AI model or the fifth AI model can be deployed on the first node together with the first AI model, or the fourth AI model or the fifth AI model is deployed on a node other than the first node.

[0112] In the embodiments of the present application, the first AI model, the second AI model, the third AI model, the fourth AI model, and the fifth AI model can be the same AI model or different AI models.

[0113] Optionally, the first node or the second node or the third node or the fourth node or the fifth node in this paper can include but is not limited to at least one of a terminal and a network side device, wherein the network side device can include but is not limited to at least one of a base station, a core network device, a third party server, an operation maintenance management (OAM), etc., wherein the core network device can include but is not limited to at least one of a sensing function (Sensing Function) network element, a positioning management function, etc.

[0114] Optionally, the base station includes a base station responsible for sensing signal transmission or reception and sensing data processing, or a base station not responsible for sensing signal transmission or reception and only responsible for sensing data processing.

[0115] Optionally, the terminal includes a terminal responsible for sensing signal transmission or reception and sensing data processing, or a terminal not responsible for sensing signal transmission or reception and only responsible for sensing data processing.

[0116] The signaling transmission between the base station and the terminal, the terminal A and the terminal B is through the radio resource control (RRC) signaling or the medium access control (MAC) control element (CE) or layer 1 signaling or other new definition perception signaling; the signaling transmission between the perception function network element and the terminal can be through the non-access layer (NAS) signaling (forwarded through the AMF) and / or through the RRC signaling or the MAC CE or the layer 1 signaling or other new definition perception signaling; the interaction between the perception function network element and the base station can be forwarded to the wireless access network through the N2 interface by the AMF; or the perception function network element sends to the UPF, and the UPF sends to the wireless access network (base station) through the N3 interface; or through a newly defined interface to the wireless access network; the signaling transmission between the base stations can be through the Xn interface.

[0117] Optionally, the perception function network element can also be called a perception network element or a perception function network element, which can be on the access network side or the core network side, refers to a network node in the core network and / or access network responsible for at least one of the following functions: perception request processing, perception resource scheduling, perception information interaction, perception data processing, etc., which can be based on the upgrade of AMF or LMF in the 5G network, or other network nodes or newly defined network nodes. Specifically, the function characteristics of the perception function network element can include at least one of the following:

[0118] 1) Interact with the wireless signal sending device or the wireless signal measuring device (including the target terminal or the service base station of the target terminal or the base station associated with the target area) to obtain the target perception result or the value of the perception measurement quantity (uplink measurement quantity or downlink measurement quantity) sent by the wireless signal measuring device, wherein the target information includes perception processing request, perception capability, perception auxiliary data, perception measurement quantity type, perception resource configuration information, etc., so as to obtain the value of the target perception result or the perception measurement quantity (uplink measurement quantity or downlink measurement quantity) sent by the wireless signal measuring device; wherein the wireless signal can also be called a perception signal.

[0119] 2) Determine the perception method to be used according to the type of perception service, the perception service consumer information, the required quality of service (QoS) requirement information, the perception capability of the wireless signal sending device, the perception capability of the wireless signal measuring device, etc. The perception method can include: base station A sends to base station B, or base station sends to terminal, or base station A self-sends and self-receives, or terminal sends to base station, or terminal self-sends and self-receives, or terminal A sends to terminal B, etc.

[0120] 3) deciding the sensing device to serve the sensing service according to the type of the sensing service, the information of the consumer of the sensing service, the required sensing QoS requirement information, the sensing capability of the wireless signal transmitting device, the sensing capability of the wireless signal measuring device, and other factors, wherein the sensing device includes the wireless signal transmitting device or the wireless signal measuring device.

[0121] 4) overall coordination and scheduling of the required resources for the sensing service, such as corresponding configuration of the sensing resources of the base station or the terminal;

[0122] 5) data processing of the value of the sensing measurement quantity, or calculation to obtain the sensing result. Further, verification of the sensing result, estimation of the sensing accuracy, and the like.

[0123] In an embodiment of the present application, the first data satisfies at least one of the following: the first data is generated by the first node, and the first data is acquired by the first node from a second node.

[0124] In an embodiment of the present application, if the first data is acquired by the first node from a second node, the method further includes:

[0125] The first node sends first information to the second node, and the first information is used to indicate the content of the first data or the format of the first data.

[0126] In an embodiment of the present application, the first data or the second data or the first input data or the second input data or the first intermediate data or the second intermediate data or the third intermediate data or the fourth intermediate data or the sensed data or the content of the sensing data includes at least one of the following:

[0127] 1) a received signal;

[0128] The received signal includes at least one of a reference signal, a synchronization signal, a data signal, and a dedicated signal. By acquiring the signal, the sensing service can be supported, for example, the sensing measurement quantity or the sensing result can be obtained by acquiring the signal. The sensing result refers to the result meeting the sensing requirement, for example: the shape of the sensing target, 2D or 3D environment reconstruction, spatial position, orientation, displacement, moving speed, acceleration; radar type sensing for target object speed, distance, angle measurement or imaging; whether a person or an object exists; the action, gesture, breathing rate, heart rate, sleep quality, and the like of the sensing target such as a person.

[0129] 2) channel information;

[0130] Optionally, the channel information includes at least one of the following: time domain channel response, frequency domain channel response, complex result of channel response, amplitude or phase, I or Q data.

[0131] 3) spectrum information calculated based on channel information or received signal;

[0132] Optionally, the spectrum information includes at least one of the following: time delay (distance) spectrum, Doppler (velocity) spectrum, angle spectrum, time delay (distance)-Doppler (velocity) spectrum, time delay (distance)-angle spectrum, time delay (distance)-Doppler (velocity)-angle spectrum, time-Doppler spectrum (micro-Doppler spectrum).

[0133] Optionally, the spectrum information can refer to complex results, for example, the time delay-Doppler spectrum refers to the time delay, Doppler index and corresponding complex value (including phase information) in the two-dimensional spectrum; the spectrum information can also refer to power spectrum, for example, the time delay-Doppler spectrum refers to the time delay, Doppler index and corresponding power value (without phase information) in the two-dimensional spectrum.

[0134] Optionally, the spectrum information can also be spectrum information calculated based on the received signal.

[0135] 4) measurement quantity;

[0136] Optionally, the measurement quantity includes at least one of the following: time delay, Doppler, angle, intensity (power); wherein the time delay can be the time delay of different paths.

[0137] Optionally, the basic measurement quantity can be the quantization result of the actual value, or can be soft information; wherein the soft information can be data described by mean and variance, or confidence interval and confidence degree. For example, the mean and variance of Gaussian distribution, or two values and respective probabilities.

[0138] 5) attribute or state of the perceived target;

[0139] Optionally, the attribute or state of the perceived target includes at least one of the following: distance, velocity, orientation, spatial position, acceleration.

[0140] 6) perception result.

[0141] Optionally, the perception result includes at least one of the following: whether the perceived target exists, trajectory, action, expression, vital sign, number of perceived targets, imaging result, weather, air quality, shape, material, composition.

[0142] Optionally, the first data or the second data or the first input data or the second input data or the first intermediate data or the second intermediate data or the third intermediate data or the fourth intermediate data can include different contents of the above-mentioned six contents 1) to 6); for example, the first data or the first input data or the second input data as input data can include channel information, and the second data as output data can include a basic measurement quantity; for another example, the first data or the first input data or the second input data as input data can include channel information, and the second data as output data can include a perception result; for another example, the first data or the first input data or the second input data can include channel information, and the first intermediate data or the second intermediate data or the third intermediate data or the fourth intermediate data can include spectrum information calculated based on the channel information or the received signal, and the second data as output data can include a perception result.

[0143] Optionally, the first data or the second data or the first input data or the second input data or the first intermediate data or the second intermediate data or the third intermediate data or the fourth intermediate data can include the same contents of the above-mentioned six contents 1) to 6); for example, the input data and the output data include a received signal or channel information, at this time, the AI model in the perception node can overcome channel estimation errors and noise to obtain more accurate data.

[0144] In an embodiment of the present application, the contents of the first data or the second data or the first input data or the second input data or the first intermediate data or the second intermediate data or the third intermediate data or the fourth intermediate data further include at least one of the following:

[0145] 1) perception signal identification information;

[0146] Optionally, the perception signal identification information includes an index of a reference signal, etc.

[0147] 2) perception measurement configuration identification information;

[0148] 3) perception service information;

[0149] Optionally, the perception service information includes a perception service identifier.

[0150] 4) data subscription identifier;

[0151] 5) measurement quantity purpose;

[0152] Optionally, the measurement quantity purpose includes at least one of the following: communication, perception, wireless perception.

[0153] 6) time information;

[0154] 7) perception node information;

[0155] Optionally, the perception node information comprises at least one of: a perception node identifier, a perception node location, a perception direction of the perception node.

[0156] 8) perception link information;

[0157] Optionally, the perception link information comprises at least one of: a perception link sequence number, a transceiving node identifier.

[0158] 9) measurement quantity description information;

[0159] Optionally, the measurement quantity description information comprises at least one of: a) a form, such as an amplitude value, a phase value, a complex value combining amplitude and phase; b) a resource type, such as a time domain measurement result, a frequency domain resource measurement result.

[0160] 10) measurement quantity index information.

[0161] Optionally, the measurement quantity index information comprises at least one of: a signal-to-noise ratio (SNR), a perception SNR, a reference signal receiving power (RSRP). In an embodiment of the present application, the perception service comprises at least one of: detecting whether a target exists, detecting a target number, positioning, trajectory tracking, speed detection, distance detection, angle detection, acceleration detection, material analysis, component analysis, shape detection, category division, radar cross section (RCS) detection, polarization scattering characteristic detection, fall detection, intrusion detection, indoor positioning, gesture recognition, lip-reading recognition, gait recognition, expression recognition, face recognition, respiration monitoring, heart rate monitoring, pulse monitoring, humidity or brightness or temperature or atmospheric pressure monitoring, air quality monitoring, weather condition monitoring, environment reconstruction, topography, building or vegetation distribution detection, people flow or vehicle flow detection, crowd density or vehicle density detection, etc.; or, the perception service can also refer to a category of perception services, i.e., multiple different perception services are classified according to certain characteristics, such as being classified into detection type perception services (such as including intrusion detection, fall detection), parameter estimation type perception services (distance, angle, speed calculation), recognition type perception services (action recognition, identity recognition), etc., can also be classified according to a perception range (short distance perception, medium distance perception, long distance perception), classified according to a perception precision (coarse granularity perception, fine granularity perception, etc.), classified according to power consumption or energy consumption, classified according to resource occupation, etc.

[0162] In an embodiment of the present application, the format of the first data or the second data or the first input data or the second input data or the first intermediate data or the second intermediate data or the third intermediate data or the fourth intermediate data comprises at least one of the following:

[0163] 1) the dimension of the channel information;

[0164] For example, the frequency domain channel response information on a certain symbol (one-dimensional channel information), the time-frequency domain channel response information on multiple symbols (two-dimensional channel information), the time-frequency-space domain channel response information on multiple symbols and multiple antennas (three-dimensional channel information), and the scale of different dimensions, i.e., the number of sampling points or the sampling interval (such as the time-frequency domain density).

[0165] 2) the range of the spectrum information;

[0166] It can be the range of different spectrum information (or referred to as the truncation window for different dimensions of spectrum information), i.e., the spectrum information can be the complete spectrum information calculated according to the channel information, or a subset of the complete spectrum information, such as the spectrum information subset corresponding to a specific time delay or Doppler range in the time delay-Doppler spectrum, and for example, the information of the path or sampling point whose power or amplitude exceeds the preset threshold in the time delay-Doppler spectrum; it can also indicate the upper limit value of the number of paths of a specific spectrum or the upper limit value of the number of sampling points supported by the AI model for input; it can also indicate the minimum granularity of the specific spectrum supported by the AI model for input, i.e., the interval between two adjacent sampling points (corresponding to the perception resolution);

[0167] For example, the spectrum information includes partial spectrum information, such as the information of the N2-N3 sampling points or the N2-N3 paths in the time delay spectrum (such as Figure 7 the middle frame line part).

[0168] For another example, the spectrum information subset is the part in the time delay-Doppler spectrum whose Doppler absolute value is less than X1 and whose time delay value is less than X2 (such as Figure 8 the middle frame line part).

[0169] 3) the number of measurement quantities supported by the AI model for input at one time;

[0170] 4) the quantization manner of the value of the supported measurement quantity;

[0171] Optionally, the quantization manner includes the quantization granularity.

[0172] 5) the type of soft information.

[0173] The soft information can be data described by mean and variance, or confidence interval and confidence.

[0174] In an embodiment of the present application, the first AI model or the second AI model or the third AI model or the fourth AI model or the fifth AI model is determined by at least one of the following manners: determined by the first node based on second information configured by other nodes, determined autonomously by the first node, agreed by a protocol, determined by the first node based on high layer signaling, selected by the first node from a first AI model pool based on third information.

[0175] In an embodiment of the present application, the second information includes at least one of the following:

[0176] 1) model structure information;

[0177] Optionally, the model structure information can include at least one of the following: type of AI model (such as Gaussian process, support vector machine, various neural networks (fully connected neural network, convolutional neural network, recurrent neural network or residual network, combination of multiple small networks, such as fully connected + convolution, convolution + residual), etc.) and structure of the model (such as number of layers of neural network, number of neurons of each layer, activation function, etc.).

[0178] 2) hyperparameter configuration of supported AI model;

[0179] Optionally, the hyperparameter configuration of the supported AI model includes at least one of the following: related parameters in kernel function, related parameters in activation function, related parameters in normalization layer, etc.

[0180] 3) supported AI model data processing mode, i.e. data preprocessing mode before data is input into the AI model. Optionally, the model data processing mode can include but is not limited to at least one of the following: normalization, up-sampling, down-sampling, etc.

[0181] 4) supported AI model allowed period, i.e. how long AI model is executed once every time;

[0182] 5) supported AI model update period, i.e. how long AI model is updated once every time;

[0183] 6) supported AI model update information;

[0184] Optionally, the AI model update information includes at least one of the following: update information of kernel function, update information of hyperparameter, update information of prediction mode, update information of calculation mode.

[0185] 7) complexity information of supported AI model, for example, number of floating point operations (FLOPs) of model inference, such as 100 iterations, hardware conditions, calculation conditions;

[0186] 8) available AI resources, including computing or storage resources available for AI use;

[0187] 9) supported AI framework or algorithm;

[0188] 10) AI model quantity information;

[0189] 11) AI model category information;

[0190] 12) AI model identification information;

[0191] 13) AI model priority information;

[0192] 14) AI model attribute information;

[0193] 15) AI model precision information;

[0194] 16) AI model error information;

[0195] 17) AI model feature information;

[0196] 18) adaptation environment information;

[0197] 19) processing delay information;

[0198] 20) AI model output result fusion mode information;

[0199] 21) AI model life cycle information;

[0200] 22) AI model input data information;

[0201] 23) AI model output data information.

[0202] In an embodiment of the present application, the third information includes at least one of the following: model error information, network side device mobility information, terminal mobility information, network side device environment information, terminal environment information, perception accuracy requirement information, perception service information, and model priority information.

[0203] In an embodiment of the present application, the input data is obtained by at least one of the following:

[0204] 1) based on perception data processing of one device;

[0205] 2) based on joint processing of perception data of multiple different devices;

[0206] 3) based on perception data obtained based on one perception mode;

[0207] Optionally, the sensing mode includes at least one of the following: base station self-sending and self-receiving, base station A sending and base station B receiving, base station sending and terminal receiving, terminal sending and base station receiving, terminal self-sending and self-receiving, terminal A sending and terminal B receiving; or the sensing mode can also refer to single-base sensing or double-base sensing.

[0208] 4) Joint processing of sensing data obtained based on multiple different sensing modes;

[0209] 5) Sensor-based sensing data processing;

[0210] 6) Joint processing of sensing data based on sensors and wireless sensing;

[0211] Optionally, the sensor includes at least one of the following: visible light camera, infrared camera, Global Navigation Satellite System (GNSS), laser radar, millimeter wave radar, thermometer, hygrometer, barometer, gyroscope, accelerometer, magnetometer, gravity sensor, sonar, rain gauge, rotation vector sensor, etc.

[0212] Optionally, the data format of the sensor can be binary format, ASCII format, etc., or a data format associated with a specific sensor type, such as YUV, RGB, etc. for image sensors (cameras).

[0213] Optionally, the accelerometer is used to measure the acceleration applied to the device, including acceleration along the x-axis, y-axis, and z-axis. Further, it can also be divided into results containing gravity acceleration, results not containing gravity acceleration, results containing bias compensation, and results not containing bias compensation.

[0214] Optionally, the gyroscope is used to measure the rotation rate (radians / second) around the x, y, and z axes of the device, which can be represented as a three-dimensional vector similar to the accelerometer.

[0215] Optionally, the magnetometer is used to monitor changes in the Earth's magnetic field, measuring the geomagnetic field strength data (in microtesla) along each of the three coordinate axes. This sensor is usually not used directly, but combined with other sensors to obtain rotation angle information.

[0216] Optionally, the rotation vector sensor is used to obtain the terminal angle through the combination of different sensors, and the format is (corresponding to the angle of rotation of the terminal around the x, y, and z axes, or the rotation angle relative to the East-North-Up / North-East-Down coordinate axes, respectively).

[0217] 7) Based on different frequency bands;

[0218] Optionally, the perception data of different frequency bands includes, but is not limited to, the perception data of sub-6GHz frequency band, the perception data of millimeter wave frequency band, and the perception data of THz frequency band.

[0219] 8) Based on the joint processing of the perception data of different radio access technologies (RATs).

[0220] Optionally, the RATs include, but are not limited to, 4G, 5G, 6G, Wireless Fidelity (Wifi), Ultra Wide Band (UWB), Bluetooth, etc.

[0221] In an embodiment of the present application, different input data, or different intermediate data, or different weights or proportions of the input data and the intermediate data are associated with the perception-related indicators corresponding to the input data.

[0222] The perception processing method of the present application can be used in at least the following scenarios:

[0223] 1) Fusion of wireless perception and sensor perception; wherein the sensors include visible light cameras, infrared cameras, GNSS, lidar, millimeter wave radar, thermometers, hygrometers, barometers, gyroscopes, accelerometers, magnetometers, gravity sensors, sonar, rain gauges, etc.; wherein the lidar can obtain ultra-high angular resolution by emitting ultra-narrow laser beams and receiving reflected echoes; at the same time, the optical frequency band has ultra-high bandwidth, so that the lidar has ultra-high range resolution. However, existing commercial lidars generally do not have speed measurement function. The advantages of visual sensors such as visible light cameras and infrared cameras over wireless perception are that they can image and identify visual features (e.g., people, vehicles, etc.) based on visual images and algorithms. The disadvantages of visual sensors compared to wireless perception are that they cannot measure speed and have poor range measurement performance. The fusion of wireless perception and sensor perception includes the following:

[0224] a) Fusion of the perception results of the same target or the same spatial area by the wireless perception system and the sensor, to improve the perception accuracy;

[0225] b) Fusion of the perception results of different targets or different spatial areas by the wireless perception system and the sensor, to expand the spatial coverage of perception to meet the perception needs;

[0226] c) Fusion of the perception results of different times by the wireless perception system and the sensor, to improve the update rate of perception.

[0227] 2) Multi-point cooperative perception fusion; that is, fusion processing of perception data from different devices or fusion processing of data from different perception modes of the same device; the perception data of different devices can be obtained from processing of perception data of one perception mode or from processing of perception data obtained from multiple different perception modes; the perception mode includes at least one of the following: base station self-initiated self-reception, base station A-initiated base station B-reception, base station-initiated terminal reception, terminal-initiated base station reception, terminal self-initiated self-reception, terminal A-initiated terminal B-reception; or the perception mode can also refer to single-base perception or double-base perception;

[0228] 3) Fusion of perception data from multiple RATs, that is, supporting processing of perception data obtained based on more than one wireless access technology (including but not limited to 4G, 5G, 6G, Wifi, UWB, Bluetooth, etc.) for perception acquisition.

[0229] In the embodiment of the application, the first node processes the first data through the first AI model to obtain the second data; wherein the first data or the second data is data related to a perception service, and the first data includes at least one of the following: at least two different types of input data; intermediate data obtained by processing at least two different types of input data through an AI model respectively; intermediate data obtained by processing one input data of at least two different types of input data and other input data of at least two different types of input data through an AI model, realizing AI-based fusion perception

[0230] The optional implementation modes of the application will be described below in combination with Embodiment 1, Embodiment 2 and Embodiment 3.

[0231] Embodiment 1

[0232] Referring to Figure 6a , the specific steps are as follows:

[0233] Step 1: The first node obtains first input data and second input data of a first AI model; wherein the first AI model is deployed in the first node;

[0234] Optionally, the first input data and the second input data can be one of the following cases:

[0235] 1) The first input data and the second input data are respectively wireless perception data and sensor perception data;

[0236] 2) The first input data and the second input data are respectively wireless perception data or sensor perception data from different devices

[0237] 3) The first input data and the second input data are respectively data of different sensing modes from the same device; for example, the first input data is data of monostatic sensing, and the second input data is data of bistatic sensing

[0238] 4) The first input data and the second input data are respectively sensing data of different frequency bands, for example, sensing data of a sub-6GHz frequency band and sensing data of a millimeter wave frequency band

[0239] 5) The first input data and the second input data are respectively sensing data of different RATs; including but not limited to sensing data of 4G, 5G, 6G, Wifi, UWB, Bluetooth, etc.

[0240] Optionally, the first input data or the second input data includes at least one of data generated by the first node and data obtained by the first node from other nodes;

[0241] Optionally, if the first input data or the second input data is data obtained by the first node (for example, a terminal) from other nodes (for example, network side devices), before the other nodes send the first data to the first node, the other nodes need to receive first information, which indicates at least one of the following: related information of the content of the first input data or the second input data, and related information of the format of the first input data or the second input data.

[0242] Optionally, before step 1, the first node determines one or more first AI models, and the specific manner includes any one of the following:

[0243] 1) The first node (for example, a terminal) receives second information sent by other nodes (for example, network side devices), and the second information includes configuration information of one or more first AI models; the first node determines one or more first AI models based on the second information;

[0244] 2) The first node (for example, a terminal) sends request information to other nodes (for example, network side devices), and the request information is used to request to configure one or more first AI models; the first node receives second information sent by other nodes; wherein, the second information includes configuration information of one or more first AI models; the first node determines one or more first AI models based on the second information;

[0245] 3) The first node determines one or more first AI models based on self-determination;

[0246] 4) The first node determines one or more first AI models based on a protocol;

[0247] 5) The first node determines one or more first AI models based on high layer signaling;

[0248] 6) The first node selects one or more first AI models from a first AI model pool based on the third information; wherein the first AI model pool includes K AI models, and K is a positive integer.

[0249] Step 2: The first node processes the first input data and the second input data through the first AI model (for example, based on inference of the first AI model) to obtain second data (i.e., output data);

[0250] Optionally, the first node sends the second data to other nodes, for example, the second data is processed by AI models or non-AI models of the other nodes to obtain final output data.

[0251] Embodiment 2

[0252] Referring to Figure 6b , the specific steps are as follows:

[0253] Step 1: The third node obtains the first input data of the second AI model, and the fourth node obtains the second input data of the third AI model; wherein the third node or the fourth node can be the same node or different nodes as the first node, that is, the second AI model or the third AI model can be deployed on the first node or on a node other than the first node.

[0254] Optionally, the first input data and the second input data can be one of the following cases:

[0255] 1) The first input data and the second input data are respectively wireless sensing data and sensor sensing data;

[0256] 2) The first input data and the second input data are respectively wireless sensing data or sensor sensing data from different devices;

[0257] 3) The first input data and the second input data are respectively data of different sensing modes from the same device; for example, the first input data is single-base sensing data, and the second input data is double-base sensing data;

[0258] 4) The first input data and the second input data are respectively sensing data of different frequency bands, for example, sub-6GHz frequency band sensing data and millimeter wave frequency band sensing data;

[0259] 5) The first input data and the second input data are respectively sensing data of different RATs; including but not limited to 4G, 5G, 6G, Wifi, UWB, Bluetooth, etc.

[0260] Optionally, when the third node or the fourth node is the same as the first node, the first input data or the second input data includes at least one of data generated by the first node and data obtained by the first node from other nodes.

[0261] Optionally, if the first input data or the second input data is data obtained by the first node (e.g., a terminal) from other nodes (e.g., a network side device), the other nodes need to receive first information before sending the first data to the first node, the first information indicating at least one of the following: related information of the content of the first input data or the second input data, and related information of the format of the first input data or the second input data.

[0262] Optionally, before step 1, the first node or the third node or the fourth node determines the first AI model, the second AI model or the third AI model, and the specific manner includes any one of the following:

[0263] 1) The first node or the third node or the fourth node (e.g., a terminal) receives second information sent by other nodes (e.g., a network side device), the second information including configuration information of one or more first AI models; the first node or the third node or the fourth node determines the first AI model, the second AI model or the third AI model based on the second information;

[0264] 2) The first node or the third node or the fourth node (e.g., a terminal) sends request information to other nodes (e.g., a network side device), the request information being used to request configuration of the first AI model, the second AI model or the third AI model; the first node or the third node or the fourth node receives second information sent by other nodes; wherein the second information includes configuration information of the first AI model, the second AI model or the third AI model; the first node or the third node or the fourth node determines the first AI model, the second AI model or the third AI model based on the second information;

[0265] 3) The first node or the third node or the fourth node determines the first AI model, the second AI model or the third AI model based on self-determination;

[0266] 4) The first node or the third node or the fourth node determines the first AI model, the second AI model or the third AI model based on a protocol;

[0267] 5) The first node or the third node or the fourth node determines the first AI model, the second AI model or the third AI model based on high layer signaling;

[0268] 6) The first node or the third node or the fourth node selects the first AI model, the second AI model or the third AI model from the first AI model pool based on the third information; wherein the first AI model pool comprises K AI models, and K is a positive integer.

[0269] Step 2: The second AI model of the third node processes the first input data (for example, based on inference of the second AI model), to obtain first intermediate data, the third AI model of the fourth node processes the second input data (for example, based on inference of the third AI model), to obtain second intermediate data, and the first node processes the first intermediate data and the second intermediate data through the first AI model (for example, based on inference of the first AI model), to obtain second data (i.e., output data);

[0270] Optionally, the first node sends the second data to other nodes, for example, processes the second data through AI models or non-AI models of the other nodes, to obtain final output data.

[0271] Embodiment 3

[0272] Referring to Figure 6d , the specific steps are as follows:

[0273] Step 1: The fifth node obtains first input data of the fifth AI model, and the first node obtains second input data of the first AI model;

[0274] Wherein, the fifth node and the first node can be the same node or different nodes, that is, the fifth AI model and the first AI model can be deployed on the first node or a node other than the first node;

[0275] Optionally, the first input data and the second input data can be one of the following cases:

[0276] 1) The first input data and the second input data are respectively wireless sensing data and sensor sensing data;

[0277] 2) The first input data and the second input data are respectively wireless sensing data or sensor sensing data from different devices;

[0278] 3) The first input data and the second input data are respectively data of different sensing modes from the same device; for example, the first input data is single-base sensing data, and the second input data is double-base sensing data;

[0279] 4) The first input data and the second input data are respectively sensing data of different frequency bands, for example, sub-6GHz frequency band sensing data and millimeter wave frequency band sensing data;

[0280] 5) The first input data and the second input data are respectively sensing data of different RATs; including but not limited to sensing data of 4G, 5G, 6G, Wifi, UWB, Bluetooth, etc.

[0281] Optionally, when the third node or the fourth node is the same node as the first node, the first input data or the second input data includes at least one of data generated by the first node and data obtained by the first node from other nodes.

[0282] Optionally, if the first input data or the second input data is data obtained by the first node (e.g., a terminal) from other nodes (e.g., network side devices), the other nodes need to receive first information before sending the first data to the first node, the first information indicating at least one of the following: related information of the content of the first input data or the second input data, and related information of the format of the first input data or the second input data.

[0283] Optionally, before step 1, the first node or the fifth node determines the first AI model or the fifth AI model, and the specific manner includes any one of the following:

[0284] 1) The first node or the fifth node (e.g., a terminal) receives second information sent by other nodes (e.g., network side devices), the second information including configuration information of one or more first AI models; the first node or the fifth node determines the first AI model or the fifth AI model based on the second information;

[0285] 2) The first node or the fifth node (e.g., a terminal) sends request information to other nodes (e.g., network side devices), the request information being used to request configuration of the first AI model or the fifth AI model; the first node or the fifth node receives second information sent by other nodes; wherein the second information includes configuration information of the first AI model or the fifth AI model; the first node or the fifth node determines the first AI model or the fifth AI model based on the second information;

[0286] 3) The first node or the fifth node determines the first AI model or the fifth AI model based on self-determination;

[0287] 4) The first node or the fifth node determines the first AI model or the fifth AI model based on a protocol;

[0288] 5) The first node or the fifth node determines the first AI model or the fifth AI model based on high layer signaling;

[0289] 6) The first node or the fifth node selects the first AI model or the fifth AI model from a first AI model pool based on third information; wherein the first AI model pool includes K AI models, K being a positive integer.

[0290] Step 2: the fifth AI model of the fifth node processes the first input data (e.g. based on inference of the fifth AI model) to obtain fourth intermediate data, and the first node processes the fourth intermediate data and the second input data (e.g. based on inference of the first AI model) to obtain the second data (i.e. output data);

[0291] Optionally, the first node sends the second data to other nodes, e.g. the second data is processed by AI models or non-AI models of the other nodes to obtain final output data.

[0292] In the above embodiments 1-3, when one AI model corresponds to multiple input data, the weight or proportion of the multiple input data (e.g. the first input data and the second input data) needs to be determined; the weight or proportion can be associated with the perception-related index corresponding to the input data; for example, the greater the perception-related index, the higher the weight or proportion of the input data. Similarly, when there are multiple intermediate data (e.g. in embodiment 2), the weight or proportion of the multiple intermediate data (e.g. the first intermediate data and the second intermediate data) needs to be determined; the weight or proportion can be associated with the perception-related index corresponding to the input data; optionally, the respective weight or proportion of the multiple input data can be input into the AI model to assist the AI model to output more accurate output information such as perception results;

[0293] In the above embodiments 1-3, when there are multiple input data or multiple intermediate data, the multiple input data or multiple intermediate data need to be preprocessed, e.g. unified coordinate system, etc.; for example, when the multiple input data or multiple intermediate data are spectral information, the coordinate intervals or granularities of the multiple spectral information are aligned;

[0294] In the above embodiments 1-3, the preprocessing process for the multiple input data or multiple intermediate data can be performed at the other nodes before the other nodes send the multiple input data or multiple intermediate data to the node where the AI model is deployed, or at the node where the AI model is located; the preprocessing method can be notified to the node performing the preprocessing.

[0295] Optionally, the perception-related index includes at least one of the following: a first index, a second index, and a third index, the first index is related to received power, the second index is related to interference and noise power, and the third index is related to received power, interference and noise power;

[0296] The perception-related index includes at least one of the following (1)-(3):

[0297] (1) the first index;

[0298] The first index is a linear average value (in W) of a received power of a target associated path in a channel response measured for a target signal on a resource unit carrying the target signal;

[0299] (2) a second index;

[0300] The second index comprises at least one of the following (2a)-(2c):

[0301] (2a) a fourth index;

[0302] The fourth index is a linear average value (in W) of a sum of a power of a path other than the target associated path in a channel response of the target signal on a target resource and a linear average value of an interference and noise power of a signal other than the target signal on a first resource, the first resource being the target resource or a resource other than the target resource; the target resource comprises a resource unit carrying the first signal, and the resource unit can be a time domain resource unit or a frequency domain resource unit;

[0303] Optionally, the fourth index = total received power - the first index; wherein the total received power can be represented as a linear average value (in W) of a total received power (including received powers of signals of a serving cell and non-serving cells, adjacent channel interference and thermal noise, etc.) on the target resource; or the total received power = RSSI*K1, K1 being a coefficient, and a measurement resource of the RSSI being the target resource or another resource (for example, a resource configured by high layer signaling).

[0304] (2b) a fifth index;

[0305] The fifth index is a linear average value of an interference and noise power of a signal other than the target signal on a second resource, the second resource being the target resource or a resource other than the target resource;

[0306] Optionally, the fifth index = total received power - first signal received power; wherein the first signal received power is a reference signal receiving power (RSRP) of the first signal.

[0307] (2c) a sixth index;

[0308] The sixth index is a linear average value (in W) of a power of a path other than the target associated path in a channel response of the target signal on the target resource;

[0309] Optionally, the sixth index = RSRP of the first signal - the first index;

[0310] (3) a third index;

[0311] The third index includes at least one of (3a)-(3d) below:

[0312] (3a) a seventh index;

[0313] The seventh index represents the first index divided by the fourth index, i.e., seventh index = first index / fourth index;

[0314] (3b) an eighth index;

[0315] The eighth index represents the first index divided by the fifth index, i.e., eighth index = first index / fifth index;

[0316] (3c) a ninth index;

[0317] The ninth index represents the first index divided by the sixth index, i.e., ninth index = first index / sixth index;

[0318] (3d) a tenth index;

[0319] The tenth index represents the first index divided by a first received power and then multiplied by a preset first coefficient, the first received power representing total received power on a target resource, or the first received power representing a product of a received signal strength indication (RSSI) and a preset second coefficient, the measurement resource of the RSSI being the target resource or another resource, i.e., tenth index = K2*first index / total received power, K2 being a coefficient.

[0320] Optionally, the method for obtaining the sensing target-associated path includes:

[0321] The first node (e.g., a terminal) performs channel estimation based on a target signal and a received signal corresponding to the target signal to obtain a channel response;

[0322] The first node transforms the channel response to a first dimension;

[0323] The first node determines a sensing target-associated path in a path corresponding to the first dimension;

[0324] The first dimension includes at least one of: a delay dimension; a Doppler dimension; an azimuth angle dimension; and an elevation angle dimension.

[0325] Optionally, the first node determines the sensing target-associated path in the path corresponding to the first dimension, including:

[0326] The first node selects, as the sensing target-associated path, a path satisfying a first condition in the path corresponding to the first dimension;

[0327] The first condition comprises at least one of the following:

[0328] 1) a first parameter of the path is greater than or equal to a first threshold or is in a first interval range;

[0329] 2) a difference between the first parameter of the path and a first parameter of a first-arrival path or a reference path is greater than or equal to a second threshold or is in a second interval range;

[0330] 3) a second parameter of the path satisfies a preset modulation rule;

[0331] The first parameter comprises at least one of the following: amplitude, power, intensity, energy, Doppler, time delay, angle;

[0332] The second parameter comprises at least one of the following: amplitude, power, intensity, energy, phase.

[0333] Optionally, the first node selects, in the path corresponding to the first dimension, a path satisfying a second condition as the perception target associated path, comprising:

[0334] The first node determines a first path set in the path corresponding to the first dimension, and a third parameter of each path in the first path set is greater than or equal to a third threshold, the third parameter comprising at least one of the following: amplitude, power, intensity, energy;

[0335] The first node determines, in the first path set, a path satisfying the first condition as the perception target associated path.

[0336] Optionally, the first index is calculated in the following manner:

[0337] The first node performs channel estimation based on a transmitted first signal (denoted as X(k) below) and a received signal corresponding to the first signal (denoted as Y(k) below) to obtain a channel response (Channel Response), i.e., H(k) = Y(k) / X(k), where k = 0, 1, 2, …, K-1 represents the resource unit index. After the first node obtains the channel response H(k), it is transformed to a first dimension, and a perception target associated path is determined in the first dimension. Then, the power of the perception target associated path is calculated as the first index, and if the perception target associated path comprises multiple paths, the sum of the powers of the multiple paths is calculated as the first index.

[0338] The first dimension comprises at least one of the following: time delay dimension; Doppler dimension; azimuth angle dimension; elevation angle dimension, for example, time delay-Doppler dimension, time delay-Doppler-angle dimension, etc.

[0339] For example, H(f) is a channel response, where f = 0, 1, 2, …, N-1 represents a frequency domain sample point (e.g. a subcarrier index), which can be transformed to a time delay dimension (first dimension) by inverse Fourier transform of H(f); for another example, H(f, t) is a channel response, where f = 0, 1, 2, …, N-1 represents a frequency domain sample point (e.g. a subcarrier index), and t = 0, 1, 2, …, M-1 represents a time domain sample point (e.g. an OFDM symbol index), which can be transformed to a time delay-Doppler dimension (first dimension) by inverse Fourier transform along the frequency domain dimension and Fourier transform along the time domain dimension; for another example, H(f, t, s) is a channel response, where f = 0, 1, 2, …, N-1 represents a frequency domain sample point (e.g. a subcarrier index), t = 0, 1, 2, …, M-1 represents a time domain sample point (e.g. an OFDM symbol index), and s = 0, 1, 2, …, P-1 represents a spatial domain sample point (an antenna index or a port index), which can be transformed to a time delay-Doppler-angle dimension (first dimension) by inverse Fourier transform along the frequency domain dimension, Fourier transform along the time domain dimension, and Fourier transform along the antenna domain dimension.

[0340] Optionally, a method for determining a path (referred to as a sensing path) associated with a sensing target from a channel response measured from a first signal:

[0341] Step 1: determining a first path set. The paths in the first path set include paths whose amplitudes or powers or intensities or energies exceed a preset threshold among all paths after the channel response is transformed to the first dimension. Figure 9 For example, paths 0, 1, 2, and 3 are paths in the first path set.

[0342] Optionally, the preset threshold can be set to be higher than a noise threshold or a noise and interference threshold.

[0343] It can be understood that the step of determining the first path set is optional, and the path associated with the sensing target can also be determined only according to Step 2.

[0344] Step 2: selecting a path satisfying a first condition from the first path set or from all paths as the path associated with the sensing target.

[0345] Optionally, the first condition includes at least one of the following:

[0346] 1) the amplitude or power or intensity or energy of the path exceeds a preset threshold or is within a preset interval range; for example, the preset threshold is 5 times higher than a noise threshold.

[0347] 2) Doppler of the path exceeds a preset threshold or is in a preset interval range;

[0348] 3) Time delay of the path exceeds a preset threshold or is in a preset interval range;

[0349] 4) Angle of the path exceeds a preset threshold or is in a preset interval range;

[0350] 5) Difference of amplitude or power or intensity or energy of the path and the first-arrived path (e.g., Line-of-Sight (LOS) path) or reference path (e.g., path of signal reflected by known target (e.g., Reconfigurable Intelligence Surface (RIS) or Backscatter or other known passive target, etc.)) exceeds a preset threshold or is in a preset interval range;

[0351] 6) Doppler difference of the path and the first-arrived path (e.g., LOS path) or reference path (e.g., path of signal reflected by known target (e.g., RIS or Backscatter device or other known passive target, etc.)) exceeds a preset threshold or is in a preset interval range;

[0352] 7) Time delay difference of the path and the first-arrived path (e.g., LOS path) or reference path (e.g., path of signal reflected by known target (e.g., RIS or Backscatter device or other known passive target, etc.)) exceeds a preset threshold or is in a preset interval range;

[0353] 8) Angle difference of the path and the first-arrived path (e.g., LOS path) or reference path (e.g., path of signal reflected by known target (e.g., RIS or Backscatter device or other known passive target, etc.)) exceeds a preset threshold or is in a preset interval range;

[0354] 9) Amplitude or power or intensity or energy or phase of the path meets a specific modulation rule, which is a modulation rule of Tag or backscatter device or RIS, i.e., the path associated with the perceived target can be a path modulated and reflected by the Tag or backscatter device or RIS.

[0355] It should be noted that the above first conditions can also be based on the results of a period of time statistics; for example, the proportion of the above indicators (e.g., Doppler of the path, time delay of the path, etc.) exceeding the preset threshold or being in the preset interval range reaches a preset proportion within a preset time window, or the number of the above indicators (e.g., Doppler of the path, time delay of the path, etc.) exceeding the preset threshold or being in the preset interval range reaches a preset number within a preset time window;

[0356] The preset threshold or the set interval range is sent by other devices to the receiving device, and is determined by other devices according to the perception prior information or the perception demand. Alternatively, the preset threshold or the set interval range is determined by the receiving device according to the perception prior information or the perception demand.

[0357] The perception prior information or the perception demand includes at least one of the following:

[0358] 1) a perception service or a perception service type;

[0359] Optionally, the perception service can include but is not limited to at least one of the following: detecting whether a target exists, positioning, speed detection, distance detection, angle detection, acceleration detection, material analysis, component analysis, shape detection, category division, radar cross section (RCS) detection, polarization scattering characteristic detection, fall detection, intrusion detection, quantity statistics, indoor positioning, gesture recognition, lip reading, gait recognition, expression recognition, face recognition, respiration monitoring, heart rate monitoring, pulse monitoring, humidity or brightness or temperature or atmospheric pressure monitoring, air quality monitoring, weather condition monitoring, environment reconstruction, topography, building or vegetation distribution detection, people flow or vehicle flow detection, crowd density, vehicle density detection, etc. The perception service type can be classified according to certain characteristics of a plurality of different perception services, for example, classified according to functions into detection type perception services (for example, including intrusion detection, fall detection), parameter estimation type perception services (distance, angle, speed calculation), recognition type perception services (action recognition, identity recognition), etc. It can also be classified according to the range of perception (short distance perception, medium distance perception, long distance perception), classified according to the degree of perception (coarse granularity perception, fine granularity perception, etc.), classified according to power consumption or energy consumption, classified according to resource occupation, etc. If the perception service is respiration monitoring, the corresponding normal respiration frequency can be determined according to the gender and age of the person (for example, male: 13-21 times / minute, female: 15-20 times / minute; adult: 12-20 times / minute, child: about 30-40 times / minute), which can be used as perception prior information.

[0360] 2) a perception target area;

[0361] Optionally, the perception target area includes a position area of a perception object, or a position area that needs to be imaged or environment reconstructed; for example, a preset interval range of a time delay of a perception target correlation radius is determined according to the approximate position or distance of the perception object;

[0362] 3) a perception object type;

[0363] Optionally, the perception objects are classified according to possible motion characteristics of the perception objects, and each perception object type includes information such as a typical motion speed range, a typical motion acceleration range, and a typical RCS range of a typical perception object.

[0364] 4) The number of targets perceived;

[0365] Optionally, the camera perception results, as a form of prior perception information, can be used to determine the number of perceived targets;

[0366] For example, Figure 9 The intermediate diameters 0, 1, 2, and 3 are the diameters in the first diameter set, where diameters 2 and 3 are the sensing diameters that satisfy the first condition (e.g., their time delay meets a preset threshold), and diameters 0 and 1 are the diameters associated with other scatterers. Figure 9 The horizontal axis represents the first dimension, and the vertical axis represents the normalized amplitude, power, intensity, or energy.

[0367] For frequency range 1, the reference point for the first indicator can be the antenna connector of the receiving device, such as a terminal. For frequency range 1, if the receiving device has multiple receiving channels, the first indicator measured and reported by the receiving device cannot be lower than the indicator of any single receiving channel. For frequency range 2, the first indicator measured for a certain receiving channel needs to be obtained by measuring the combined signal on multiple antenna elements corresponding to that receiving channel.

[0368] Optionally, the first indicator can be calculated as follows:

[0369] Optionally, when calculating the received power of the sensing target correlation path, it can also be the power of the sensing target correlation path in the first dimension and... The difference is used as the first indicator, where N1 represents the number of paths associated with the perceived target. It represents the average power of multiple paths outside the first path set in the first dimension.

[0370] In one embodiment of this application, the received power of the first signal is calculated as follows:

[0371] The received power of the first signal can be obtained by the receiving device after obtaining the channel response H(k), transforming it to the first dimension, determining the first path set in the first dimension, and then calculating the sum of the power of all paths in the first path set.

[0372] Optionally, the received power of the first signal can be calculated as follows:

[0373] The received power of the first signal can also be the sum of the powers of all paths in the first path set in the first dimension. The difference, where N2 represents the number of paths in the first path set.

[0374] Optional method for calculating total received power: Total Received Power

[0375] Optional, the calculation method for the third indicator:

[0376] The channel response H(k) is processed by the first filter to obtain H. filter1 (k), then according to H filter1 The received signal Y after the first filtering process is calculated from (k) and the first signal X(k). filter1 (k), i.e., Y filter1 (k)=H filter1 (k)X(k). Then subtract the received signal Y(k) after the first filtering process from the received signal Y(k). filter1 (k) thus obtaining the interference and noise signal Y σ1 (k), i.e., Y σ1 (k)=Y(k)-Y filter1 (k), and then calculate the third index.

[0377] The first filtering process is used to eliminate noise and interference in the first dimension, as well as paths associated with non-perceptual targets. For example, the first filtering process will... Figure 9 The amplitude, power, intensity, or energy of all paths other than the sensing target-correlated path are set to zero. The channel response H after the first filtering process... filter1 (k) does not contain noise and interference, nor does it contain paths associated with non-perceived targets; it only contains paths associated with perceived targets.

[0378] Optionally, the fourth indicator can be calculated as follows:

[0379] The channel response H(k) is processed by a second filter to obtain H. filter2 (k), then according to H filter2 The received signal Y after the second filtering process is calculated from the first signal X(k) and the first signal X(k). filter2 (k), i.e., Y filter2 (k)=H filter2 (k)X(k). Then subtract the received signal Y(k) after the second filtering process from the received signal Y(k). filter2 (k) thus obtaining the interference and noise signal Y σ2 (k), i.e., Y σ2 (k)=Y(k)-Y filter2 (k), and then calculate the fourth index.

[0380] The second filtering process can be noise interference suppression processing in the first dimension (e.g.) Figure 9(The amplitude, power, intensity, or energy of all paths other than the first path set are set to zero), or minimum mean square error (MMSE) filtering is applied. The channel response H after the second filtering process is... filter2 (k) does not contain noise and interference, but only contains paths from the first path set.

[0381] Optionally, the fourth indicator can be calculated as follows:

[0382] Based on the average power of multiple paths outside the first path set in the first dimension The fourth index P was calculated. σ2 ,Right now Where N represents the number of sampling points in the first dimension.

[0383] If the receiving device identifies multiple sensing targets, or if the receiving device obtains the number of sensing targets based on prior sensing information or sensing requirements, the following methods are available:

[0384] Method 1: Calculate the perception-related indicators for each perception target separately. For example, in Figure 9 The path associated with each sensing target is determined separately, and then the sensing-related indicators corresponding to each sensing target are calculated separately. When calculating the third indicator corresponding to a certain sensing target (such as sensing target A), there are two methods: that is, the third indicator of sensing target A = total received power - the first indicator of sensing target A; or, the third indicator of sensing target A = total received power - the first indicator of sensing target A - the first indicator of sensing target B; (assuming there are two sensing targets: A and B). Similarly, there are also two ways to calculate the fifth indicator: the fifth indicator of sensing target A = the RSRP of the first signal - the first indicator of sensing target A; or, the fifth indicator of sensing target A = the RSRP of the first signal - the first indicator of sensing target A - the first indicator of sensing target B; (assuming there are two sensing targets: A and B).

[0385] Method 2: Calculate a perception-related index for multiple perception targets. For example, in Figure 9 The process involves identifying paths associated with any given sensing target and then treating all these paths as paths associated with that target; this is equivalent to treating multiple sensing targets as a virtual sensing target and then calculating the sensing-related indicators corresponding to that virtual sensing target.

[0386] See Figure 10 The embodiments of this application provide a sensing processing apparatus applied to a first node. The apparatus 100 includes a first transceiver unit 101 and a first processing unit 102.

[0387] The first processing unit 102 is used to process the first data through the first AI model to obtain the second data;

[0388] Wherein, the first data or the second data is data related to perception business, and the first data includes at least one of the following: at least two different types of input data; intermediate data obtained by processing at least two different types of input data through an AI model; and intermediate data obtained by processing one type of input data and other types of input data through an AI model.

[0389] In one embodiment of this application, the at least two different types of input data include first input data and second input data, and the first input data and second input data have at least the following differences:

[0390] 1) The devices used to sense the data are different;

[0391] 2) The ways of perceiving data are different;

[0392] 3) Different RATs;

[0393] 4) Different perception modes;

[0394] For example, monostatic sensing or bistatic sensing.

[0395] 5) Different frequency bands.

[0396] For example, frequency bands include, but are not limited to: sub-6GHz band and millimeter wave band.

[0397] In one embodiment of this application, intermediate data obtained by processing at least two different types of input data through an AI model includes: first intermediate data and second intermediate data. The first intermediate data is obtained by processing the first input data through a second AI model, and the second intermediate data is obtained by processing the second input data through a third AI model.

[0398] In one embodiment of this application, intermediate data obtained by processing one type of input data from at least two different types of input data and other input data from at least two different types of input data through an AI model includes at least one of the following:

[0399] The first input data and the third intermediate data, wherein the third intermediate data is obtained by processing the second input data through the fourth AI model;

[0400] The second input data and the fourth intermediate data, wherein the fourth intermediate data is obtained by processing the first input data through the fifth AI model.

[0401] In one embodiment of this application, the sensor includes at least one of the following: a visible light camera, an infrared camera, a global navigation satellite system, a lidar, a millimeter-wave radar, a thermometer, a hygrometer, a barometer, a gyroscope, an accelerometer, a magnetometer, a gravity sensor, a sonar, and a rain gauge.

[0402] In one embodiment of this application, the second, third, fourth, or fifth AI model is deployed on the first node, or deployed on a node other than the first node.

[0403] In one embodiment of this application, the first data satisfies at least one of the following: the first data is generated by the first node, or the first data is obtained by the first node from the second node.

[0404] In one embodiment of this application, the first transceiver unit 101 is used to send first information to the second node, the first information being used to indicate the content of the first data or the format of the first data.

[0405] In one embodiment of this application, the content of the first data or the second data or the first input data or the second input data or the first intermediate data or the second intermediate data or the third intermediate data or the fourth intermediate data includes at least one of the following: received signal, channel information, spectral information calculated based on channel information or received signal, measurement quantity, attribute of the perceived target, state of the perceived target, and perception result.

[0406] In one embodiment of this application, the content of the first data or the second data or the first input data or the second input data or the first intermediate data or the second intermediate data or the third intermediate data or the fourth intermediate data may further include at least one of the following: sensing signal identification information, sensing measurement configuration identification information, sensing service information, data subscription identification, measurement purpose, time information, sensing node information, sensing link information, measurement description information, and measurement index information.

[0407] In one embodiment of this application, the format of the first data or the second data or the first input data or the second input data or the first intermediate data or the second intermediate data or the third intermediate data or the fourth intermediate data includes at least one of the following: the dimension of channel information, the range of spectral information, the number of measurements that can be input into the AI ​​model at one time, the quantization method of the values ​​of the supported measurements, and the type of soft information.

[0408] In one embodiment of this application, the first AI model, the second AI model, the third AI model, the fourth AI model, or the fifth AI model is determined by at least one of the following methods: determined by the first node based on second information configured by other nodes, determined autonomously by the first node, agreed upon by the protocol, determined by the first node based on higher-level signaling, or selected by the first node from the first AI model pool based on third information.

[0409] In one embodiment of this application, the second information includes at least one of the following: model structure information, hyperparameter configuration of supported AI models, supported model data processing methods, supported model allowed cycles, supported model update cycles, supported model update information, supported model complexity information, available AI resources, supported AI frameworks or algorithms, number of models, model category information, model identification information, model priority information, model attribute information, model accuracy information, model error information, model feature information, adaptation environment information, processing latency information, fusion method information of model output results, model lifecycle information, information on model input data, and information on model output data.

[0410] In one embodiment of this application, the third information includes at least one of the following: model error information, mobility information of network-side devices, mobility information of terminals, environmental information of network-side devices, environmental information of terminals, perception accuracy requirement information, perception service information, and model priority information.

[0411] In one embodiment of this application, the input data is obtained through at least one of the following methods:

[0412] Based on the processing of sensing data from a single device;

[0413] Joint processing of sensing data from multiple different devices;

[0414] Processing of sensory data acquired based on a sensing mode;

[0415] Joint processing of sensing data acquired from multiple different sensing modes;

[0416] Sensor-based sensing data processing;

[0417] Joint processing of sensor data based on sensor and wireless sensing;

[0418] Joint processing of sensing data from different frequency bands;

[0419] Joint processing of perception data based on different RATs.

[0420] In one embodiment of this application, different input data, or different intermediate data, or different weights or proportions of input data and intermediate data, wherein the weights or proportions are associated with perception-related indicators corresponding to the input data.

[0421] The apparatus provided in this application embodiment can achieve... Figure 5 The various processes implemented in the method embodiments achieve the same technical effect, and will not be described again here to avoid repetition.

[0422] This application embodiment also provides a terminal, including a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement, for example... Figure 5 The steps in the method embodiment shown are illustrated. This terminal embodiment corresponds to the above-described terminal-side method embodiment. All implementation processes and methods of the above-described method embodiments can be applied to this terminal embodiment and achieve the same technical effect. The terminal can be... Figure 5 The sensor processing device shown. Specifically, Figure 11 A schematic diagram of the hardware structure of a terminal to implement an embodiment of this application.

[0423] The terminal 1100 includes, but is not limited to, at least some of the following components: radio frequency unit 1101, network module 1102, audio output unit 1103, input unit 1104, sensor 1105, display unit 1106, user input unit 1107, interface unit 1108, memory 1109, and processor 1010.

[0424] Those skilled in the art will understand that the terminal 1100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1010 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 11 The terminal structure shown does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0425] It should be understood that, in this embodiment, the input unit 1104 may include a graphics processor 11041 and a microphone 11042. The graphics processor 11041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1106 may include a display panel 11061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1107 includes at least one of a touch panel 11071 and other input devices 11072. The touch panel 11071 is also called a touch screen. The touch panel 11071 may include a touch detection device and a touch controller. Other input devices 11072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0426] In this embodiment, after receiving downlink data from the network-side device, the radio frequency unit 1101 can transmit it to the processor 1010 for processing; in addition, the radio frequency unit 1101 can send uplink data to the network-side device. Typically, the radio frequency unit 1101 includes, but is not limited to, antennas, amplifiers, transceivers, couplers, low-noise amplifiers, duplexers, etc.

[0427] The memory 1109 can be used to store software programs or instructions, as well as various data. The memory 1109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1109 may include volatile memory or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1109 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0428] Processor 1110 may include one or more processing units; optionally, processor 1110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1110.

[0429] The processor 1110 is used to process the first data through a first AI model to obtain the second data; the first data or the second data is data related to perception business, and the first data includes at least one of the following: at least two different types of input data; intermediate data obtained by processing the at least two different types of input data through the AI ​​model; intermediate data obtained by processing one type of input data and other types of input data through the AI ​​model.

[0430] It is understood that the implementation process of each implementation method mentioned in this embodiment can be referred to the method embodiment. Figure 5 The relevant descriptions and the achievement of the same or corresponding technical effects will not be repeated here to avoid duplication.

[0431] This application also provides a network-side device. For example... Figure 12 As shown, the network-side device 1200 includes: a processor 1201, a network interface 1202, and a memory 1203. This network-side device can be... Figure 5 The sensor processing apparatus shown. The network interface 1202 is, for example, a common public radio interface (CPRI).

[0432] Specifically, the network-side device 1200 in this application embodiment further includes: instructions or programs stored in memory 1203 and executable on processor 1201, wherein processor 1201 calls the instructions or programs in memory 1203 to execute. Figure 5 The methods executed by each unit shown achieve the same technical effect, and will not be elaborated here to avoid repetition.

[0433] This application embodiment also provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the above-described functionality. Figure 5 The various processes in the method embodiments can achieve the same technical effect, and will not be described again here to avoid repetition.

[0434] The processor mentioned above is the processor in the terminal described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk. In some examples, the readable storage medium may be a non-transient readable storage medium.

[0435] This application embodiment also provides a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the above. Figure 5 The various processes in the method embodiments can achieve the same technical effect, and will not be described again here to avoid repetition.

[0436] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0437] This application embodiment also provides a computer program / program product, which is stored in a storage medium and executed by at least one processor to implement the above. Figure 5 The various processes in the method embodiments can achieve the same technical effect, and will not be described again here to avoid repetition.

[0438] This application also provides a communication system, including: a terminal and a network-side device, wherein the terminal can be used to perform the above-mentioned functions. Figure 5 The steps of the method shown, or the network-side device, can be used to perform the above. Figure 5 Figure 5 The steps of the method shown.

[0439] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0440] From the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of computer software products plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes several instructions to cause the terminal or network-side device to execute the methods described in the various embodiments of this application.

[0441] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other implementations under the guidance of this application without departing from the spirit and scope of the claims. All of these implementations are within the protection scope of this application.

Claims

1. A method for sensory processing, characterized in that, include: The first node processes the first data using the first artificial intelligence (AI) model to obtain the second data; Wherein, the first data or the second data is data related to perception business, and the first data includes at least one of the following: at least two different types of input data; intermediate data obtained by processing at least two different types of input data through an AI model; and intermediate data obtained by processing one type of input data and other types of input data through an AI model.

2. The method according to claim 1, characterized in that, The at least two different types of input data include first input data and second input data, and the first input data and second input data have at least the following differences: Different devices are used to sense data; The ways of perceiving data are different; Different RATs; Different perception modes; Different frequency bands.

3. The method according to claim 1 or 2, characterized in that, Intermediate data obtained by processing at least two different types of input data through an AI model include: first intermediate data and second intermediate data. The first intermediate data is obtained by processing the first input data through a second AI model, and the second intermediate data is obtained by processing the second input data through a third AI model.

4. The method according to claim 1 or 2, characterized in that, Intermediate data obtained by processing one type of input data from at least two different types of input data and other input data from at least two different types of input data through an AI model, including at least one of the following: The first input data and the third intermediate data, wherein the third intermediate data is obtained by processing the second input data through the fourth AI model; The second input data and the fourth intermediate data, wherein the fourth intermediate data is obtained by processing the first input data through the fifth AI model.

5. The method according to claim 3 or 4, characterized in that, The second, third, fourth, or fifth AI model is deployed on the first node, or on a node other than the first node.

6. The method according to claim 1, characterized in that, The first data satisfies at least one of the following: the first data is generated by the first node, or the first data is obtained by the first node from the second node.

7. The method according to claim 7, characterized in that, If the first data is obtained by the first node from the second node, the method further includes: The first node sends first information to the second node, the first information being used to indicate the content or format of the first data.

8. The method according to claim 1, 2, 3, 4, or 7, characterized in that, The content of the first data or the second data or the first input data or the second input data or the first intermediate data or the second intermediate data or the third intermediate data or the fourth intermediate data includes at least one of the following: received signal, channel information, spectral information calculated based on channel information or received signal, measurement quantity, attribute of the perceived target, state of the perceived target, and perception result.

9. The method according to claim 8, characterized in that, The content of the first data or the second data or the first input data or the second input data or the first intermediate data or the second intermediate data or the third intermediate data or the fourth intermediate data may further include at least one of the following: sensing signal identification information, sensing measurement configuration identification information, sensing service information, data subscription identification, measurement purpose, time information, sensing node information, sensing link information, measurement description information, and measurement index information.

10. The method according to claim 1, 2, 3, 4, or 7, characterized in that, The format of the first data, the second data, the first input data, the second input data, the first intermediate data, the second intermediate data, the third intermediate data, or the fourth intermediate data includes at least one of the following: the dimension of the channel information, the range of the spectral information, the number of measurements that can be input into the AI ​​model at one time, the quantization method of the values ​​of the supported measurements, and the type of soft information.

11. The method according to claim 1, 3, or 4, characterized in that, The first AI model, or the second AI model, or the third AI model, or the fourth AI model, or the fifth AI model is determined by at least one of the following methods: determined by the first node based on second information configured by other nodes, determined autonomously by the first node, agreed upon by the protocol, determined by the first node based on higher-level signaling, or selected by the first node from the first AI model pool based on third information.

12. The method according to claim 11, characterized in that, The second information includes at least one of the following: model structure information, hyperparameter configuration of supported AI models, supported model data processing methods, allowed model cycles, supported model update cycles, supported model update information, supported model complexity information, available AI resources, supported AI frameworks or algorithms, number of models, model category information, model identification information, model priority information, model attribute information, model accuracy information, model error information, model feature information, adaptation environment information, processing latency information, fusion method information of model output results, model lifecycle information, information on model input data, and information on model output data.

13. The method according to claim 11, characterized in that, The third information includes at least one of the following: model error information, network-side device mobility information, terminal mobility information, network-side device environmental information, terminal environmental information, perception accuracy requirement information, perception service information, and model priority information.

14. The method according to claim 1, characterized in that, The input data is obtained through at least one of the following methods: Based on the processing of sensing data from a single device; Joint processing of sensing data from multiple different devices; Processing of sensory data acquired based on a sensing mode; Joint processing of sensing data acquired from multiple different sensing modes; Sensor-based sensing data processing; Joint processing of sensor data based on sensor and wireless sensing; Joint processing of sensing data from different frequency bands; Joint processing of perception data based on different RATs.

15. The method according to claim 1, characterized in that, Different input data, or different intermediate data, or different weights or proportions of input data and intermediate data, wherein the weights or proportions are associated with perception-related indicators corresponding to the input data.

16. A sensing processing apparatus, applied to a first node, characterized in that, include: First transceiver unit and first processing unit; The first processing unit is used to process the first data through a first AI model to obtain the second data; wherein the first data or the second data is data related to perception business, and the first data includes at least one of the following: at least two different types of input data; intermediate data obtained by processing the at least two different types of input data through the AI ​​model; intermediate data obtained by processing one type of input data and other types of input data through the AI ​​model.

17. The apparatus according to claim 16, characterized in that, The at least two different types of input data include first input data and second input data. The first input data and the second input data have at least the following differences: Different devices are used to sense data; The ways of perceiving data are different; Different RATs; Different perception modes; Different frequency bands.

18. A terminal, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the perceptual processing method as described in any one of claims 1 to 16.

19. A network-side device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the perceptual processing method as described in any one of claims 1 to 16.

20. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the perceptual processing method as described in any one of claims 1 to 15.

21. A computer program product, characterized in that, Includes computer instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 15.

Images