Method and device for monitoring reasoning performance of AI unit and communication equipment

Abstract

The invention discloses an AI unit reasoning performance monitoring method and device and communication equipment, and belongs to the technical field of communication. The AI unit reasoning performance monitoring method comprises the steps that first equipment obtains sensing measurement related data and communication measurement related data, generating first information based on the sensing measurement related data and the communication measurement related data; the first device sends the first information to a second device; wherein the first information comprises a first data set, the first data set comprises a monitoring sample used for carrying out AI unit reasoning performance monitoring, the first information is used for the second equipment to generate a first index, and the first index comprises an AI unit reasoning performance monitoring index.

Description

Technical Field

[0001] This application belongs to the field of communication technologies, and particularly relates to a method, apparatus, and communication device for monitoring the inference performance of an AI unit. Background Art

[0002] Currently, the network can obtain perception results through sensors or sensing measurement signals, and sensing measurement has become one of the main research directions in the current communication field. In addition, with the development of Artificial Intelligence (AI) technology, AI units have also been applied to communication systems, such as AI-based beam prediction (for example, obtaining predicted beam information through AI unit inference, and the AI unit is also called an AI model), AI-based Channel State Information (CSI) prediction, AI-based positioning, etc. To ensure the accuracy of the output results of the AI unit, the inference performance of the AI unit is usually monitored. However, how to monitor the inference performance of the AI unit based on the perception results is not yet clear. Summary of the Invention

[0003] Embodiments of this application provide a method, apparatus, and communication device for monitoring the inference performance of an AI unit, which can solve the problem that it is not yet clear how to monitor the inference performance of the AI unit based on the perception results.

[0004] In a first aspect, a method for monitoring the inference performance of an AI unit is provided, which is executed by a first device. The method includes:

[0005] The first device obtains perception measurement-related data and communication measurement-related data, and generates first information based on the perception measurement-related data and the communication measurement-related data;

[0006] The first device sends the first information to a second device;

[0007] Wherein, the first information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of the AI unit, and the first information is used for the second device to generate a first metric, and the first metric includes an AI unit inference performance monitoring metric.

[0008] In a second aspect, a method for monitoring the inference performance of an AI unit is provided, which is executed by a second device. The method includes:

[0009] The second device receives the first information sent by the first device;

[0010] The second device monitors the inference performance of the AI unit based on the first information, and generates a first metric, and the first metric includes an AI unit inference performance monitoring metric;

[0011] Among them, the first information is generated based on perception measurement-related data and communication measurement-related data. The first information includes a first data set, and the first data set includes monitoring samples for monitoring the inference performance of the AI unit.

[0012] In a third aspect, a method for monitoring the inference performance of an AI unit is provided, which is executed by a third device. The method includes:

[0013] The third device sends perception measurement-related data or a first signal to the first device, and the first signal is used to obtain the perception measurement-related data.

[0014] Among them, the perception measurement-related data is used by the first device to generate first information. The first information includes a first data set, and the first data set includes monitoring samples for monitoring the inference performance of the AI unit. The first information is used by the second device to generate a first metric, and the first metric includes a monitoring metric for the inference performance of the AI unit.

[0015] In a fourth aspect, a monitoring device for the inference performance of an AI unit is provided, which is applied to a first device. The device includes:

[0016] An acquisition module, configured to acquire perception measurement-related data and communication measurement-related data, and generate first information based on the perception measurement-related data and the communication measurement-related data.

[0017] A first sending module, configured to send the first information to a second device.

[0018] Among them, the first information includes a first data set, and the first data set includes monitoring samples for monitoring the inference performance of the AI unit. The first information is used by the second device to generate a first metric, and the first metric includes a monitoring metric for the inference performance of the AI unit.

[0019] In a fifth aspect, a monitoring device for the inference performance of an AI unit is provided, which is applied to a second device. The device includes:

[0020] A second receiving module, configured to receive the first information sent by the first device.

[0021] A monitoring module, configured to monitor the inference performance of the AI unit based on the first information and generate a first metric, where the first metric includes a monitoring metric for the inference performance of the AI unit.

[0022] Among them, the first information is generated based on perception measurement-related data and communication measurement-related data. The first information includes a first data set, and the first data set includes monitoring samples for monitoring the inference performance of the AI unit.

[0023] In a sixth aspect, a monitoring device for the inference performance of an AI unit is provided, which is applied to a third device. The device includes:

[0024] A third sending module, configured to send perception measurement-related data or a first signal to a first device, where the first signal is used to obtain the perception measurement-related data;

[0025] Wherein, the perception measurement-related data is used for the first device to generate first information, the first information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of the AI unit, the first information is used for the second device to generate first metrics, and the first metrics include AI unit inference performance monitoring metrics.

[0026] In a seventh aspect, a communication device is provided. The communication device includes a processor and a memory. The memory stores a program or instruction that can run on the processor. When the program or instruction is executed by the processor, the steps of the method described in the first aspect or the second aspect or the third aspect are implemented.

[0027] In an eighth aspect, a communication device is provided, including a processor and a communication interface. When the communication device is the first device, the processor is configured to obtain perception measurement-related data and communication measurement-related data, and generate first information based on the perception measurement-related data and the communication measurement-related data. The communication interface is configured to send the first information to the second device. Wherein, the first information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of the AI unit, and the first information is used for the second device to generate first metrics, and the first metrics include AI unit inference performance monitoring metrics;

[0028] Alternatively, when the communication device is the second device, the communication interface is configured to receive the first information sent by the first device; the processor is configured to monitor the inference performance of the AI unit based on the first information and generate first metrics, and the first metrics include AI unit inference performance monitoring metrics. Wherein, the first information is generated based on perception measurement-related data and communication measurement-related data, and the first information includes a first data set, and the first data set includes monitoring samples for monitoring the inference performance of the AI unit;

[0029] Alternatively, when the communication device is a third device, the communication interface is used to send perception measurement related data or a first signal to the first device, and the first signal is used to obtain the perception measurement related data; wherein, the perception measurement related data is used for the first device to generate a first piece of information, the first piece of information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of the AI unit, and the first piece of information is used for the second device to generate a first metric, and the first metric includes an AI unit inference performance monitoring metric.

[0030] In a ninth aspect, a readable storage medium is provided, on which a program or instructions are stored, and when the program or instructions are executed by a processor, the steps of the method as described in the first aspect, or the steps of the method as described in the second aspect, or the steps of the method as described in the third aspect are implemented.

[0031] In a tenth aspect, a wireless communication system is provided, including: a first device, a second device, and a third device, where the first device can be used to execute the steps of the method as described in the first aspect, the second device can be used to execute the steps of the method as described in the second aspect, and the third device can be used to execute the steps of the method as described in the third aspect.

[0032] In an eleventh aspect, a chip is provided, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run a program or instructions to implement the steps of the method as described in the first aspect, the second aspect, or the third aspect.

[0033] In a twelfth aspect, a computer program / program product is provided, the computer program / program product is stored in a storage medium, and the program / program product is executed by at least one processor to implement the steps of the method as described in the first aspect, the second aspect, or the third aspect.

[0034] In the embodiments of the present application, the first device acquires perception measurement related data and communication measurement related data, generates a first piece of information based on the perception measurement related data and the communication measurement related data, and further sends the first piece of information to the second device. Wherein, the first piece of information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of the AI unit, and the first piece of information is used for the second device to generate an AI unit inference performance monitoring metric. Since the first piece of information is generated based on the perception measurement related data and the communication measurement related data, it further enables the second device to integrate the perception measurement related data in the monitoring of the AI unit inference performance. Compared with only using the communication measurement related data to monitor the AI unit inference performance, the solution provided in the present application can better implement the monitoring of the AI unit inference performance. Description of the Drawings

[0035] Figure 1a It is a block diagram of a wireless communication system to which the embodiments of the present application can be applied;

[0036] Figure 1b It is a schematic diagram of a sensing mode to which the embodiments of the present application can be applied;

[0037] Figure 2 It is one of the flowcharts of a method for monitoring the inference performance of an AI unit provided by the embodiments of the present application;

[0038] Figures 3a to 3r It is a scenario diagram of a method for monitoring the inference performance of an AI unit provided by the embodiments of the present application;

[0039] Figure 4 It is the second flowchart of a method for monitoring the inference performance of an AI unit provided by the embodiments of the present application;

[0040] Figure 5 It is the third flowchart of a method for monitoring the inference performance of an AI unit provided by the embodiments of the present application;

[0041] Figure 6 It is the first structural diagram of a device for monitoring the inference performance of an AI unit provided by the embodiments of the present application;

[0042] Figure 7 It is the second structural diagram of a device for monitoring the inference performance of an AI unit provided by the embodiments of the present application;

[0043] Figure 8 It is the third structural diagram of a device for monitoring the inference performance of an AI unit provided by the embodiments of the present application;

[0044] Figure 9 It is the structural diagram of a communication device provided by the embodiments of the present application;

[0045] Figure 10 It is the structural diagram of a terminal provided by the embodiments of the present application;

[0046] Figure 11 It is the structural diagram of a network-side device provided by the embodiments of the present application;

[0047] Figure 12 It is the structural diagram of another network-side device provided by the embodiments of the present application. Detailed implementation manners

[0048] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, rather than all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0049] The terms "first", "second", etc. in this application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances so that the embodiments of this application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same category, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "or" in this application means at least one of the connected objects. For example, "A or B" covers three scenarios, namely, Scenario 1: including A and not including B; Scenario 2: including B and not including A; Scenario 3: including both A and B. The character " / " generally indicates an "or" relationship between the associated objects before and after.

[0050] The term "indicate" in this application can be either a direct indication (or an explicit indication) or an indirect indication (or an implicit indication). Among them, a direct indication can be understood as that the sender clearly tells the receiver specific information, operations to be performed, or request results, etc. in the sent indication; an indirect indication can be understood as that the receiver determines the corresponding information according to the indication sent by the sender, or makes a judgment and determines the operations to be performed or request results, etc. according to the judgment result.

[0051] It should be noted that the technologies described in the embodiments of this application are not limited to Long Term Evolution (LTE) / LTE-Advanced (LTE-A) systems, and can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA), or other systems. The terms "system" and "network" in the embodiments of this application are often used interchangeably, and the described technologies can be used in the above-mentioned systems and radio technologies, as well as in other systems and radio technologies. The following description describes the New Radio (NR) system for example purposes, and the NR terms are used in most of the following descriptions, but these technologies can also be applied to systems other than the NR system, such as the 6th generation (6th Generation, 6G) communication system.

[0052] Figure 1aThe block diagram of a wireless communication system to which embodiments of the present application can be applied is shown. The wireless communication system includes a terminal 11 and a network-side device 12. Among them, the terminal 11 can be a mobile phone, a tablet personal computer, a laptop computer, a notebook computer, a personal digital assistant (PDA), a handheld computer, a netbook, an ultra-mobile personal computer (UMPC), a mobile internet device (MID), an augmented reality (AR), a virtual reality (VR) device, a robot, a wearable device, a flight vehicle, a vehicle user equipment (VUE), a shipborne device, a pedestrian user equipment (PUE), a smart home (home appliances with wireless communication functions, such as refrigerators, TVs, washing machines or furniture, etc.), a game console, a personal computer (PC), a teller machine or a self-service machine, etc. Wearable devices include: smart watches, smart bracelets, smart earphones, smart glasses, smart jewelry (smart bracelets, smart bracelets, smart rings, smart necklaces, smart anklets, smart ankle chains, etc.), smart wristbands, smart clothing, etc. Among them, the vehicle user equipment can also be called a vehicle terminal, a vehicle controller, a vehicle module, a vehicle component, a vehicle chip or a vehicle unit, etc. It should be noted that the specific type of the terminal 11 is not limited in the embodiments of the present application. The network-side device 12 can include an access network device or a core network device. Among them, the access network device can also be called a radio access network (RAN) device, a radio access network function or a radio access network unit. The access network device can include a base station, a wireless local area network (WLAN) access point (AP) or a wireless fidelity (WiFi) node, etc.Among them, the base station can be referred to as Node B (NB), Evolved Node B (eNB), the next generation Node B (gNB), New Radio Node B (NR Node B), access point, Relay Base Station (RBS), Serving Base Station (SBS), Base Transceiver Station (BTS), radio base station, radio transceiver, Basic Service Set (BSS), Extended Service Set (ESS), home Node B (HNB), home evolved Node B, Transmission Reception Point (TRP), or some other suitable term in the art. As long as the same technical effect is achieved, the base station is not limited to specific technical terms. It should be noted that in the embodiments of this application, only the base station in the NR system is taken as an example for introduction, and the specific type of the base station is not limited.

[0053] The core network device may include, but is not limited to, at least one of the following: core network node, core network function, Mobility Management Entity (MME), Access and Mobility Management Function (AMF), Session Management Function (SMF), User Plane Function (UPF), Policy Control Function (PCF), Policy and Charging Rules Function (PCRF), Edge Application Server Discovery Function (EASDF), Unified Data Management (UDM), Unified Data Repository (UDR), Home Subscriber Server (HSS), Centralized network configuration (CNC), Location Management Function (LMF), Network Repository Function (NRF), Network Exposure Function (NEF), Local NEF (L-NEF), Binding Support Function (BSF), Application Function (AF), etc. It should be noted that in the embodiments of this application, only the core network devices in the NR system are taken as examples for introduction, and the specific types of core network devices are not limited.

[0054] For better understanding, the following explains the related concepts and principles that may be involved in the embodiments of this application.

[0055] Communication sensing integration (also known as communication and sensing integration):

[0056] Future mobile communication systems such as B5G systems or 6G systems will not only have communication capabilities but also sensing capabilities. The sensing capabilities refer to one or more devices with sensing capabilities that can sense information such as the orientation, distance, and speed of a target object through the transmission and reception of wireless signals, or detect, track, identify, image, etc. a target object, event, or environment. In the future, with the deployment of small base stations with high-frequency band and large bandwidth capabilities such as millimeter waves and terahertz in the 6G network, the sensing resolution will be significantly improved compared to centimeter waves, enabling the 6G network to provide more refined sensing services. A typical sensing function and application scenario are shown in Table 1.

[0057] Table 1

[0058]

[0059] Communication and sensing integration (abbreviated as communication-sensing integration) means that in the same system, through spectrum sharing and hardware sharing, the integration design of communication and sensing functions is realized. While the system is transmitting information, it can sense information such as orientation, distance, and speed, detect, track, and identify a target device or event. The communication system and the sensing system complement each other, achieving an improvement in overall performance and bringing a better service experience.

[0060] The integration of communication and radar belongs to a typical application of communication and sensing integration (communication-sensing fusion). In the past, radar systems and communication systems were strictly separated due to different research objects and focuses, and the two systems were studied independently in most scenarios. In fact, both radar and communication systems are typical ways of information transmission, acquisition, processing, and exchange, and there are many similarities in terms of working principles, system architectures, and frequency bands. The design of communication and radar integration has great feasibility, mainly reflected in the following aspects: First, both the communication system and the sensing system are based on the electromagnetic wave theory, using the emission and reception of electromagnetic waves to complete information acquisition and transmission; Second, both the communication system and the sensing system have structures such as antennas, transmitters, receivers, and signal processors, with a large overlap in hardware resources; With the development of technology, there is also an increasing overlap in their working frequency bands; In addition, there are similarities in key technologies such as signal modulation, reception detection, and waveform design. The integration of communication and radar systems can bring many advantages, such as cost savings, size reduction, power consumption reduction, spectrum efficiency improvement, mutual interference reduction, etc., thus improving the overall performance of the system.

[0061] According to the different sensing signal sending nodes and receiving nodes, it is divided into 6 basic sensing modes, as Figure 1b shown, specifically including:

[0062] (1) Base station echo sensing. In this sensing mode, base station A sends a sensing signal and performs sensing measurements by receiving the echo of the sensing signal.

[0063] (2) Air interface sensing between base stations. At this time, base station B receives the sensing signal sent by base station A and performs sensing measurements.

[0064] (3) Uplink air interface sensing. At this time, base station A receives the sensing signal sent by terminal A and performs sensing measurements.

[0065] (4) Downlink air interface sensing. At this time, terminal B receives the sensing signal sent by base station B and performs sensing measurements.

[0066] (5) Terminal echo sensing. At this time, terminal A sends a sensing signal and performs sensing measurements by receiving the echo of the sensing signal.

[0067] (6) Sidelink sensing between terminals. At this time, terminal B receives the sensing signal sent by terminal A and performs sensing measurements.

[0068] It should be noted that Figure 1b each sensing mode in uses a sensing signal sending node and a sensing signal receiving node as examples. In an actual system, according to different sensing use cases and sensing requirements, one or more different sensing modes can be selected, and there can be one or more sending nodes and receiving nodes for each sensing mode. Figure 1b The sensing targets in use people and vehicles as examples, and it is assumed that neither people nor vehicles carry or install signal transceiver devices. The sensing targets in the actual scenario will be more diverse.

[0069] Currently, the sensing results are usually obtained in the following ways.

[0070] 1. Obtain sensing results with A sending and B receiving

[0071] By node A sending a sensing reference signal, node B receiving the sensing reference signal, and node B obtaining the sensing measurement / sensing result. Among them, node A can be base station 1, node B can be the target UE, or a UE near the target UE, or another base station 2.

[0072] 2. Obtain sensing results with self-sending and self-receiving

[0073] By node A sending a sensing reference signal, node A receiving the sensing reference signal, and node A obtaining the sensing measurement / sensing result. Among them, node A can be the target UE, or the serving base station of the target UE, or another base station 2.

[0074] 3. Grid nodes obtain sensing results through sensors

[0075] The network node obtains the sensed measurement / sensed result through its own deployment or sensor-like sensing devices in the environment. Sensor-like sensing: A sensing method that performs sensing services through means other than the communication sensing integrated system. Typical devices include: lidar, millimeter-wave radar, vision sensors (including: monocular vision, binocular vision, infrared sensors), inertial measurement unit (IMU), and various other sensors (rain gauges, thermometers, hygrometers, etc.).

[0076] With the development of AI technology, AI units have also been applied to communication systems. Currently, there are mainly the following use cases:

[0077] 1. AI-based beam prediction

[0078] In the AI-based beam prediction use case discussed in 5G, when monitoring the model, the key performance indicator (KPI) for monitoring is generally the prediction accuracy of the strongest beam or the prediction error of the beam quality. Among them, the prediction accuracy of the strongest beam is calculated based on the predicted strongest beam identifier and the identifier of the strongest beam corresponding to the true value; the prediction error of the beam quality is calculated based on the predicted beam quality and the beam quality corresponding to the true value.

[0079] 2. AI-based CSI prediction

[0080] In the AI-based CSI prediction use case discussed in 5G, when monitoring the model, the KPI for monitoring is generally the squared cosine similarity. Among them, the squared cosine similarity number is calculated based on the predicted channel matrix, precoding matrix indicator (PMI), channel eigenvector / eigenvalue, and the corresponding channel matrix, PMI, channel eigenvector / eigenvalue of the true value.

[0081] 3. AI-based positioning

[0082] In the AI-based positioning use case discussed in 5G, when monitoring the model, the KPIs for monitoring are generally the input data distribution deviation, output data distribution deviation, position estimation error, and TOA estimation error. Among them, the input data distribution deviation is calculated based on the reference input data distribution and the data distribution input to the actual model; the output data distribution deviation is calculated based on the reference output data distribution and the data distribution input to the actual model; the position estimation error is calculated based on the predicted distance relative to the base station and the distance relative to the base station corresponding to the true value; the TOA estimation error is calculated based on the predicted time of arrival and the time of arrival corresponding to the true value.

[0083] Among them, the model input includes the time-domain channel-based positioning reference signal measurement results;

[0084] The model output is the predicted distance relative to the base station or the time of arrival (TOA).

[0085] The ground truth / label is the actually measured distance relative to the base station or the TOA.

[0086] The AI unit described in this application may also be referred to as an AI unit, a machine learning (ML) model, an ML unit, an AI structure, an AI function, an AI feature, a machine learning model, a neural network, a neural network function, a neural network function, etc. Alternatively, the AI unit / AI unit may also refer to a processing unit capable of implementing specific algorithms, formulas, processing flows, capabilities, etc. related to AI. Alternatively, the AI unit / AI unit may be a processing method, algorithm, function, module, or unit for a specific data set. Alternatively, the AI unit / AI unit may be a processing method, algorithm, function, module, or unit running on AI / ML-related hardware such as a Graphics Processing Unit (GPU), a Neural network Processing Unit (NPU), a Tensor Processing Unit (TPU), or an Application Specific Integrated Circuit (ASIC). This application does not make specific limitations on this. Optionally, the specific data set includes the input and / or output of the AI unit / AI unit.

[0087] Optionally, the identifier of the AI unit / AI unit may be an AI unit identifier, an AI structure identifier, an AI algorithm identifier, or the identifier of a specific data set associated with the AI unit / AI unit, or the identifier of a specific scenario, environment, channel feature, or device related to AI / ML, or the identifier of a function, feature, capability, or module related to AI / ML. This application does not make specific limitations on this.

[0088] The signal configuration information involved in this application includes at least one of the following:

[0089] (a) Signal resource identity (ID), used to distinguish different signal resource configurations;

[0090] (b) Signal usage, indicating that the signal is for communication (such as channel measurement, channel estimation, synchronization, carrying data information, etc.), for sensing, or for both communication and sensing. Specifically, it can also be the signal for which sensing service or which type of sensing service;

[0091] (c) Waveform, such as Orthogonal Frequency Division Multiplexing (OFDM), Single-carrier Frequency-Division Multiple Access (SC-FDMA), Orthogonal Time Frequency Space (OTFS), Chirp, Frequency Modulated Continuous Wave (FMCW), pulse signal, etc.;

[0092] (d) Subcarrier spacing, for example, the subcarrier spacing of the OFDM system is 30KHz;

[0093] (e) Guard interval, which is the time interval between the end of signal transmission and the reception of the latest echo signal of the signal; this parameter is proportional to the maximum sensing distance; for example, it can be calculated by c / (2R max ), where R max is the maximum sensing distance (belonging to sensing requirement information). For example, for a self-transmitting and self-receiving sensing signal, R max represents the maximum distance from the sensing signal transceiver point to the signal emission point; in some cases, the cyclic prefix (CP) of the OFDM signal can act as the minimum guard interval; c is the speed of light;

[0094] (f) Starting frequency domain position, that is, the starting frequency point, which can also be the starting Resource Element (RE), Resource Block (RB) index;

[0095] (g) Starting time domain position, that is, the starting time point, which can also be the starting symbol index, time slot index, frame index;

[0096] (h) Ending frequency domain position, that is, the ending frequency point, which can be represented by the ending RE, RB index;

[0097] (i) Ending time domain position, that is, the ending time point, which can be represented by the ending RE, RB index;

[0098] (j) The length of the frequency-domain resource, i.e., the frequency-domain bandwidth, and the frequency-domain bandwidth is inversely proportional to the range resolution. The frequency-domain bandwidth B of each of the first signals satisfies B ≥ c / (2ΔR), where c is the speed of light and ΔR is the range resolution;

[0099] (k) The length of the time-domain resource, also known as the burst duration, and the time-domain resource length is inversely proportional to the Doppler resolution.

[0100] (l) The frequency-domain resource interval, representing the interval between adjacent signal frequency-domain resource units, which can be expressed by the number of resource elements (REs) or the number of resource blocks (RBs), or can be represented by the density value Density. For example, Density = 1 means that there is one RE in each RB for carrying signals. The frequency-domain resource interval is inversely proportional to the maximum unambiguous range / delay. For an OFDM system, when the subcarriers are continuously mapped, the frequency-domain interval is equal to the subcarrier interval;

[0101] (m) The time-domain resource interval, which is the time interval between two adjacent signal resource units, and the time-domain resource interval is associated with the maximum unambiguous Doppler shift or the maximum unambiguous speed;

[0102] (n) The time-domain resource characteristics, such as periodic transmission, semi-persistent transmission, and aperiodic transmission;

[0103] (o) The signal power, for example, taking values every 2 dBm from -20 dBm to 23 dBm;

[0104] (p) The sequence information, including the sequence type information (ZC sequence, PN sequence, etc.), the sequence generation method, the sequence length, etc.;

[0105] (q) The signal direction, that is, the angle information or beam information of the signal transmission;

[0106] (r) The Quasi co-location (QCL) relationship. For example, the sensing signal includes multiple resources, and each resource has a QCL with a Synchronization Signal and PBCH block (SSB). The QCL includes Type A, B, C, or D.

[0107] (s) The antenna port information, such as the maximum number of antenna ports and the antenna port index.

[0108] (t) The Cyclic prefix (CP) information, including the CP type (such as Normal Cyclic Prefix (NCP), Extended Cyclic Prefix (ECP), or a newly designed CP dedicated to sensing measurement, etc.), the CP length, etc.

[0109] The sensing requirements involved in this application include at least one of the following:

[0110] 1. Sensing service or sensing service type. The sensing service can be, for example, detecting the presence of a target, positioning, speed detection, distance detection, angle detection, acceleration detection, material analysis, component analysis, shape detection, category classification, radar cross section (RCS) detection, polarization scattering characteristic detection, fall detection, intrusion detection, quantity statistics, indoor positioning, gesture recognition, lip reading recognition, gait recognition, expression recognition, face recognition, breathing monitoring, heart rate monitoring, pulse monitoring, humidity / brightness / temperature / atmospheric pressure monitoring, air quality monitoring, weather condition monitoring, environment reconstruction, terrain and landform, building / vegetation distribution detection, pedestrian or vehicle flow detection, crowd density, vehicle density detection, etc.; The sensing service type can classify multiple different sensing services according to certain characteristics. For example, it can be classified into detection-type sensing services (such as intrusion detection, fall detection), parameter estimation-type sensing services (distance, angle, speed calculation), recognition-type sensing services (action recognition, identity recognition), etc. according to function, or it can be, for example, target detection and tracking-type sensing services (including the presence of a target, target ranging / ranging / angle measurement / localization / trajectory tracking), environment monitoring-type sensing services (including rainfall detection, flood monitoring), action detection-type sensing services (including gesture / action recognition, breathing / heartbeat detection, fall detection), etc. It can also be classified according to the sensing range (close-range sensing, medium-range sensing, long-range sensing), according to the sensing fineness (coarse-grained sensing, fine-grained sensing, etc.), according to power consumption / energy consumption, according to resource occupancy, etc.

[0111] 2. Sensing target area: It refers to the location area where the sensing object may exist, or the location area where imaging or environment reconstruction needs to be performed.

[0112] 3. Sensing object type: Classify the sensing objects according to the possible motion characteristics of the sensing objects. Each sensing object type contains information such as the motion speed, motion acceleration, and typical RCS of typical sensing objects.

[0113] 4. Sensing quality of service (QoS): Performance indicators for sensing the sensing target area or sensing objects, including at least one of the following:

[0114] a) Sensing resolution (which can be further divided into: ranging resolution, angle measurement resolution, speed measurement resolution, imaging resolution), etc.;

[0115] b) Sensing accuracy (which can be further divided into: ranging accuracy, angle measurement accuracy, speed measurement accuracy, positioning accuracy, etc.);

[0116] c) Perception range (which can be further divided into: ranging range, velocity measurement range, angle measurement range, imaging range, etc.);

[0117] d) Perception delay (the time interval from the sending of the perception signal to obtaining the perception result, or the time interval from the initiation of the perception requirement to obtaining the perception result);

[0118] e) Perception update rate (the time interval between two adjacent executions of perception and obtaining the perception result);

[0119] f) Detection probability (the probability of being correctly detected when the perception object exists);

[0120] g) False alarm probability (the probability of erroneously detecting the perception target when the perception object does not exist);

[0121] h) Maximum number of perceivable targets.

[0122] The types of perception measurement quantities related to integrated communication and sensing involved in this application include at least one of the following:

[0123] First-level measurement quantity (received signal / raw channel information), including: complex result of received signal / channel response, amplitude / phase, I-channel / Q-channel and their operation results (operations include addition, subtraction, multiplication, division, matrix addition, subtraction, multiplication, matrix transpose, trigonometric relation operations, square root operation, power operation, etc., and threshold detection results, maximum / minimum value extraction results, etc. of the above operation results; operations also include Fast Fourier Transform (FFT) / Inverse Fast Fourier Transform (IFFT), Discrete Fourier Transform (DFT) / Inverse Discrete Fourier Transform (IDFT), 2D-FFT, 3D-FFT, matched filtering, autocorrelation operation, wavelet transform, digital filtering, etc., and threshold detection results, maximum / minimum value extraction results, etc. of the above operation results).

[0124] Second-level measurement quantity (basic measurement quantity), including: delay, Doppler, angle, intensity, and their multi-dimensional combined representations. For example, it can be a delay value, Doppler value, or delay power spectrum, Doppler power spectrum, velocity power spectrum, angle power spectrum, delay-Doppler spectrum, delay-angle spectrum, Doppler-angle spectrum, delay-Doppler-angle spectrum, etc.

[0125] Third-level measurement quantity (perception result / perception intermediate result), including:

[0126] Parameters for characterizing basic attributes / status, including: distance, speed, orientation, spatial position, acceleration;

[0127] Parameters for characterizing advanced attributes / status, including: whether the target exists, trajectory, action, expression, vital signs, quantity, imaging result, weather, air quality, shape, material, composition.

[0128] Parameters for characterizing the result of environmental reconstruction, such as trajectory, etc.

[0129] Types of perception measurement quantities sensed by different types of sensors:

[0130] Perception measurement quantities related to lidar, including at least one of the following:

[0131] - Lidar point cloud data, each point in the lidar point cloud data includes: X / Y / Z position information, and, additional information;

[0132] - The angle and distance of the target obtained from the lidar point cloud data;

[0133] - Visual features of the target identified from the lidar point cloud data, such as: people, vehicles, etc.;

[0134] - The number of targets identified from the lidar point cloud data.

[0135] The additional information in the lidar point cloud data includes at least one of the following:

[0136] - Intensity: The echo intensity of the laser pulse generating the lidar point;

[0137] - Echo number: The echo number is the total number of echoes of a given pulse;

[0138] - Point classification: Each post-processed lidar point can have a classification defining the type of object reflecting the lidar pulse, and the lidar points can be divided into many categories, such as: ground, bare surface, top of the tree canopy, and water area, etc.;

[0139] - RGB: The RGB band can be used as an attribute of the lidar data, and this attribute usually comes from the image collected during the lidar measurement.

[0140] - GPS time: The GPS timestamp when the laser point is emitted from the aircraft.

[0141] - Scanning angle:

[0142] - Scanning direction: The traveling direction of the laser scanning mirror, where the value 1 represents the positive scanning direction and the value 0 represents the negative scanning direction.

[0143] Visual-related perception measurement quantities, including at least one of the following:

[0144] - Visual images;

[0145] - The luminosity of image pixels;

[0146] - The RGB values of image pixels;

[0147] - The visual features of the targets recognized from the images, such as: people, vehicles, etc.

[0148] - The angles and distances of the targets recognized from the images (especially for binocular vision);

[0149] - The number of targets recognized from the images.

[0150] Radar-related perception measurement quantities, including at least one of the following:

[0151] - Radar point clouds, each point in the point clouds includes: at least one of distance / speed / azimuth angle / pitch angle, or at least one of X / Y / Z / speed;

[0152] - The distances, speeds, and angles of the recognized targets;

[0153] - Radar imaging;

[0154] - The number of targets.

[0155] Inertial measurement unit-related perception measurement quantities, including at least one of the following:

[0156] - Acceleration: at least one of the three directions of X / Y / Z;

[0157] - Speed: at least one of the three directions of X / Y / Z;

[0158] - Angular velocity: at least one of the three axes of X / Y / Z.

[0159] Perception measurement quantities of a position sensor such as GNSS, including the position information of the device and the relative distance / angle information between the device and a specific object.

[0160] Other perception measurement quantities, including at least one of the following: whether a target exists, trajectory, action, expression, vital signs, quantity, imaging result, weather, air quality, shape, material, composition, etc.

[0161] Next, in combination with the accompanying drawings, through some embodiments and their application scenarios, the monitoring method for the inference performance of the AI unit provided by the embodiments of the present application will be described in detail.

[0162] Please refer to Figure 2 , Figure 2It is a flowchart of a method for monitoring the inference performance of an AI unit provided by an embodiment of the present application, and the method is executed by a first device. As Figure 2 shown, the method includes the following steps:

[0163] Step 201, the first device obtains perception measurement-related data and communication measurement-related data, and generates first information based on the perception measurement-related data and the communication measurement-related data.

[0164] Among them, the first information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of the AI unit, and the first information is used by the second device to generate AI unit inference performance monitoring metrics.

[0165] In an embodiment of the present application, the perception measurement-related data includes but is not limited to perception reference signals, perception measurement quantities, perception performance, perception results, etc. The communication measurement-related data includes but is not limited to channel state information (CSI), beam measurement information, positioning reference signal measurement information, phase measurement information, etc. Specifically, for example, the reference signal received power (RSRP) of a beam, the reference signal received quality (RSRQ) of a beam, the signal-to-noise and interference ratio (SINR) of a beam, the RSRP of a cell channel, the RSRQ of a cell channel, the SINR of a cell channel, the received signal strength indication (RSSI) of a cell channel, the impulse response of a cell channel, the precoding matrix indicator (PMI), the rank indicator (RI), the channel quality indicator (CQI), the beam identifier, and the subband identifier.

[0166] Exemplarily, taking the perception measurement related data including the perception measurement quantity and the communication measurement related data including CSI as an example, the first device acquires the perception measurement quantity and CSI, and generates first information based on the perception measurement quantity and CSI. Among them, the first information includes a first data set, that is, the first device can generate a first data set based on the perception measurement quantity and CSI. Among them, the first data set includes monitoring samples for monitoring the inference performance of the AI unit. For example, the first device can take the perception measurement quantity and CSI as the input of the AI unit, and obtain the output of the AI unit. The input and output of the AI unit can be used as monitoring samples for monitoring the inference performance of the AI unit (for example, using the input distribution of the AI unit or the output distribution of the AI unit as model monitoring parameter items), that is, the first data set includes the input and output of the AI unit. Of course, the first data set may also include other possible parameters or contents.

[0167] Optionally, the monitoring samples include at least one of the following:

[0168] The input of the AI unit;

[0169] The output of the AI unit;

[0170] True value (also called label), which is used to indicate the actual measured value of the parameter corresponding to the output of the AI unit. For example, if the output of the AI unit is the predicted CSI, then the true value can be the actually measured CSI.

[0171] Exemplarily, the monitoring samples in the first data set can only include the input of the AI unit (for example, using the input distribution of the AI unit as a model monitoring parameter item), or the monitoring samples can also include the output and true value of the AI unit (for example, monitoring the inference accuracy or error of the AI unit using the output and true value of the AI unit), or the monitoring samples can also include the output of the AI unit (for example, using the output distribution of the AI unit as a model monitoring parameter item). Of course, the content included in the monitoring samples can also be other possible situations, which will not be specifically elaborated here.

[0172] Optionally, the input of the AI unit includes the perception measurement related data, and the perception measurement related data includes the perception measurement quantity and at least one of the following:

[0173] Indicator of the perception measurement quantity, such as a parameter item used to indicate a predefined perception measurement quantity; the parameter item includes at least one of Doppler of the path, time delay, power, and angle;

[0174] Timestamp of the perception measurement quantity, such as the measurement time of the perception measurement quantity or the reception time of the perception measurement quantity;

[0175] Information about the sending device of the sensed measurement quantity, such as the identity (ID) of the sending device of the sensed measurement quantity, the location of the sending device, the orientation of the sending device, the motion information of the sending device, etc.;

[0176] Information about the receiving device of the sensed measurement quantity, such as the ID of the receiving device of the sensed measurement quantity, the location of the receiving device, the orientation of the receiving device, the motion information of the receiving device, etc.;

[0177] Coordinate information of the sensed measurement quantity, such as whether the result of the sensed measurement quantity is based on the local coordinate system or the global coordinate system, the description information of the local coordinate system, such as the rotation angles relative to the global coordinate system: α (bearing angle), β (dip angle), and γ (tilt angle), etc.;

[0178] Information for indicating the performance metrics of the sensed measurement quantity, such as the resolution of the sensed measurement quantity (which can be time-delay resolution, ranging resolution, angle measurement resolution, Doppler resolution, velocity measurement resolution, imaging resolution, etc., that is, the granularity of the reported measurement quantity value), signal-to-noise and interference ratio (SINR), sensed SINR (the ratio of the power of the path associated with the sensed target to the power of the noise and interference), etc.;

[0179] Information for indicating the source of the sensed measurement quantity, such as whether the sensed measurement quantity is from the sensing reference signal or from sensor sensing, etc.;

[0180] Information for indicating the type of the sensed measurement quantity;

[0181] Information for indicating the sensing mode of the sensed measurement quantity;

[0182] Configuration information of the first signal, such as the configuration information of the sensing reference signal;

[0183] Information about the sending device of the first signal, such as the ID of the sending device of the sensing reference signal, the location of the sending device, the orientation of the sending device, the motion information of the sending device, etc.;

[0184] Information about the receiving device of the first signal, such as the ID of the receiving device of the sensing reference signal, the location of the receiving device, the orientation of the receiving device, the motion information of the receiving device, etc.;

[0185] Among them, the first signal is a signal used for sensing. For example, the first signal is a dedicated signal used for sensing, such as a CSI reference signal (CSI-RS), a tracking reference signal (TRS), a sounding reference signal (SRS), or a synchronization signal, etc.

[0186] In an embodiment of the present application, when the monitoring samples of the first data set include the input of the AI unit, the input of the AI unit may include a sensing measurement quantity and at least one item related to the sensing measurement quantity described above. By using the sensing measurement quantity and the information related thereto as the input of the AI unit and as the monitoring samples of the inference performance of the AI unit, it is more helpful to improve the inference accuracy of the AI unit. At this time, compared with only using communication measurement-related data to monitor the inference performance of the AI unit, the solution provided by the present application can better monitor the inference performance of the AI unit.

[0187] Step 202: The first device sends the first information to the second device.

[0188] It can be understood that after the first device generates the first information based on the sensing measurement-related data and the communication measurement-related data, the first device sends the first information including the above-mentioned first data set to the second device. The first information is used for the second device to generate a first metric, and the first metric includes a monitoring metric for the inference performance of the AI unit. That is, the second device can monitor the inference performance of the AI unit according to the first information to calculate the monitoring metric, and the monitoring metric is used to characterize the inference performance of the AI unit, such as whether the inference performance of the AI unit is good or bad, etc.

[0189] In an embodiment of the present application, the first device obtains sensing measurement-related data and communication measurement-related data, generates the first information based on the sensing measurement-related data and the communication measurement-related data, and further sends the first information to the second device. Among them, the first information includes a first data set, and the first data set includes monitoring samples for monitoring the inference performance of the AI unit. The first information is used for the second device to generate a monitoring metric for the inference performance of the AI unit. Since the first information is generated based on the sensing measurement-related data and the communication measurement-related data, it further enables the second device to integrate the sensing measurement-related data in the monitoring of the inference performance of the AI unit. Compared with only using communication measurement-related data to monitor the inference performance of the AI unit, the solution provided by the present application can better monitor the inference performance of the AI unit.

[0190] Optionally, the first information further includes description information of the first data set, and the description information of the first data set includes at least one of the following:

[0191] The identifier of the AI unit;

[0192] The identifier of the monitoring sample;

[0193] The quantity of the monitoring samples;

[0194] The number of types of the monitoring samples;

[0195] A first indication for indicating the sensed measurement quantities known to the second device (which may also refer to the sensed measurement quantities already stored in the second device), where the sensed measurement quantities known to the second device may be the sensed measurement quantities that the first device has sent to the second device, or the sensed measurement quantities obtained by the second device from other devices or the sensed measurement quantities sensed based on sensors, etc.;

[0196] A second indication for indicating the valid time of the monitoring samples;

[0197] The description information of the sensed measurement quantities corresponding to each type of the monitoring samples;

[0198] The proportion of the valid sensed measurement quantities in each type of the monitoring samples;

[0199] The valid time of the sensed measurement quantity set;

[0200] A third indication for indicating the monitoring samples associated with the sensed measurement quantities carried in the first data set;

[0201] A fourth indication for indicating the quantity or proportion of the monitoring samples associated with the sensed measurement quantities carried in the first data set.

[0202] Optionally, the description information of the sensed measurement quantities includes at least one of the following:

[0203] The parameter items of the sensed measurement quantities;

[0204] The number of the parameter items of the sensed measurement quantities;

[0205] The processing level of the sensed measurement quantities;

[0206] Information for indicating the sending device of the sensed measurement quantities, such as the sending device ID, the sending device location, the sending device orientation, the sending device motion information, etc.;

[0207] Information for indicating the receiving device of the sensed measurement quantities, such as the receiving device ID, the receiving device location, the receiving device orientation, the receiving device motion information, etc.;

[0208] Source information of the sensed measurement quantity, for example, whether the sensed measurement quantity is from a sensed reference signal or sensor sensing;

[0209] Sensed measurement quantity link identification information, used to distinguish which sensing link or which sensing mode the sensed measurement result comes from, such as monostatic sensing mode or bistatic sensing mode, etc.;

[0210] Signal configuration identification, where the signal configuration identification is used to indicate the signal corresponding to the sensed measurement quantity;

[0211] Sensing service information, such as sensing service ID, sensing service type ID, etc.;

[0212] Data subscription identification;

[0213] Information indicating the use of the sensed measurement quantity, for example, for communication, sensing, communication and sensing, AI inference, AI unit training, etc.;

[0214] Information about the device corresponding to the sensed measurement quantity, such as device ID, device location, device orientation, device movement information, etc.;

[0215] Coordinate information of the sensed measurement quantity, used to illustrate whether the sensed measurement quantity is based on the result in the local coordinate system or the global coordinate system, and the description information in the local coordinate system, such as the rotation angles relative to the global coordinate system: α (bearing angle), β (dip angle), and γ (tilt angle), etc.;

[0216] Performance index information corresponding to the sensed measurement quantity, such as resolution (which can be time-delay resolution, ranging resolution, angle measurement resolution, Doppler resolution, speed measurement resolution, imaging resolution, etc., that is, the granularity of the reported measurement quantity value), SINR, sensing SINR (the ratio of the power of the path associated with the sensing target to the power of noise and interference), etc.

[0217] In the embodiment of the present application, the first information sent by the first device to the second device includes the first data set and also includes the description information of the first data set. Furthermore, the second device can better know the information content related to the monitored samples and the sensed measurement quantities associated with the monitored samples in the first data set based on the description information of the first data set, so that the second device can better perform AI unit inference performance monitoring based on these information contents, thereby avoiding the impact caused by the deterioration of the AI unit inference performance on the network.

[0218] Optionally, in the embodiment of the present application, the first indication includes at least one of the following:

[0219] The identification of the monitored sample;

[0220] The identification of the resources used for the communication measurement result;

[0221] Reporting identifier of communication measurement result

[0222] Identifier of resources used for perception measurement quantity

[0223] Reporting identifier of perception measurement quantity

[0224] Reference sample identifier of perception measurement quantity

[0225] Measurement timestamp indication of perception measurement quantity

[0226] Parameter item indication of perception measurement quantity

[0227] Perception measurement quantity identifier

[0228] Optionally, the second indication includes at least one of the following:

[0229] Third signal measurement timestamp carried by each monitoring sample in the first dataset

[0230] Third signal measurement timestamp carried by the first monitoring sample in the first dataset, and relative time with respect to the third signal measurement timestamp carried by other monitoring samples in the first dataset except the first monitoring sample

[0231] Third signal measurement timestamp carried by the first monitoring sample in the first dataset

[0232] Third signal measurement timestamp carried by the first monitoring sample in the first dataset, and relative time of the last monitoring sample with respect to the timestamp

[0233] Third signal measurement timestamp carried by the last monitoring sample in the first dataset

[0234] Third timestamp information of each type of monitoring sample in the first dataset

[0235] Third timestamp information of the first type of monitoring sample in the first dataset, and relative time of the remaining other types of monitoring samples with respect to the third timestamp of the first type of monitoring sample

[0236] Wherein, the third timestamp information of the first type of monitoring sample includes at least one of the following:

[0237] Third signal measurement timestamp of the first monitoring sample in the first type of monitoring sample

[0238] Identifier of the first monitoring sample in the first type of monitoring sample

[0239] Third signal measurement timestamp of the last monitoring sample in the first type of monitoring sample

[0240] The third signal measurement timestamp of the first monitoring sample in the first type of monitoring samples, and the relative time of the third signal measurement of the last monitoring sample with respect to the third signal measurement timestamp of the first monitoring sample;

[0241] Wherein, the third signal is a signal used for communication measurement, and the first type of monitoring samples is any type of monitoring samples in the first dataset.

[0242] It should be noted that, in different embodiments, the specific contents included in the above first indication and second indication may be different, which will be specifically described in subsequent embodiments and will not be elaborated here too much.

[0243] For the last case: the third signal measurement timestamp of the first monitoring sample in the first type of monitoring samples, and the relative time of the third signal measurement of the last monitoring sample with respect to the third signal measurement timestamp of the first monitoring sample; at this time, preferably, the first type of monitoring samples is the first type of monitoring samples that are the earliest in time.

[0244] Optionally, in step 201, the first device obtains perception measurement related data, which may include any one of the following:

[0245] The first device receives the perception measurement related data sent by the second device;

[0246] The first device receives the first signal sent by the second device or the third device, and obtains perception measurement related data according to the first signal;

[0247] The first device receives the perception measurement related data sent by the third device, where the perception measurement related data is obtained by the third device based on sensor device measurement or based on second signal measurement, or is perception measurement related data received from other perception devices;

[0248] Wherein, the first signal or the second signal is a signal used for perception, such as a perception reference signal.

[0249] Table 2

[0250]

[0251] Exemplarily, please refer to Table 2 above. In the embodiments of the present application, the first device may also be referred to as a data set collection device, that is, it collects or obtains the first data set. For example, the first device may be a target UE, a serving base station, etc.; the functions of the first device include at least one of the following: receiving a first configuration and sending a first message. The second device is an AI unit monitoring and control device. Optionally, it may also be referred to as a metric calculation device, that is, it calculates the AI unit inference performance monitoring metric. For example, the second device may be a serving base station, a core network element (such as AMF, LMF, etc.); the functions of the second device include at least one of the following: sending a first configuration, receiving a first message, calculating a monitoring metric, and sending a sensing measurement quantity. The third device may be a device that sends sensing measurement related data or a first signal, such as a serving base station, a neighboring base station, a UE near the target UE, etc. For better understanding, the following takes the case where the sensing measurement related data includes sensing measurement quantities, the first device is the target UE, the second device is the serving base station, and the third device is the neighboring base station or a UE near the target UE as an example to illustrate the above three cases.

[0252] For example, in one implementation, the target UE (the first device) may receive the sensing measurement quantity sent by the serving base station (the second device). The sensing measurement quantity may be obtained by the serving base station according to the sensing sensor device, or may also be obtained according to the sensing reference signal sent by other devices (such as other UEs). Then, the target UE generates the first message according to the sensing measurement quantity and the communication measurement related data, and sends the first message to the serving base station (the second device) so that the serving base station can perform AI unit inference performance monitoring according to the first message. At this time, the serving base station is the second device, or the second device and the third device are co-located.

[0253] Or, in another implementation, the target UE (the first device) receives the sensing reference signal sent by the serving base station (the second device) or the neighboring base station (the third device), obtains the sensing measurement quantity according to the sensing reference signal, and then generates the first message according to the sensing measurement quantity and the communication measurement related data, and sends the first message to the serving base station (the second device). At this time, the serving base station is the second device, or the second device and the third device are co-located.

[0254] Or, in yet another implementation, the target UE (the first device) may receive the sensing measurement quantity sent by the nearby UE (the second UE, which is the third device at this time). The sensing measurement quantity may be obtained by the second UE according to the sensing reference signal sent by the serving base station. Further, the target UE generates the first message according to the sensing measurement quantity and the communication measurement related data, and sends the first message to the serving base station (the second device).

[0255] It should be noted that the specific interaction processes between the above different devices will be described in detail in the subsequent embodiments, and will not be listed in detail here. In the embodiments of the present application, the first device can obtain perception measurement quantities through different methods, so that the solution provided by the embodiments of the present application can be applied to different scenarios based on the differences of the first device.

[0256] Optionally, before the first device receives the first signal sent by the second device or the third device, the method further includes:

[0257] The first device receives the first configuration sent by the second device or the fourth configuration sent by the third device, and the first configuration or the fourth configuration is used to indicate at least one of the following:

[0258] The identifier of the AI unit;

[0259] The reporting method of the first data set;

[0260] The number of monitoring samples;

[0261] The reporting method of the monitoring samples;

[0262] Whether to include the third indication;

[0263] Whether to include the fourth indication;

[0264] The data set identifier, and the data set corresponding to the data set identifier includes perception data;

[0265] The configuration identifier of the perception measurement quantity;

[0266] Information for indicating the number threshold of perception measurement quantities;

[0267] Information for indicating the source of perception measurement quantities;

[0268] Information for indicating the type of perception measurement quantities;

[0269] Information for indicating the use of perception measurement quantities;

[0270] Information for indicating the perception mode of perception measurement quantities;

[0271] Information for indicating the perception requirement;

[0272] The configuration information of the first signal;

[0273] The information of the sending device of the first signal;

[0274] The information of the receiving device of the first signal;

[0275] The information of the perception measurement link of the first signal.

[0276] In the embodiment of the present application, the first configuration is related to the monitoring samples and the sensed measurement quantities in the first data set. Thus, by sending the first configuration to the first device, the second device or the third device can assist the first device in better determining how to collect and report the first data set based on the first configuration.

[0277] Optionally, in the embodiment of the present application, the sensed measurement-related data includes sensed measurement quantities, and the method further includes at least one of the following:

[0278] The first device determines whether to generate the first information based on the sensed measurement quantity according to the first timestamp, the second timestamp, and the valid duration of the sensed measurement quantity;

[0279] The first device determines whether to generate the first information based on the sensed measurement quantity according to the fifth indication and the second timestamp;

[0280] The first device determines whether to generate the first information based on the sensed measurement quantity according to whether the sixth indication is received;

[0281] Wherein, the first timestamp is the timestamp related to the sensed measurement quantity;

[0282] The second timestamp is the measurement timestamp of the last communication measurement result corresponding to the true value in the communication measurement result;

[0283] The fifth indication is used to indicate the failure time or the failure time difference of the sensed measurement quantity;

[0284] The sixth indication is used to indicate the failure of the sensed measurement quantity.

[0285] Optionally, the first timestamp includes at least one of the following:

[0286] The measurement timestamp of the sensed measurement quantity;

[0287] The reception timestamp of the sensed measurement quantity.

[0288] In the embodiment of the present application, the first device generates the first information based on the sensed measurement-related data and the communication measurement-related data, including at least one of the following:

[0289] When the time corresponding to the second timestamp is earlier than the first time, the first device determines to generate the first information based on the sensed measurement quantity and the communication measurement-related data; the first time is the time corresponding to the first timestamp plus the valid duration of the sensed measurement quantity;

[0290] When the time corresponding to the second timestamp is earlier than the second time, the first device determines to generate the first information based on the sensed measurement quantity and the communication measurement related data; the second time is the expiration time of the sensed measurement quantity determined based on the fifth indication.

[0291] When the first device does not receive the sixth indication, the first device determines to generate the first information based on the sensed measurement quantity and the communication measurement related data.

[0292] Exemplarily, in one implementation, the first device determines whether to generate the first information based on the sensed measurement quantity according to the first timestamp, the second timestamp, and the effective duration of the sensed measurement quantity, that is, determines whether to combine the sensed measurement quantity and the communication measurement related data as a monitoring sample for AI unit inference, and the input of the AI unit; if the time corresponding to the second timestamp is earlier than the time corresponding to the first timestamp plus the effective duration of the sensed measurement quantity, that is, in the communication measurement result, the measurement timestamp of the last communication measurement result (such as CSI) corresponding to the true value of the monitoring sample is earlier than the time corresponding to the measurement timestamp of the sensed measurement quantity plus the effective duration of the sensed measurement quantity, then the first device may combine the sensed measurement quantity and the communication measurement related data as a monitoring sample for AI unit inference performance monitoring, that is, generate the first information based on the sensed measurement quantity and the communication measurement related data.

[0293] It should be noted that if the time corresponding to the second timestamp is not earlier than the time corresponding to the first timestamp plus the effective duration of the sensed measurement quantity, the sensed measurement quantity may not be used as a monitoring sample for AI unit inference performance monitoring.

[0294] Alternatively, in another implementation, the first device determines whether to generate the first information based on the sensed measurement quantity according to the expiration time or expiration time difference of the sensed measurement quantity, and the second timestamp, that is, determines whether to combine the sensed measurement quantity and the communication measurement related data as a monitoring sample for AI unit inference performance monitoring; if the second timestamp is earlier than the expiration time of the sensed measurement quantity determined according to the fifth indication, that is, in the communication measurement result, the measurement timestamp of the last communication measurement result (such as CSI) corresponding to the true value of the monitoring sample is earlier than the expiration time of the sensed measurement quantity determined based on the fifth indication, then the first device may combine the sensed measurement quantity and the communication measurement related data as a monitoring sample for AI unit inference performance monitoring, that is, generate the first information based on the sensed measurement quantity and the communication measurement related data.

[0295] It should be noted that if, in the communication measurement result, the measurement timestamp of the last communication measurement result corresponding to the true value of the monitored sample is not earlier than the expiration time of the sensed measurement quantity determined based on the fifth indication, the first device may not use the sensed measurement quantity as a monitored sample for monitoring the inference performance of the AI unit.

[0296] Alternatively, in another implementation, the first device determines whether to generate the first information based on the sensed measurement quantity, that is, determines whether to combine the sensed measurement quantity with the communication measurement related data as a monitored sample for monitoring the inference performance of the AI unit, based on whether it receives a sixth indication, that is, whether it receives an indication indicating the expiration of the sensed measurement quantity; if the first device does not receive the sixth indication, that is, does not receive an indication indicating the expiration of the sensed measurement quantity, the first device may combine the sensed measurement quantity with the communication measurement related data as a monitored sample for monitoring the inference performance of the AI unit, that is, generate the first information based on the sensed measurement quantity and the communication measurement related data. It should be noted that if the first device receives the sixth indication, that is, receives an indication indicating the expiration of the sensed measurement quantity, the first device may not use the sensed measurement quantity as a monitored sample for monitoring the inference performance of the AI unit.

[0297] In the embodiments of the present application, the first device can determine whether to combine the sensed measurement quantity with the communication measurement related data as a monitored sample for monitoring the inference performance of the AI unit based on the above different methods, making the generation method of the first information more flexible.

[0298] Optionally, in the embodiments of the present application, the first information may include a first part and a second part. The first part includes some or all of the description information of the first data set, and the second part includes some or all of the first data set, and the information length of the second part is determined according to the information length of the first part. For example, the first part may include the identifier of the AI unit, the first indication, the second indication, the description information of the sensed measurement quantity, the fourth indication, etc.; the second part may include the input of the AI unit, the output of the AI unit, the true value, etc. Of course, the specific content included in the first part and the second part may also be other possible situations, which will not be specifically elaborated here.

[0299] Optionally, the first information further includes a second information and a third information, where the second information is used to characterize the characteristics of the first data set, and the third information is used to characterize the acquisition description information of the sensed measurement quantity.

[0300] Optionally, the second information includes at least one of the following:

[0301] An indication of whether the monitored sample is associated with the sensed measurement quantity;

[0302] The number of monitoring samples of the correlation-aware measurement quantity;

[0303] The proportion of monitoring samples of the correlation-aware measurement quantity;

[0304] The number of monitoring samples of the correlation-effective awareness measurement quantity;

[0305] The proportion of monitoring samples of the correlation-effective awareness measurement quantity;

[0306] Whether the quantity of monitoring samples of the correlation-aware measurement quantity meets the minimum quantity threshold;

[0307] Whether the proportion of monitoring samples of the correlation-aware measurement quantity meets the minimum proportion threshold;

[0308] The proportion of the awareness measurement quantities that meet the target timeliness among the awareness measurement quantities associated with the monitoring samples;

[0309] The quantity of the awareness measurement quantities that meet the target timeliness among the awareness measurement quantities associated with the monitoring samples;

[0310] The effective time information of each awareness measurement quantity;

[0311] The first threshold, where the first threshold is the ratio of the number of effective awareness measurement quantities associated with a monitoring sample to the total number of awareness measurement quantities associated with the monitoring sample, and the first threshold can default to 1 or be configured as a value less than 1;

[0312] The second threshold, where the second threshold is the ratio of the actual number of awareness measurement quantities associated with a monitoring sample to the total number of the maximum supportable awareness measurement quantities associated with the monitoring sample, and the second threshold can default to 0 or be configured as a value greater than 0 and less than 1.

[0313] Optionally, the third information may include at least one of the following:

[0314] The parameter items of the awareness measurement quantities associated with the monitoring sample, and the parameter items may include Doppler, time delay, power, angle, etc. of the signal path (path);

[0315] The indication of the number of awareness measurement quantities associated with the monitoring sample;

[0316] The processing level of the awareness measurement quantities associated with the monitoring sample;

[0317] The source indication of the awareness measurement quantities associated with the monitoring sample;

[0318] The indication of the sending device of the awareness measurement quantities associated with the monitoring sample;

[0319] An indication of a receiving device for a sensed measurement quantity associated with the monitored sample;

[0320] An indication of the type of the sensed measurement quantity associated with the monitored sample.

[0321] The coordinates of the sensed measurement quantity associated with the monitored sample;

[0322] The sensing mode of the sensed measurement quantity associated with the monitored sample;

[0323] The sensing service of the sensed measurement quantity associated with the monitored sample;

[0324] The use of the sensed measurement quantity associated with the monitored sample;

[0325] The performance index of the sensed measurement quantity associated with the monitored sample;

[0326] An indication that the sensed measurement quantity associated with the monitored sample is a fixed sensed measurement quantity or a combination of sensed measurement quantities;

[0327] An indication that the sensed measurement quantity associated with the monitored sample is a variable sensed measurement quantity or a combination of sensed measurement quantities;

[0328] An indication that the number of the sensed measurement quantities associated with the monitored sample is variable;

[0329] Configuration information of the fourth signal;

[0330] Wherein, the fourth signal is a signal corresponding to the sensed measurement quantity associated with the monitored sample and used for sensing.

[0331] In an embodiment of the present application, the first information further includes the above-mentioned second information and third information. The second information is used to characterize the characteristics of the first data set, and the third information is used to characterize the acquisition description information of the sensed measurement quantity. Thus, through the second information and the third information, the first device and the second device can more accurately know the content related to the first data set and the sensed measurement quantity, which is more helpful for monitoring the inference performance of the AI unit.

[0332] Exemplarily, as Figure 3a shown, if the CSI measurement timestamp corresponding to the ground truth in step 5 is earlier than the sensing result expiration time in step 6, and the timestamp of obtaining the sensed measurement quantity in step 2 is earlier than the CSI reference signal measurement timestamp corresponding to the input of the AI unit in step 4, then the monitored sample or the sensed measurement quantity is marked as valid, and the first device generates the first information based on the sensed measurement quantity.

[0333] If the monitored sample is associated with N sensed measurement quantities, and M of them are valid, if M / N ≥ the first threshold, then the monitored sample is marked as the sensed measurement quantity being used effectively; the first threshold can be defaulted to 1 or configured as a value less than 1.

[0334] Alternatively, as Figure 3b shown, if the time when the sensed measurement quantity is obtained in step 2 is later than the earliest time when the AI unit inputs the corresponding CSI reference signal measurement in step 4, then the monitored sample or the sensed measurement quantity is marked as unused.

[0335] If the monitored sample is associated with N sensed measurement quantities, and L of them are valid, if L > 0, then the monitored sample is marked as having used the sensed measurement quantity; if L / N ≥ the second threshold, then the monitored sample is marked as having used the minimum number of sensed measurement quantities; the second threshold can be defaulted to 0 or configured as a value greater than 0 and less than 1.

[0336] Alternatively, as Figure 3c shown, if the sensing result invalidation time in step 6 is earlier than the last CSI measurement timestamp corresponding to the true value in step 5, then the monitored sample or the sensed measurement quantity is marked as invalid or unused.

[0337] It should be noted that the timeliness (valid time) of the sensed measurement quantity can be sent to the first device in the following ways:

[0338] Way 1. Initially configure the valid time of a sensed measurement quantity, and when sending the signal for sensing each time, do not carry the valid time additionally;

[0339] Way 2. Initially configure the valid time of a sensed measurement quantity, and when sending the signal for sensing each time, carry an adjustment value (such as -100 ms);

[0340] Way 3. When sending the signal for sensing each time, carry the absolute valid time of the sensed measurement quantity.

[0341] For better understanding, the AI unit inference performance monitoring method provided by the present application is explained below through specific embodiments.

[0342] Embodiment 1

[0343] As Figure 3d shown, the UE is the first device, and the serving base station is the second device, or it can also be the second device and the third device. The method includes the following steps:

[0344] Step 11a. The serving base station sends a first signal (such as a sensing reference signal) to the UE;

[0345] Step 12a. The UE receives the first signal sent by the serving base station, obtains the perception measurement related data (such as perception measurement quantities), combines the perception measurement related data and the communication measurement related data (such as CSI measurement results) to generate the first information including the first data set, and the first data set includes the monitoring samples for monitoring the inference performance of the AI unit. A monitoring sample may include at least one of the AI unit input, the AI unit output, and the true value (or label).

[0346] Step 13a. The UE sends the first information to the serving base station.

[0347] Step 14a. The serving base station calculates the AI unit inference performance monitoring metric (the above first metric) based on the first information. Further, the serving base station may make relevant decisions on the subsequent operations of the AI unit based on the metric, such as deactivating the AI unit, or switching to another AI unit, or triggering model training or fine-tuning to adjust the AI unit parameters, etc.

[0348] Optionally, before step 11a, the serving base station sends the first configuration to the UE, and the first configuration includes at least one of the following:

[0349] The identifier of the AI unit;

[0350] The reporting method of the first data set. For example, all the monitoring samples associated with a certain perception measurement quantity in the first data set are reported together, or the monitoring samples reported at one time are associated with the same or the same group of perception measurement quantities, or the monitoring samples are reported according to a fixed number of monitoring samples, and these monitoring samples may be associated with different perception measurement quantities;

[0351] The number of the monitoring samples;

[0352] The reporting method of the monitoring samples;

[0353] Whether to include the third indication;

[0354] Whether to include the fourth indication;

[0355] The first data set identifier for implicitly indicating the first data set. For example, it implicitly indicates the configuration or reporting configuration of the first signal used in the previous training process or the inference process of the first data set;

[0356] The configuration identifier of the perception measurement quantity for implicitly indicating the perception measurement quantity associated with the monitoring sample;

[0357] The information for indicating the number threshold of the perception measurement quantity;

[0358] Information for indicating the source of the sensed measurement quantity. For example, the measurement is derived from a sensed reference signal (i.e., integrated communication and sensing), or from sensor sensing, or from which type of sensor for sensor sensing;

[0359] Information for indicating the type of the sensed measurement quantity;

[0360] Information for indicating the sensing mode of the sensed measurement quantity, such as indicating which one of the aforementioned 6 modes;

[0361] Information for indicating the sensing requirement;

[0362] Configuration information of the first signal;

[0363] Information of the transmitting device of the first signal;

[0364] Information of the receiving device of the first signal.

[0365] Optionally, in this embodiment, in another alternative scenario, the UE may also receive the first signal sent by the second base station. At this time, the UE is the first device, the serving base station is the second device, and the second base station is the third device. As Figure 3e shown, the method includes the following processes:

[0366] Step 11b. The second base station sends a first signal (e.g., a sensed reference signal) to the UE;

[0367] Step 12b. The UE receives the first signal sent by the second base station, obtains sensed measurement-related data (e.g., sensed measurement quantity), and generates first information including a first data set by combining the sensed measurement-related data and communication measurement-related data (e.g., CSI measurement result). The first data set includes monitoring samples for monitoring the inference performance of the AI unit, and one monitoring sample may include at least one of the AI unit input, AI unit output, and true value (or label);

[0368] Step 13b. The UE sends the first information to the serving base station;

[0369] Step 14b. The serving base station calculates the AI unit inference performance monitoring metric (the aforementioned first metric) based on the first information.

[0370] In this scenario, the serving base station and the second base station can negotiate the first configuration, which may specifically include the following methods:

[0371] Method 1. The serving base station determines the first configuration based on the monitoring requirement and sends the first configuration to the second base station;

[0372] Method 2. The serving base station receives the third configuration sent by the second base station and determines the first configuration;

[0373] Method 3. The serving base station receives the second configuration sent by the second base station and sends feedback information to the second base station based on the monitoring requirements. The feedback information includes the configuration difference between the first configuration and the second configuration, or the desired first configuration.

[0374] In this embodiment, the monitoring samples in the first information (including the first data set) may or may not be associated with the sensed measurement quantity. The reporting method of the monitoring samples or the reporting method of the first data set may specifically include the following situations:

[0375] Situation 1 (please refer to Figure 3f ): The monitoring samples in the first data set do not carry the sensed measurement quantity

[0376] In this case, the first information includes the following content: AI unit identifier, the output of the AI unit (i.e., the prediction result of the AI unit, such as the predicted strongest beam identifier, the predicted future channel, the predicted TOA, etc.), the true value (such as the actually measured strongest beam identifier, the actually measured future channel at a certain moment, the actually measured TOA, etc.), the CSI measurement quantity corresponding to the input of the AI unit (such as the historical beam quality, the historical channel quality, etc.), the identifier of the monitoring sample, the number of monitoring samples, the first indication (used to indicate the sensed measurement quantity reported by the UE), and the second indication (used to indicate the valid time of the monitoring sample).

[0377] Among them, the first indication may include the following content to indicate the sensed measurement quantity sent by the UE:

[0378] The reporting identifier of the sensed measurement quantity, using the sensed measurement quantity in the previous sensed measurement quantity report as the sensed measurement quantity of the monitoring sample in the first information;

[0379] The identifier of the monitoring sample (sensed measurement quantity reference sample identifier), using the sensed measurement quantity associated with the previously reported monitoring sample identifier as the sensed measurement quantity of the monitoring sample in the first information;

[0380] The measurement timestamp indication of the sensed measurement quantity, indicating the sensed measurement quantity used by the monitoring sample in the first information through the previously reported sensed measurement quantity timestamp;

[0381] The parameter item indication of the sensed measurement quantity, using the sensed measurement quantity that matches the parameter item in the previously reported sensed measurement quantity as the sensed measurement quantity of the monitoring sample in the first information;

[0382] The sensed measurement quantity identifier, using the sensed measurement quantity corresponding to the identifier of the previously reported sensed measurement quantity as the sensed measurement quantity of the monitoring sample in the first information;

[0383] The identifier of the resource used for sensing the measurement quantity, and use the sensing measurement quantity corresponding to the identifier of the resource used for sensing the measurement quantity before, as the sensing measurement quantity of the monitoring sample in the first information;

[0384] The sensing measurement quantity associated with the communication measurement result corresponding to the resource used for the communication measurement result, as the sensing measurement quantity of the monitoring sample in the first information;

[0385] The identifier of the resource used for the communication measurement result, and use the sensing measurement quantity associated with the communication measurement result corresponding to the resource used for the communication measurement result before, as the sensing measurement quantity of the monitoring sample in the first information;

[0386] The reporting identifier of the communication measurement result, and use the sensing measurement quantity associated with the communication measurement result corresponding to the reporting of the communication measurement result before, as the sensing measurement quantity of the monitoring sample in the first information.

[0387] In this case 1, the second indication may include any one of the following:

[0388] 1. The third signal measurement timestamp carried by the first monitoring sample in the first dataset (that is, the first sample in the figure, and the samples described in the subsequent drawings are also monitoring samples, which will not be repeated hereinafter), and the remaining monitoring samples carry the relative time with respect to the third signal measurement timestamp. Wherein, the third signal measurement timestamp includes a first timestamp and a second timestamp, the first timestamp is the timestamp of the last CSI measurement corresponding to the true value, and the second timestamp is the earliest time of the CSI reference signal measurement corresponding to the input of the AI unit;

[0389] 2. The third signal measurement timestamp carried by the first monitoring sample in the first dataset, and the last sample carries the relative time with respect to the third signal measurement timestamp;

[0390] 3. The third signal measurement timestamp carried by the last sample in the first dataset;

[0391] Wherein, the third signal is a signal used for communication measurement, such as a CSI measurement signal or an RRM measurement signal.

[0392] As Figure 3f As shown, the first sample to the third sample use the reported sensing measurement quantity, and determine which sensing measurement quantity to associate with through the first indication. Or through the second indication and the valid time of the sensing measurement quantity, it is indicated whether the second device associates the first sample to the third sample with the reported sensing measurement quantity.

[0393] In this case 1, when reporting the first information, it can be divided into a first part and a second part. Among them, the first part includes: AI unit identifier, indication of whether the output of the AI unit is included, indication of the output format of the AI unit, indication of whether the true value is included, true value format indication, indication of whether the input of the AI unit is included, input format indication of the AI unit, indication of the number of monitoring samples, indication of the format of the sensed measurement quantity, the first indication, and the second indication.

[0394] The information length of the second part is determined by the information length of the first part. Specifically, the bit length occupied by the second part is determined by the indication of the number of monitoring samples, the indication of whether the output of the AI unit is included, the output format indication of the AI unit, the indication of whether the true value is included, the true value format indication, the indication of whether the input of the AI unit is included, and the input format indication of the AI unit. Among them, the second part includes the input of the AI unit in the monitoring samples, the output of the AI unit, the part related to communication measurement in the true value, and the time stamp indication of the monitoring samples. Case 2 (please refer to Figure 3g ): The monitoring samples in the first dataset carry a sensed measurement quantity, and all the monitoring samples in the first dataset are associated with this sensed measurement quantity

[0395] In this case, the first information includes the following: AI unit identifier, output of the AI unit (i.e., the prediction result of the AI unit, such as the predicted strongest beam identifier, the predicted future channel, the predicted TOA, etc.), true value (such as the actually measured strongest beam identifier, the actually measured channel at a future moment, the actually measured TOA, etc.), measurement time stamp corresponding to the true value, identifier of the monitoring sample, number of monitoring samples, description information of the sensed measurement quantity associated with the monitoring sample (such as parameter items of the sensed measurement quantity, processing level of the sensed measurement quantity, number of parameter items of the sensed measurement quantity, time stamp of the sensed measurement quantity, sending device of the sensed measurement quantity, receiving device of the sensed measurement quantity, indication of whether the sensed measurement quantity comes from sensor sensing or sensed signal measurement), measured value of the sensed measurement quantity, and a third indication (used to indicate which monitoring samples the sensed measurement quantity carried in the first dataset is associated with. For example, in this case, it is associated with all the monitoring samples in the first dataset).

[0396] As Figure 3g shown, the first sample to the third sample use the sensed measurement quantity carried in the first dataset, and the third indication is used to indicate that the second device associates the first sample to the third sample with the sensed measurement quantity carried in the first dataset. At this time, compared with case 1, the first indication and the second indication do not need to be included in the first information.

[0397] In this case 2, when the first information is reported, it can be divided into a first part and a second part. Among them, the first part includes: AI unit identifier, indication of whether the output of the AI unit is included, indication of the output format of the AI unit, indication of whether the true value is included, true value format indication, indication of whether the input of the AI unit is included, input format indication of the AI unit, indication of the number of monitoring samples, indication of the format of the sensed measurement quantity, description information of the sensed measurement quantity (including at least one of the following: parameter items of the sensed measurement quantity, indication of the number of sensed measurement quantities, processing level of the sensed measurement quantity, sending device of the sensed measurement quantity, receiving device of the sensed measurement quantity, indication of whether the sensed measurement quantity comes from sensor sensing or sensed signal measurement, indication of the source category of the sensed measurement quantity).

[0398] The information length of the second part is determined by the information length of the first part. Specifically, the bit length occupied by the second part is determined by the indication of the number of monitoring samples, the indication of whether the output of the AI unit is included, the output format indication of the AI unit, the indication of whether the true value is included, the true value format indication, the indication of whether the input of the AI unit is included, and the input format indication of the AI unit. Among them, the second part includes the input of the AI unit in the monitoring sample, the output of the AI unit, the part related to communication measurement in the true value, and the time stamp indication of the monitoring sample (optionally). And the measured value corresponding to the sensed measurement quantity in the monitoring sample can be placed in the first part or the second part.

[0399] Case 3 (please refer to Figure 3h ): The monitoring samples in the first dataset carry a sensed measurement quantity, and only some of the monitoring samples in the first dataset are associated with this sensed measurement quantity

[0400] In this case, the first information includes the following: AI unit identifier, output of the AI unit (i.e., the prediction result of the AI unit, such as the predicted strongest beam identifier, the predicted future channel, the predicted TOA, etc.), ground truth (such as the actually measured strongest beam identifier, the actually measured future channel at a certain moment, the actually measured TOA, etc.), the measurement timestamp corresponding to the ground truth, the identifier of the monitoring sample, the number of monitoring samples, the description information of the sensed measurement quantity associated with the monitoring sample (such as the parameter items of the sensed measurement quantity, the processing level of the sensed measurement quantity, the number of parameter items of the sensed measurement quantity, the timestamp of the sensed measurement quantity, the sending device of the sensed measurement quantity (optional), the receiving device of the sensed measurement quantity (optional), the indication of whether the sensed measurement quantity comes from sensor sensing or sensed signal measurement), the measured value of the sensed measurement quantity, the first indication (used to indicate the sensed measurement quantity that has been reported, and the first indication is optional), the second indication (used to indicate the valid time of the AI unit monitoring sample), the third indication (used to indicate which monitoring samples the sensed measurement quantity carried in the first dataset is associated with, for example, in this case, it is associated with the monitoring samples in the latter part of the first dataset), the number of types of monitoring samples, the fourth indication (used to indicate the number or proportion of monitoring samples associated with the sensed measurement quantity carried in the first dataset).

[0401] In this case 3, the second indication may include any one of the following:

[0402] Method 1. The third signal measurement timestamp carried by the last monitoring sample in the first type of monitoring samples in the first dataset, and the identifier of the first type of monitoring samples;

[0403] Method 2. The third signal measurement timestamp carried by the last monitoring sample of all types of monitoring samples in the first dataset;

[0404] Method 3. The identifier and the carried third signal measurement timestamp of the last monitoring sample in each type of monitoring samples in the first dataset;

[0405] Method 4. The third signal measurement timestamp carried by the last monitoring sample in the first type of monitoring samples in the first dataset, and the identifier of the last monitoring sample in the remaining other types of monitoring samples and the relative time with respect to the third signal measurement timestamp carried by the last monitoring sample in the first type of monitoring samples;

[0406] Method 5. The identifier and the corresponding third signal measurement timestamp of the first monitoring sample in each type of monitoring samples in the first dataset, and the identifier and the corresponding third signal measurement timestamp of the last monitoring sample in each type of monitoring samples;

[0407] Method 6. The third signal measurement timestamp carried by the first monitored sample in the first dataset, the identifier of the first monitored sample in each type of monitored sample and the relative time with respect to the third signal measurement timestamp carried by the first monitored sample, and the identifier of the last monitored sample in each type of monitored sample and the relative time with respect to the third signal measurement timestamp carried by the first monitored sample.

[0408] Wherein, the third signal is a signal used for communication measurement, and the first type of monitored sample is a monitored sample associated with the sensed measurement quantity that has been reported.

[0409] Optionally, in this Case 3, the content included in the second indication can preferably be the above Method 1. Through the second indication, the identifier of the last monitored sample in the first type of monitored sample can be known. Therefore, among all the monitored samples, the monitored samples before this identifier are all associated with the sensed measurement quantities reported previously.

[0410] As Figure 3h shown, the first sample uses the sensed measurement quantity that has been reported, and it can be indicated by the first indication which sensed measurement quantity it is associated with; or through the second indication and the valid time of the sensed measurement quantity, it can be indicated whether the second device associates the first sample to the third sample with the sensed measurement quantity that has been reported.

[0411] The second sample to the fourth sample are associated with the sensed measurement quantities reported in the first information.

[0412] Since this case includes two types of monitored samples, compared with Case 1, there are one more third indication and fourth indication.

[0413] In addition, through the third indication, it can be known that the later part of the monitored samples in the monitored samples is associated with the sensed measurement quantity carried in the first dataset. Then, through the quantity or proportion indicated by the fourth indication, it can be determined which of the later part of the monitored samples is associated with the sensed measurement quantity carried in the first dataset.

[0414] In this case 3, when reporting the first information, it can be divided into a first part and a second part. Among them, the first part includes: AI unit identifier, indication of whether it includes the output of the AI unit, indication of the output format of the AI unit, indication of whether it includes the true value, true value format indication, indication of whether it includes the input of the AI unit, input format indication of the AI unit, indication of the number of monitoring samples, indication of the perception measurement quantity format, the first indication (used to indicate the perception measurement quantity that has been reported), the format of the second indication (used to represent how to indicate the timestamp), description information of the perception measurement quantity of each type of monitoring sample (including at least one of the following: parameter items of the perception measurement quantity, indication of the number of perception measurement quantities, processing level of the perception measurement quantity, sending device of the perception measurement quantity, receiving device of the perception measurement quantity, indication of whether the perception measurement quantity comes from sensor perception or perception signal measurement, indication of the source category of the perception measurement quantity), indication of the number of types of monitoring samples, the fourth indication (used to indicate the number or proportion of monitoring samples associated with the perception measurement quantity carried in the first dataset).

[0415] The information length of the second part is determined by the information length of the first part. Specifically, it is determined by the indication of the number of monitoring samples, the indication of whether it includes the output of the AI unit, the output format indication of the AI unit, the indication of whether it includes the true value, the true value format indication, the indication of whether it includes the input of the AI unit, and the input format indication of the AI unit to determine the bit length occupied by the second part. Among them, each second part corresponds to a type of monitoring sample. The second part includes the input of the AI unit, the output of the AI unit, the part related to communication measurement in the true value, and the timestamp indication of the monitoring sample in the monitoring sample. And the measured value corresponding to the perception measurement quantity in the monitoring sample can be placed in the first part or the second part. Or, in the case of including multiple second parts, the contents of the multiple second parts can also be combined together.

[0416] Case 4 (please refer to Figure 3i ): The monitoring samples in the first dataset carry a perception measurement quantity, and only some of the monitoring samples (the previous monitoring samples) in the first dataset are associated with this perception measurement quantity

[0417] In this case, the first information includes the following: AI unit identifier, output of the AI unit (i.e., the prediction result of the AI unit, such as the predicted strongest beam identifier, the predicted future channel, the predicted TOA, etc.), ground truth (such as the actually measured strongest beam identifier, the actually measured channel at a future moment, the actually measured TOA, etc.), the measurement timestamp corresponding to the ground truth, the identifier of the monitoring sample, the number of monitoring samples, the description information of the sensed measurement quantity associated with the monitoring sample (such as the parameter items of the sensed measurement quantity, the processing level of the sensed measurement quantity, the number of parameter items of the sensed measurement quantity, the timestamp of the sensed measurement quantity, the sending device of the sensed measurement quantity (optional), the receiving device of the sensed measurement quantity (optional), the indication of whether the sensed measurement quantity comes from sensor sensing or sensed signal measurement), the measured value of the sensed measurement quantity, the second indication (used to indicate the valid time of the AI unit monitoring sample), the third indication (used to indicate which monitoring samples the sensed measurement quantity carried in the first dataset is associated with, for example, in this case, it is associated with the monitoring samples in the front part of the first dataset), the number of types of monitoring samples, the indication of the type of monitoring sample (such as indicating whether it is associated with the sensed measurement quantity), the fourth indication (used to indicate the number or proportion of monitoring samples associated with the sensed measurement quantity carried in the first dataset).

[0418] As Figure 3i shown, the first monitoring sample to the third monitoring sample are associated with the sensed measurement quantity carried in the first dataset. During the communication measurement - related data collection process corresponding to the fourth monitoring sample, the sensed measurement quantity has become invalid. Therefore, the fourth monitoring sample is not associated with the sensed measurement quantity carried in the first dataset, and since no new sensed measurement quantity has been obtained, the fourth monitoring sample is not associated with the sensed measurement quantity either.

[0419] In this case 4, the second indication may include any one of the following:

[0420] 1. The identifier of the first monitoring sample in the second - type monitoring samples in the first dataset and the corresponding third - signal measurement timestamp;

[0421] 2. The identifier of the last monitoring sample in the second - type monitoring samples in the first dataset and the corresponding third - signal measurement timestamp;

[0422] 3. The identifier of the last monitoring sample in the first - type monitoring samples in the first dataset and the corresponding third - signal measurement timestamp;

[0423] Among them, the third signal is the signal used for communication measurement. The first - type monitoring samples are the monitoring samples associated with the sensed measurement quantity carried in the first dataset (the first monitoring sample to the third monitoring sample), and the second - type monitoring samples are the monitoring samples not associated with the sensed measurement quantity (the fourth monitoring sample).

[0424] Optionally, in this Case 4, the content included in the second indication may preferably be the above-mentioned Method 1 or 2. Since this case requires determining whether the second type of monitoring sample is within the failure area of the sensed measurement carried in the first dataset, therefore, the first sample can be looked at first. If the first sample is within the failure area, then the subsequent samples are also within the failure area. Through the timestamp of the sensed measurement in the first information and the second indication, it can be known that if the last monitoring sample of the second type of monitoring sample is already within the failure area of the sensed measurement carried in the first dataset, it can be inferred that all samples in the second type of monitoring sample are within the failure area of this sensed measurement.

[0425] Since this case includes two types of monitoring samples, the first type of monitoring samples are the monitoring samples (the first sample to the third sample) associated with the sensed measurement carried in the first dataset, and the second type of monitoring samples are the monitoring samples (the fourth sample) not associated with the sensed measurement. Therefore, compared with Case 1, optionally, the third indication and the fourth indication can be additionally included. Through the third indication, it can be known that the previous part of the monitoring samples in the monitoring samples is associated with the sensed measurement carried in the first dataset, and then through the quantity or ratio indicated by the fourth indication, it can be determined which several monitoring samples are in the subsequent part.

[0426] Alternatively, the third indication and the fourth indication are not included, but instead, which of the previous part of the monitoring samples are associated with the sensed measurement in the first information is determined through the timestamp of the sensed measurement in the first information and the second indication.

[0427] In addition, since this case does not need to be associated with the sensed measurement known to the second device. Therefore, the first indication may not be required.

[0428] In this Case 4, when the first information is reported, it can be divided into a first part and a second part. Among them, the first part includes: AI unit identifier, indication of whether it includes the output of the AI unit, indication of the output format of the AI unit, indication of whether it includes the true value, true value format indication, indication of whether it includes the input of the AI unit, input format indication of the AI unit, indication of the number of monitoring samples, indication of the sensed measurement format, format of the second indication (used to indicate how to indicate the timestamp), description information of the sensed measurement of each type of monitoring sample (including at least one of the following: parameter items of the sensed measurement, quantity indication of the sensed measurement, processing level of the sensed measurement, sending device of the sensed measurement, receiving device of the sensed measurement, indication of whether the sensed measurement comes from sensor sensing or sensed signal measurement, source category indication of the sensed measurement), indication of the number of types of monitoring samples, second indication, third indication, fourth indication (used to indicate the number or ratio of the monitoring samples associated with the sensed measurement carried in the first dataset).

[0429] The information length of the second part is determined by the information length of the first part. Specifically, it is determined by the indication of the number of monitoring samples, the indication of whether to include the output of the AI unit, the indication of the output format of the AI unit, the indication of whether to include the true value, the true value format indication, the indication of whether to include the input of the AI unit, and the input format indication of the AI unit to determine the bit length occupied by the second part. Among them, each second part corresponds to a monitoring sample type, and the second part includes the input of the AI unit in the monitoring sample, the output of the AI unit, the part related to communication measurement in the true value, and the time stamp indication of the monitoring sample. The measurement value corresponding to the sensing measurement quantity in the monitoring sample can be placed in the first part or the second part. Alternatively, in the case of including multiple second parts, the contents of the multiple second parts can also be combined together.

[0430] Case 5 (please refer to Figure 3j ): The monitoring samples in the first dataset carry multiple sensing measurement quantities, and only some of the monitoring samples in the first dataset are associated with this sensing measurement quantity

[0431] In this case, the first information includes the following: AI unit identifier, the output of the AI unit (i.e., the prediction result of the AI unit, such as the predicted strongest beam identifier, the predicted future channel, the predicted TOA, etc.), the true value (such as the actually measured strongest beam identifier, the actually measured future channel at a certain moment, the actually measured TOA, etc.), the measurement timestamp corresponding to the true value, the identifier of the monitoring sample, the number of monitoring samples, the number of monitoring sample types, the description information of the sensing measurement quantity associated with each type of monitoring sample (such as the parameter items of the sensing measurement quantity, the processing level of the sensing measurement quantity, the number of parameter items of the sensing measurement quantity, the timestamp of the sensing measurement quantity set, the sending device of the sensing measurement quantity (optional), the receiving device of the sensing measurement quantity (optional), the indication of whether the sensing measurement quantity comes from sensor sensing or sensing signal measurement), the measurement value of the sensing measurement quantity associated with each type of monitoring sample, the first indication (used to indicate the reported sensing measurement quantity), the second indication (used to indicate the valid time of the AI unit monitoring sample), the third indication (used to indicate which monitoring samples in the first dataset are associated with the carried sensing measurement quantity, for example, in this case, it is associated with the subsequent monitoring samples in the first dataset), the fourth indication (used to indicate the number or proportion of monitoring samples associated with the carried sensing measurement quantity in the first dataset), and the proportion of valid sensing measurement quantities associated with each type of monitoring sample.

[0432] As Figure 3j shown, the first sample is associated with the reported sensing measurement quantity, so it can be determined which sensing measurement quantity is associated through the first indication; the second sample to the third sample are associated with the first sensing measurement quantity carried in the first dataset, and the fourth sample is associated with the second sensing measurement quantity carried in the first dataset.

[0433] In this case, the first information can be used to indicate which monitoring samples in the first dataset belong to which type. For example, in one implementation, the first information may include the following:

[0434] The number of types of monitoring samples (which can be equal to or greater than the number of sets of sensed measurement quantities);

[0435] The number of sets of sensed measurement quantities, where each set of sensed measurement quantities represents a plurality of sensed measurement quantities sharing the timestamp and valid time of the sensing measurement. For example, based on one sensing reference signal, the power, delay, and Doppler of the signal path are obtained simultaneously, and these sensed measurement quantities share the timestamp and valid time of the sensing measurement;

[0436] The timestamp of the set of sensed measurement quantities;

[0437] The first indication, used to indicate the sensed measurement quantities that have been reported;

[0438] The second indication, used to indicate the valid time of the monitoring samples of the AI unit. At this time, the second indication needs to include the third signal measurement timestamp of the first monitoring sample in the first type of monitoring samples, the identifier of the first monitoring sample in each type of monitoring samples and the relative time with respect to the third signal measurement timestamp of the first monitoring sample in the first type of monitoring samples, and the identifier of the last monitoring sample in each type of monitoring samples and the relative time with respect to the third signal measurement timestamp of the first monitoring sample in the first type of monitoring samples. The first type of samples is the monitoring samples associated with the sensed measurement quantities that have been reported.

[0439] Alternatively, in another implementation, the first information may include the following:

[0440] The number of types of monitoring samples (which can be equal to or greater than the number of sets of sensed measurement quantities); the third indication (used to indicate which monitoring samples the sensed measurement quantities carried in the first dataset are associated with. For example, in this case, they are associated with the monitoring samples in the latter part of the first dataset)

[0441] The fourth indication, used to indicate the number or proportion of the monitoring samples associated with the sensed measurement quantities carried in the first dataset.

[0442] Since this case includes three types of monitoring samples, optionally, compared with Case 1, it further includes a third indication and a fourth indication. Through the third indication, it can be known that the latter part of the monitoring samples in the monitoring samples is associated with the sensed measurement quantities carried in the first dataset. Then, by the number or proportion of each type of sensed measurement quantity (set) indicated by the fourth indication, it can be determined which of the latter part of the monitoring samples are respectively associated with which sensed measurement quantities in the first set. Note that here the fourth indication can respectively indicate the number of samples corresponding to multiple sensed measurement quantities.

[0443] For example Figure 3j As shown, a method of the fourth indication is 0.5, 0.25. In this way, it can be known that:

[0444] For the first sensed measurement quantity carried in the first dataset, the corresponding total sample starts from the ((0.5 + 0.25) × 4 (total number of samples)) = 3rd sample counted from the back, that is, from the 2nd sample counted from the front;

[0445] For the second sensed measurement quantity carried in the first dataset, the corresponding total sample starts from the (0.25 × 4 (total number of samples)) = 1st sample counted from the back, that is, from the 4th sample counted from the front;

[0446] Then it can be obtained that:

[0447] The first sensed measurement quantity carried in the first dataset is associated with the second sample to the third sample;

[0448] The second sensed measurement quantity carried in the first dataset is associated with the fourth sample;

[0449] This is an example of the fourth indication, and there can be other indication methods.

[0450] Alternatively, without including the third indication and the fourth indication, it is determined which of the latter part of the monitoring samples are associated with which sensed measurement quantities in the first information by the timestamps of the sensed measurement quantities in the first information and the second indication.

[0451] In addition, since this case needs to be associated with the sensed measurement quantities known to the second device. Therefore, it further includes a first indication.

[0452] In this case 5, when the first information is reported, it can be divided into a first part and multiple second parts. Among them, the first part includes: AI unit identifier, indication of whether it includes the output of the AI unit, indication of the output format of the AI unit, indication of whether it includes the true value, true value format indication, indication of whether it includes the input of the AI unit, input format indication of the AI unit, indication of the number of monitoring samples, indication of the format of the perceived measurement quantity, format of the second indication (used to indicate how to indicate the timestamp), description information of the perceived measurement quantity of each type of monitoring sample (including at least one of the following: indication used to indicate the reported perceived measurement quantity, parameter items of the perceived measurement quantity, indication of the number of perceived measurement quantities, processing level of the perceived measurement quantity, sending device of the perceived measurement quantity, receiving device of the perceived measurement quantity, indication of whether the perceived measurement quantity comes from sensor perception or perceived signal measurement, indication of the source category of the perceived measurement quantity), indication of the number of types of monitoring samples, first indication, third indication, fourth indication (used to indicate the number or proportion of monitoring samples associated with the perceived measurement quantity carried in the first dataset).

[0453] The information length of the second part is determined by the information length of the first part. Specifically, the bit length occupied by the second part is determined by the indication of the number of monitoring samples, indication of whether it includes the output of the AI unit, output format indication of the AI unit, indication of whether it includes the true value, true value format indication, indication of whether it includes the input of the AI unit, and input format indication of the AI unit. Each second part corresponds to a type of monitoring sample. The second part includes the input of the AI unit, output of the AI unit, and the part related to communication measurement in the true value in the monitoring sample, as well as the timestamp indication of the monitoring sample. The measured value corresponding to the perceived measurement quantity in the monitoring sample can be placed in the first part or the second part. Or, the contents of multiple second parts can also be combined together.

[0454] Embodiment 2

[0455] As Figure 3k shown, the UE is the first device, the serving base station is the second device, or the second device and the third device are co-located. The method includes the following steps:

[0456] Step 21. The serving base station sends the sensor-based perceived measurement quantity to the UE;

[0457] Step 22. The UE combines the perceived measurement quantity and the CSI measurement result to generate the first information including the first dataset. The first dataset includes monitoring samples for monitoring the inference performance of the AI unit. A monitoring sample can include at least one of the AI unit input, AI unit output, and true value (or label);

[0458] Step 23. The UE sends the first information to the serving base station;

[0459] Step 24. The serving base station calculates the AI unit inference performance monitoring metrics based on the first information. Further, the serving base station makes decisions on the subsequent operations of the AI unit based on the metrics.

[0460] Optionally, before step 21, the UE may receive a first configuration sent by the base station, and the first configuration may indicate the following: the identifier of the AI unit, the number of monitoring samples, the reporting method of the monitoring samples, whether the third indication is included, and whether the fourth indication is included.

[0461] It should be noted that, in this embodiment, the monitoring sample - associated perception measurement quantities in the first dataset may include the following situations.

[0462] Situation 6: The monitoring samples in the first dataset do not carry perception measurement quantities, and the monitoring samples in the first dataset are associated with one perception measurement quantity stored on the base station side (similar to Figure 3f ).

[0463] The main difference from Situation 1 is that the content indicated by the first indication is the measurement quantity already stored by the base station (for example, from base - station sensors, or perception measurement quantities obtained by the base station through self - transmission and self - reception, or perception measurement quantities obtained from other devices). At this time, after the UE obtains the perception measurement quantity from the base station, it does not need to report it to the base station (because the base station already knows these perception measurement quantities), thereby saving the transmission overhead of the first information.

[0464] Compared with the case where the base station sends the first signal, the UE must report the perception measurement quantity to the base station so that the base station can know the corresponding perception measurement quantity.

[0465] In this case, the first information may include the following: AI unit identifier, the output of the AI unit (i.e., the prediction result of the AI unit, such as the predicted strongest beam identifier, the predicted future channel, the predicted TOA, etc.), the true value (such as the actually measured strongest beam identifier, the actually measured future channel at a certain moment, the actually measured TOA, etc.), the CSI measurement quantity corresponding to the input of the AI unit (such as beam quality, historical channel quality, etc.), the identifier of the monitoring sample, the number of monitoring samples, the first indication (used to indicate the perception measurement quantity already stored on the base station side, which does not need to be reported by the UE), and the second indication (used to indicate the valid time of the AI unit monitoring sample).

[0466] Among them, the first indication may include the following content to indicate the perception measurement quantity already stored on the base station side:

[0467] The reference sample identifier of the perception measurement quantity, such as the one indicated by the base station to the UE during previous AI unit performance inference;

[0468] Parameter item indication of the sensed measurement quantity, for example, when performing AI unit performance inference previously, the sensed measurement quantity indicated by the base station to the UE;

[0469] Case 7: The monitoring samples in the first dataset do not carry the sensed measurement quantity, and the monitoring samples in the first dataset are associated with multiple sets of sensed measurement quantities stored on the base station side (similar to Figure 3f )

[0470] The main difference from Case 1 is that the content indicated by the first indication is the measurement quantity already stored by the base station (for example, the sensed measurement quantity from the base station sensor, or the sensed measurement quantity obtained by the base station through self-transmission and self-reception, or the sensed measurement quantity obtained from other devices). At this time, after the UE obtains the sensed measurement quantity from the base station, it does not need to report it to the base station again (because the base station already knows these sensed measurement quantities), thereby saving the transmission overhead of the first information.

[0471] Compared with the case where the base station sends the first signal, the UE must report the sensed measurement quantity to the base station before the base station can know the corresponding sensed measurement quantity.

[0472] Moreover, compared with Case 6, the first indication needs to indicate the associated sensed measurement quantities for multiple types of monitoring samples in the first dataset respectively.

[0473] In this case, the first information may include the following: AI unit identifier, output of the AI unit (i.e., the prediction result of the AI unit, such as the predicted strongest beam identifier, predicted future channel, predicted TOA, etc.), ground truth (such as the actually measured strongest beam identifier, future channel, actually measured TOA, etc.), CSI measurement quantity corresponding to the input of the AI unit (such as beam quality, historical channel quality, etc.), identifier of the monitoring sample, number of monitoring samples, second indication (used to indicate the valid time of the AI unit monitoring sample), type of the monitoring sample, first indication for each type of monitoring sample (used to indicate the sensed measurement quantity already stored on the base station side, without the need for the UE to report), number of each type of monitoring sample, identifier of the first sample of each type of monitoring sample, identifier of the last sample of each type of monitoring sample, third signal measurement timestamp corresponding to the first sample of each type of monitoring sample, and third signal measurement timestamp corresponding to the last sample of each type of monitoring sample. Among them, the third signal is the signal used for communication measurement.

[0474] Among them, when the first information is reported, it can be divided into a first part and multiple second parts. The first part includes the following content: AI unit identifier, indication of whether it includes the output of the AI unit, indication of the output format of the AI unit, indication of whether it includes the true value, true value format indication, indication of whether it includes the input of the AI unit, indication of the input format of the AI unit (which also includes the format indication of the sensed measurement quantity and can use the description information of the sensed measurement quantity), indication of the number of monitoring samples, indication of the format of the sensed measurement quantity, format of the second indication (used to indicate how to indicate the timestamp), description information of the sensed measurement quantity of each type of monitoring sample (including indication of the sensed measurement quantity that has been reported, parameter items of the sensed measurement quantity), indication of the number of types of monitoring samples,).

[0475] The information length of the second part is determined by the information length of the first part. Specifically, the bit length occupied by the second part is determined by the indication of the number of monitoring samples, indication of whether it includes the output of the AI unit, indication of the output format of the AI unit, indication of whether it includes the true value, true value format indication, indication of whether it includes the input of the AI unit, and indication of the input format of the AI unit. Among them, each second part corresponds to a type of monitoring sample, and the second part includes the input of the AI unit in the monitoring sample, the output of the AI unit, the part related to communication measurement in the true value, and the timestamp indication of the monitoring sample. Alternatively, the content of multiple second parts can also be combined together.

[0476] Embodiment 3

[0477] As Figure 3l shown, the UE is the first device, the first base station is the second device (or co-located with the third device), and the second base station is the third device. The method includes the following steps:

[0478] Step 31a. The first base station (serving base station) sends a sensing reference signal to the second base station;

[0479] Step 32a. The second base station obtains a sensed measurement quantity or a sensing result based on the sensing reference signal;

[0480] Step 33a. The second base station sends the fourth information to the first base station. The fourth information includes the sensed measurement quantity (name and measurement value), the measurement timestamp and the valid time of the sensed measurement quantity;

[0481] Step 34a. The first base station sends the fifth information to the UE. The fifth information includes the sensed measurement quantity (name and measurement value), the measurement timestamp and the valid time of the sensed measurement quantity, and the second base station indication;

[0482] Step 35a. The UE generates first information including a first data set based on the fifth information and in combination with CSI measurement results. The first data set includes monitoring samples for monitoring the inference performance of the AI unit. A monitoring sample may include at least one of the AI unit input, the AI unit output, and the ground truth (or label).

[0483] Step 36a. The UE sends the first information to the first base station.

[0484] Step 37a. The first base station calculates the AI unit inference performance monitoring metrics based on the first information. Further, the serving base station may make decisions on subsequent operations of the AI unit based on the metrics.

[0485] Wherein, the fifth information may further include the following: an indication for indicating the perception measurement quantity, an indication for indicating the source of the perception measurement quantity, a perception signal sending device indication, a perception signal receiving device indication, a perception measurement quantity processing level indication, a perception measurement quantity configuration density indication, an indication of the total number or the minimum number of the perception measurement quantities of the AI unit input.

[0486] As Figure 3m shown, the UE is the first device, the first base station is the second device (or co-located with the third device), and the second base station is the third device. The method includes the following steps:

[0487] Step 31b. The first base station (serving base station) sends a perception reference signal to the second base station.

[0488] Step 32b. The second base station obtains perception measurement quantities or perception results based on the perception reference signal.

[0489] Step 33b. The second base station sends the fifth information to the UE. The fifth information includes the perception measurement quantity (name and measurement value), the measurement timestamp and the valid time of the perception measurement quantity, and the second base station indication.

[0490] Step 34b. The UE generates first information including a first data set based on the fifth information and in combination with CSI measurement results. The first data set includes monitoring samples for monitoring the inference performance of the AI unit. A monitoring sample may include at least one of the AI unit input, the AI unit output, and the ground truth (or label).

[0491] Step 35b. The UE sends the first information to the first base station.

[0492] Step 36b. The first base station calculates the AI unit inference performance monitoring metrics based on the first information. Further, the serving base station may make decisions on subsequent operations of the AI unit based on the metrics.

[0493] Before step 31b, the first base station sends information to the second base station to indicate that the fifth information is to be sent to the target UE.

[0494] Embodiment 4

[0495] As Figure 3n shown, the target UE is the first device, the first base station is the second device (or co-located with the third device), and the second UE is the third device. The method includes the following steps:

[0496] Step 41a. The first base station (serving base station) sends a sensing reference signal to the second UE;

[0497] Step 42a. The second UE obtains a sensing measurement or a sensing result based on the sensing reference signal;

[0498] Step 43a. The second UE sends the fourth information to the first base station. The fourth information includes the sensing measurement (name and measurement value), the measurement timestamp and the valid time of the sensing measurement;

[0499] Step 44a. The first base station sends the fifth information to the target UE. The fifth information includes the sensing measurement (name and measurement value), the measurement timestamp and the valid time of the sensing measurement, and the second base station indication;

[0500] Step 45a. The target UE generates the first information including the first data set based on the fifth information and in combination with the CSI measurement result. The first data set includes monitoring samples for monitoring the inference performance of the AI unit. A monitoring sample can be at least one of the AI unit input, the AI unit output, and the ground truth (or label);

[0501] Step 46a. The target UE sends the first information to the first base station;

[0502] Step 47a. The first base station calculates the AI unit inference performance monitoring metric based on the first information. Further, the serving base station can make decisions on subsequent operations of the AI unit based on the metric.

[0503] Alternatively, as Figure 3o shown, the target UE is the first device, the first base station is the second device (or co-located with the third device), the second base station is the third device (sending the sensing reference signal), and the second UE is the third device (sending the sensing measurement). The serving cell of the second UE is the second base station. At this time, the second UE cannot directly send the fourth information to the first base station and needs to be forwarded by the second base station. The method includes the following steps:

[0504] Step 41b. The second base station sends a sensing reference signal to the second UE;

[0505] Step 42b. The second UE obtains a sensing measurement quantity or a sensing result based on the sensing reference signal;

[0506] Step 43b. The second UE sends fourth information to the second base station, where the fourth information includes the sensing measurement quantity (name and measurement value), the measurement timestamp of the sensing measurement quantity, and the valid time;

[0507] Step 44b. The second base station sends fifth information to the target UE through the first base station, where the fifth information includes the sensing measurement quantity (name and measurement value), the measurement timestamp of the sensing measurement quantity, the valid time, and the second base station indication;

[0508] Step 45b. The target UE generates first information including a first data set based on the fifth information and in combination with the CSI measurement result. The first data set includes monitoring samples for monitoring the inference performance of the AI unit. A monitoring sample may include at least one of the AI unit input, the AI unit output, and the ground truth (or label);

[0509] Step 46b. The target UE sends the first information to the first base station;

[0510] Step 47b. The first base station calculates the AI unit inference performance monitoring metrics based on the first information. Further, the serving base station may make a decision on the subsequent operations of the AI unit based on the metrics.

[0511] Optionally, before step 41b, step 40b may further be included. The first base station and the second base station negotiate a first configuration, and the specific content of the first configuration may refer to the description in the previous method embodiments.

[0512] Or, as Figure 3p shown, the target UE is the first device, the first base station is the second device (or co-located with the third device), the second base station is the third device (sending the sensing reference signal), and the second UE is the third device (sending the sensing measurement quantity). The serving of the second UE is the first base station. At this time, the second UE may directly send the fourth information to the first base station. Although the second UE is not the serving UE of the second base station, it can receive the sensing reference signal from the second base station. The method includes the following steps:

[0513] Step 41c. The second base station sends a sensing reference signal to the second UE;

[0514] Step 42c. The second UE obtains a sensing measurement quantity or a sensing result based on the sensing reference signal;

[0515] Step 43c. The second UE sends fourth information to the first base station, where the fourth information includes the sensing measurement quantity (name and measurement value), the measurement timestamp of the sensing measurement quantity, and the valid time;

[0516] Step 44c. The first base station sends the fifth information to the target UE, where the fifth information includes the sensing measurement quantity (name and measurement value), the measurement timestamp and the valid time of the sensing measurement quantity, and the second base station indication;

[0517] Step 45c. The target UE generates the first information including the first data set according to the fifth information and in combination with the CSI measurement result. The first data set includes the monitoring samples for monitoring the inference performance of the AI unit. A monitoring sample may include at least one of the AI unit input, the AI unit output, and the true value (or label);

[0518] Step 46c. The target UE sends the first information to the first base station;

[0519] Step 47c. The first base station calculates the AI unit inference performance monitoring metrics based on the first information. Further, the serving base station may make decisions on the subsequent operations of the AI unit based on the metrics.

[0520] Optionally, before step 41c, step 40c may further be included. The first base station and the second base station negotiate the first configuration, and the specific content of the first configuration may refer to the description in the previous method embodiments.

[0521] Embodiment 5

[0522] As Figure 3q shown, the target UE is the first device, the first base station is the second device (or co-located with the third device), the second base station is the third device (sending the sensing reference signal), and the second UE is the third device (sending the sensing measurement quantity). The method includes the following steps:

[0523] Step 51. The first base station sends the sensing reference signal to the second UE;

[0524] Step 52. The second UE obtains the sensing measurement quantity or the sensing result based on the sensing reference signal;

[0525] Step 53. When the second UE and the target UE have established sidelink communication, the second UE sends the fifth information to the target UE, where the fifth information includes the sensing measurement quantity (name and measurement value), the measurement timestamp and the valid time of the sensing measurement quantity, and the second base station indication;

[0526] Step 54. The target UE generates the first information including the first data set according to the fifth information and in combination with the CSI measurement result. The first data set includes the monitoring samples for monitoring the inference performance of the AI unit. A monitoring sample may include at least one of the AI unit input, the AI unit output, and the true value (or label);

[0527] Step 55. The target UE sends the first information to the first base station;

[0528] Step 56. The first base station calculates the AI unit inference performance monitoring metrics based on the first information. Further, the first base station can make decisions on the subsequent operations of the AI unit based on the metrics.

[0529] Embodiment Six

[0530] As Figure 3r shown, the target UE is the first device, and the first base station is the second device (or co-located with the third device). The method includes the following steps:

[0531] Step 61. The first base station sends a sensing reference signal and receives the sensing reference signal (for the self-transmitting and self-receiving mode);

[0532] Step 62. The first base station sends the fifth information to the target UE. The fifth information includes sensing measurement quantities (name and measurement value), the measurement timestamp and the valid time of the sensing measurement quantities, and a third device indication (optional. The device that may perform self-transmitting and self-receiving regarding the indication may be the serving base station, or a neighboring base station, or a UE near the target UE);

[0533] Step 63. The target UE generates the first information including the first data set according to the fifth information and in combination with the CSI measurement result. The first data set includes monitoring samples for monitoring the AI unit inference performance. A monitoring sample may include at least one of the AI unit input, the AI unit output, and the true value (or label);

[0534] Step 64. The target UE sends the first information to the serving base station;

[0535] Step 65. The serving base station calculates the AI unit inference performance monitoring metrics based on the first information. Further, the serving base station can make decisions on the subsequent operations of the AI unit based on the metrics.

[0536] Please refer to Figure 4 , Figure 4 is the second flowchart of a method for monitoring the AI unit inference performance provided by an embodiment of the present application. The method is applied to the second device. As Figure 4 shown, the method includes the following steps:

[0537] Step 401. The second device receives the first information sent by the first device;

[0538] Step 402. The second device monitors the AI unit inference performance based on the first information and generates a first metric. The first metric includes the AI unit inference performance monitoring metric;

[0539] Among them, the first information is generated based on perception measurement-related data and communication measurement-related data. The first information includes a first data set, and the first data set includes monitoring samples for monitoring the inference performance of the AI unit.

[0540] Optionally, the monitoring samples include at least one of the following:

[0541] The input of the AI unit;

[0542] The output of the AI unit;

[0543] The true value, which is used to indicate the actual measurement value of the parameter corresponding to the output of the AI unit.

[0544] Optionally, the input of the AI unit includes the perception measurement-related data, and the perception measurement-related data includes perception measurement quantities and at least one of the following:

[0545] An indication of the perception measurement quantity;

[0546] The timestamp of the perception measurement quantity;

[0547] Information about the sending device of the perception measurement quantity;

[0548] Information about the receiving device of the perception measurement quantity;

[0549] Coordinate information of the perception measurement quantity;

[0550] Information indicating the performance index of the perception measurement quantity;

[0551] Information indicating the source of the perception measurement quantity;

[0552] Information indicating the type of the perception measurement quantity;

[0553] Information indicating the perception mode of the perception measurement quantity;

[0554] Configuration information of the first signal;

[0555] Information about the sending device of the first signal;

[0556] Information about the receiving device of the first signal;

[0557] Among them, the first signal is a signal used for perception.

[0558] Optionally, the first information further includes description information of the first data set, and the description information of the first data set includes at least one of the following:

[0559] The identifier of the AI unit;

[0560] The identifier of the monitoring sample;

[0561] The quantity of the monitored samples;

[0562] The quantity of types of the monitored samples;

[0563] A first indication for indicating the quantity of sensed measurement known to the second device;

[0564] A second indication for indicating the valid time of the monitored samples;

[0565] The description information of the sensed measurement corresponding to each type of the monitored samples;

[0566] The proportion of valid sensed measurements in each type of the monitored samples;

[0567] The valid time of the set of sensed measurements;

[0568] A third indication for indicating the monitored samples associated with the sensed measurements carried in the first dataset;

[0569] A fourth indication for indicating the quantity or proportion of the monitored samples associated with the sensed measurements carried in the first dataset.

[0570] Optionally, the description information of the sensed measurement includes at least one of the following:

[0571] The parameter items of the sensed measurement;

[0572] The quantity of the parameter items of the sensed measurement;

[0573] The processing level of the sensed measurement;

[0574] The information for indicating the sending device of the sensed measurement;

[0575] The information for indicating the receiving device of the sensed measurement;

[0576] The source information of the sensed measurement;

[0577] The link identification information of the sensed measurement;

[0578] A sensed signal configuration identifier for indicating the sensed signal corresponding to the sensed measurement;

[0579] The sensed service information;

[0580] The data subscription identifier;

[0581] The information for indicating the use of the sensed measurement;

[0582] The information of the device corresponding to the sensed measurement;

[0583] Coordinate information of the sensed measurement quantity;

[0584] Performance index information corresponding to the sensed measurement quantity.

[0585] Optionally, the first indication includes at least one of the following:

[0586] The identifier of the monitored sample;

[0587] The identifier of the resource used for communicating the measurement result;

[0588] The reporting identifier of the communication measurement result;

[0589] The identifier of the resource used for the sensed measurement quantity;

[0590] The reporting identifier of the sensed measurement quantity;

[0591] The reference sample identifier of the sensed measurement quantity;

[0592] The measurement timestamp indication of the sensed measurement quantity;

[0593] The parameter item indication of the sensed measurement quantity;

[0594] The sensed measurement quantity identifier.

[0595] Optionally, the second indication includes at least one of the following:

[0596] The third signal measurement timestamp carried by each monitored sample in the first dataset;

[0597] The third signal measurement timestamp carried by the first monitored sample in the first dataset, and the relative time of the other monitored samples in the first dataset except the first monitored sample relative to the third signal measurement timestamp;

[0598] The third signal measurement timestamp carried by the first monitored sample in the first dataset;

[0599] The third signal measurement timestamp carried by the first monitored sample in the first dataset, and the relative time of the last monitored sample relative to the timestamp;

[0600] The third signal measurement timestamp carried by the last monitored sample in the first dataset;

[0601] The third timestamp information of each type of monitored sample in the first dataset;

[0602] The third timestamp information of the first type of monitored sample in the first dataset, and the relative time of the remaining other types of monitored samples relative to the third timestamp of the first type of monitored sample;

[0603] Among them, the third timestamp information of the first type of monitoring samples includes at least one of the following:

[0604] The third signal measurement timestamp of the first monitoring sample in the first type of monitoring samples;

[0605] The identifier of the first monitoring sample in the first type of monitoring samples;

[0606] The third signal measurement timestamp of the last monitoring sample in the first type of monitoring samples;

[0607] The third signal measurement timestamp of the first monitoring sample in the first type of monitoring samples, and the relative time of the third signal measurement of the last monitoring sample relative to the third signal measurement timestamp of the first monitoring sample;

[0608] Among them, the third signal is a signal used for communication measurement, and the first type of monitoring samples is any type of monitoring samples in the first dataset.

[0609] Optionally, before the second device receives the first information sent by the first device, the method further includes any one of the following:

[0610] The second device sends the perception measurement related data to the first device;

[0611] The second device sends a first signal to the first device, the first signal is used to obtain the perception measurement related data, and the first signal is a signal used for perception.

[0612] Optionally, before the second device sends the first signal to the first device, the method further includes:

[0613] The second device sends a first configuration to the first device, and the first configuration is used to indicate at least one of the following:

[0614] The identifier of the AI unit;

[0615] The reporting method of the first dataset;

[0616] The number of monitoring samples;

[0617] The reporting method of the monitoring samples;

[0618] Whether to include the third indication;

[0619] Whether to include the fourth indication;

[0620] The first dataset identifier, and the dataset corresponding to the first dataset identifier includes perception data;

[0621] The configuration identifier of the perception measurement quantity;

[0622] Information for indicating the number threshold of sensed measurement quantities;

[0623] Information for indicating the source of sensed measurement quantities;

[0624] Information for indicating the types of sensed measurement quantities;

[0625] Information for indicating the uses of sensed measurement quantities;

[0626] Information for indicating the sensing modes of sensed measurement quantities;

[0627] Information for indicating sensing requirements;

[0628] Configuration information of the first signal;

[0629] Information about the sending device of the first signal;

[0630] Information about the receiving device of the first signal;

[0631] Information about the sensing measurement link of the first signal.

[0632] Optionally, the method further includes any one of the following:

[0633] The second device determines the first configuration based on monitoring requirements and sends the first configuration to the third device;

[0634] The second device receives a third configuration sent by the third device and determines the first configuration;

[0635] The second device receives a second configuration sent by the third device and sends feedback information to the third device, where the feedback information is used to indicate the difference between the first configuration and the second configuration, or is used to indicate the desired first configuration.

[0636] Exemplarily, the first device is the target UE, the second device is the serving base station, and the third device is the neighboring base station. When the target UE receives the sensing reference signal sent by the neighboring base station and obtains the sensing measurement quantity according to the sensing reference signal, the first configuration can also be interacted or negotiated between the serving base station and the neighboring base station. For example, the serving base station determines the first configuration according to the monitoring requirements, sends the first configuration to the neighboring base station, and can further receive the feedback of the neighboring base station on the first configuration, such as agreement or the negotiated configuration; or, the third device determines the first configuration by itself and then sends the first configuration to the serving base station; or, the serving base station receives the second configuration sent by the neighboring base station. The serving base station can determine the first configuration according to the second configuration and send feedback information to the neighboring base station based on the monitoring requirements. The feedback information is used to indicate the difference between the second configuration and the first configuration, or to indicate the first configuration expected by the serving base station for negotiation with the neighboring base station to obtain the desired first configuration. In the embodiments of the present application, the determination method of the first configuration is made more flexible.

[0637] Optionally, the first information further includes: second information and third information, where the second information is used to characterize the characteristics of the first data set, and the third information is used to characterize the acquisition description information of the sensing measurement quantity.

[0638] Optionally, the second information includes at least one of the following:

[0639] An indication of whether the monitoring sample is associated with the sensing measurement quantity;

[0640] The number of monitoring samples associated with the sensing measurement quantity;

[0641] The proportion of monitoring samples associated with the sensing measurement quantity;

[0642] The number of monitoring samples associated with the valid sensing measurement quantity;

[0643] The proportion of monitoring samples associated with the valid sensing measurement quantity;

[0644] Whether the number of monitoring samples associated with the sensing measurement quantity meets the minimum quantity threshold;

[0645] Whether the proportion of monitoring samples associated with the sensing measurement quantity meets the minimum proportion threshold;

[0646] The proportion of the sensing measurement quantity that meets the target timeliness among the sensing measurement quantities associated with the monitoring sample;

[0647] The number of the sensing measurement quantity that meets the target timeliness among the sensing measurement quantities associated with the monitoring sample;

[0648] The effective time information of each sensing measurement quantity;

[0649] The first threshold, where the first threshold is the ratio of the number of valid perception measurement quantities associated with a monitoring sample to the total number of perception measurement quantities associated with the monitoring sample;

[0650] The second threshold, where the second threshold is the ratio of the number of perception measurement quantities actually associated with a monitoring sample to the total number of the maximum supportable perception measurement quantities associated with the monitoring sample.

[0651] Optionally, the third information includes at least one of the following:

[0652] The parameter items of the perception measurement quantities associated with the monitoring sample;

[0653] The indication of the number of perception measurement quantities associated with the monitoring sample;

[0654] The processing level of the perception measurement quantities associated with the monitoring sample;

[0655] The source indication of the perception measurement quantities associated with the monitoring sample;

[0656] The indication of the sending device of the perception measurement quantities associated with the monitoring sample;

[0657] The indication of the receiving device of the perception measurement quantities associated with the monitoring sample;

[0658] The indication of the type of the perception measurement quantities associated with the monitoring sample.

[0659] The coordinates of the perception measurement quantities associated with the monitoring sample;

[0660] The perception mode of the perception measurement quantities associated with the monitoring sample;

[0661] The perception service of the perception measurement quantities associated with the monitoring sample;

[0662] The use of the perception measurement quantities associated with the monitoring sample;

[0663] The performance indicators of the perception measurement quantities associated with the monitoring sample;

[0664] The indication that the perception measurement quantities associated with the monitoring sample are fixed perception measurement quantities or a combination of perception measurement quantities;

[0665] The indication that the perception measurement quantities associated with the monitoring sample are variable perception measurement quantities or a combination of perception measurement quantities;

[0666] The indication that the number of perception measurement quantities associated with the monitoring sample is variable;

[0667] The configuration information of the fourth signal;

[0668] Wherein, the fourth signal is the signal used for perception corresponding to the perception measurement quantity associated with the monitoring sample.

[0669] It should be noted that the specific implementation process and related concepts involved in the embodiments of the present application can be referred to the descriptions in the foregoing method embodiments, and will not be elaborated in this embodiment.

[0670] In the embodiments of the present application, the first information received by the second device includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of the AI unit, and the first information is used for the second device to generate an AI unit inference performance monitoring index. Since the first information is generated based on perception measurement-related data and communication measurement-related data, it further enables the second device to integrate perception measurement-related data in the monitoring of the AI unit inference performance. Compared with only using communication measurement-related data to monitor the AI unit inference performance, the solution provided by the present application can better implement the monitoring of the AI unit inference performance.

[0671] Please refer to Figure 5 , Figure 5 is a flowchart of a method for monitoring the inference performance of an AI unit provided by an embodiment of the present application, and the method is applied to a third device. As Figure 5 shown, the method includes the following steps:

[0672] Step 501, the third device sends perception measurement-related data or a first signal to the first device, and the first signal is used to obtain the perception measurement-related data;

[0673] Wherein, the perception measurement-related data is used for the first device to generate first information, the first information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of the AI unit, the first information is used for the second device to generate a first index, and the first index includes an AI unit inference performance monitoring index.

[0674] Optionally, the method further includes:

[0675] The third device sends a fourth configuration to the first device, and the fourth configuration is used to indicate at least one of the following:

[0676] The identifier of the AI unit;

[0677] The reporting method of the first data set;

[0678] The number of monitoring samples;

[0679] The reporting method of the monitoring samples;

[0680] Whether it includes a third indication for indicating a monitoring sample associated with the sensed measurement quantity carried in the first data set;

[0681] Whether it includes a fourth indication for indicating the number or proportion of monitoring samples associated with the sensed measurement quantity carried in the first data set;

[0682] The first data set identifier, and the data set corresponding to the first data set identifier includes sensed data;

[0683] The configuration identifier of the sensed measurement quantity;

[0684] Information for indicating the number threshold of the sensed measurement quantity;

[0685] Information for indicating the source of the sensed measurement quantity;

[0686] Information for indicating the type of the sensed measurement quantity;

[0687] Information for indicating the use of the sensed measurement quantity;

[0688] Information for indicating the sensing mode of the sensed measurement quantity;

[0689] Information for indicating the sensing requirement;

[0690] The configuration information of the first signal;

[0691] Information about the sending device of the first signal;

[0692] Information about the receiving device of the first signal;

[0693] Information about the sensing measurement link of the first signal. Optionally, the method further includes any one of the following:

[0694] The third device determines the third configuration and sends the third configuration to the second device, and the third configuration may be the same as or different from the above fourth configuration;

[0695] The third device receives the first configuration sent by the second device;

[0696] The third device sends a second configuration to the second device and receives feedback information sent by the second device, where the feedback information is used to indicate the difference between the first configuration and the second configuration, or to indicate the desired first configuration.

[0697] It should be noted that the specific implementation processes and related concepts involved in the embodiments of the present application may refer to the descriptions in the foregoing method embodiments, and will not be elaborated in this embodiment.

[0698] In an embodiment of the present application, a third device sends perception measurement-related data or a first signal to a first device. The perception measurement-related data is used for the first device to generate first information, the first information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of an AI unit, and the first information is used for a second device to generate an AI unit inference performance monitoring metric. Since the first information is generated based on the perception measurement-related data and the communication measurement-related data, it enables the second device to incorporate the perception measurement-related data in the monitoring of the AI unit inference performance. Compared with monitoring the AI unit inference performance only based on the communication measurement-related data, the solution provided in the present application can better achieve the monitoring of the AI unit inference performance.

[0699] For the method for monitoring the inference performance of an AI unit provided in an embodiment of the present application, the execution subject may be a monitoring device for the inference performance of an AI unit. In an embodiment of the present application, taking the monitoring device for the inference performance of an AI unit executing the method for monitoring the inference performance of an AI unit as an example, the monitoring device for the inference performance of an AI unit provided in an embodiment of the present application is described.

[0700] Please refer to Figure 6 , Figure 6 which is one of the structural diagrams of a monitoring device for the inference performance of an AI unit provided in an embodiment of the present application. The device is applied to a first device. As Figure 6 shown, the monitoring device 600 for the inference performance of an AI unit includes:

[0701] An acquisition module 601, configured to acquire perception measurement-related data and communication measurement-related data, and generate first information based on the perception measurement-related data and the communication measurement-related data;

[0702] A first sending module 602, configured to send the first information to a second device;

[0703] Wherein, the first information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of an AI unit, the first information is used for the second device to generate a first metric, and the first metric includes an AI unit inference performance monitoring metric.

[0704] Optionally, the monitoring samples include at least one of the following:

[0705] The input of the AI unit;

[0706] The output of the AI unit;

[0707] True value, where the true value is used to indicate the actual measured value of a parameter corresponding to the output of the AI unit.

[0708] Optionally, the input of the AI unit includes the perception measurement related data, and the perception measurement related data includes the perception measurement quantity and at least one of the following:

[0709] An indication of the perception measurement quantity;

[0710] A timestamp of the perception measurement quantity;

[0711] Information about the sending device of the perception measurement quantity;

[0712] Information about the receiving device of the perception measurement quantity;

[0713] Coordinate information of the perception measurement quantity;

[0714] Information for indicating the performance index of the perception measurement quantity;

[0715] Information for indicating the source of the perception measurement quantity;

[0716] Information for indicating the type of the perception measurement quantity;

[0717] Information for indicating the perception mode of the perception measurement quantity;

[0718] Configuration information of the first signal;

[0719] Information about the sending device of the first signal;

[0720] Information about the receiving device of the first signal;

[0721] Wherein, the first signal is a signal used for perception.

[0722] Optionally, the first information further includes the description information of the first data set, and the description information of the first data set includes at least one of the following:

[0723] The identifier of the AI unit;

[0724] The identifier of the monitoring sample;

[0725] The quantity of the monitoring sample;

[0726] The number of types of the monitoring sample;

[0727] A first indication for indicating the perception measurement quantity known to the second device;

[0728] A second indication for indicating the valid time of the monitoring sample;

[0729] The perception measurement quantity description information corresponding to each type of the monitoring sample;

[0730] The proportion of valid perception measurement quantities in each type of the monitoring sample;

[0731] The valid time of the set of sensed measurement quantities;

[0732] A third indication for indicating a monitoring sample associated with the sensed measurement quantity carried in the first dataset;

[0733] A fourth indication for indicating the number or proportion of monitoring samples associated with the sensed measurement quantity carried in the first dataset.

[0734] Optionally, the description information of the sensed measurement quantity includes at least one of the following:

[0735] The parameter item of the sensed measurement quantity;

[0736] The number of parameter items of the sensed measurement quantity;

[0737] The processing level of the sensed measurement quantity;

[0738] Information for indicating the sending device of the sensed measurement quantity;

[0739] Information for indicating the receiving device of the sensed measurement quantity;

[0740] The source information of the sensed measurement quantity;

[0741] The link identification information of the sensed measurement quantity;

[0742] A sensing signal configuration identifier for indicating the sensing signal corresponding to the sensed measurement quantity;

[0743] Sensing service information;

[0744] Data subscription identifier;

[0745] Information for indicating the use of the sensed measurement quantity;

[0746] Information about the device corresponding to the sensed measurement quantity;

[0747] The coordinate information of the sensed measurement quantity;

[0748] Performance index information corresponding to the sensed measurement quantity.

[0749] Optionally, the obtaining module is further configured to perform at least one of the following:

[0750] Receive the sensed measurement-related data sent by the second device;

[0751] Receive a first signal sent by the second device or the third device, and obtain sensed measurement-related data according to the first signal;

[0752] Receive the perception measurement-related data sent by the third device, where the perception measurement-related data is obtained by the third device based on sensor device measurements or based on second-signal measurements, or is the perception measurement-related data received from other perception devices;

[0753] Wherein, the first signal or the second signal is a signal used for perception.

[0754] Optionally, the device further includes:

[0755] A first receiving module, configured to receive the first configuration sent by the second device or the fourth configuration sent by the third device, where the first configuration or the fourth configuration is used to indicate at least one of the following:

[0756] The identifier of the AI unit;

[0757] The reporting method of the first data set;

[0758] The number of monitoring samples;

[0759] The reporting method of the monitoring samples;

[0760] Whether to include a third indication, where the third indication is used to indicate the monitoring samples associated with the perception measurement quantity carried in the first data set;

[0761] Whether to include a fourth indication, where the fourth indication is used to indicate the number or proportion of the monitoring samples associated with the perception measurement quantity carried in the first data set;

[0762] The first data set identifier, and the data set corresponding to the first data set identifier includes perception data;

[0763] The configuration identifier of the perception measurement quantity;

[0764] The information for indicating the number threshold of the perception measurement quantity;

[0765] The information for indicating the source of the perception measurement quantity;

[0766] The information for indicating the type of the perception measurement quantity;

[0767] The information for indicating the use of the perception measurement quantity;

[0768] The information for indicating the perception mode of the perception measurement quantity;

[0769] The information for indicating the perception requirement;

[0770] The configuration information of the first signal;

[0771] The information of the sending device of the first signal;

[0772] Information of the receiving device of the first signal;

[0773] Information of the sensing measurement link of the first signal.

[0774] Optionally, the first indication includes at least one of the following:

[0775] Identity of the monitoring sample;

[0776] Identity of the resources used for communicating measurement results;

[0777] Reporting identity of the communication measurement results;

[0778] Identity of the resources used for sensing measurement quantities;

[0779] Reporting identity of the sensing measurement quantities;

[0780] Identity of the reference sample of the sensing measurement quantity;

[0781] Indication of the measurement timestamp of the sensing measurement quantity;

[0782] Indication of the parameter item of the sensing measurement quantity;

[0783] Identity of the sensing measurement quantity.

[0784] Optionally, the second indication includes at least one of the following:

[0785] The third signal measurement timestamp carried by each monitoring sample in the first dataset;

[0786] The third signal measurement timestamp carried by the first monitoring sample in the first dataset, and the relative time of the other monitoring samples in the first dataset except the first monitoring sample relative to the third signal measurement timestamp;

[0787] The third signal measurement timestamp carried by the first monitoring sample in the first dataset;

[0788] The third signal measurement timestamp carried by the first monitoring sample in the first dataset, and the relative time of the last monitoring sample relative to the timestamp;

[0789] The third signal measurement timestamp carried by the last monitoring sample in the first dataset;

[0790] The third timestamp information of each type of monitoring sample in the first dataset;

[0791] The third timestamp information of the first type of monitoring sample in the first dataset, and the relative time of the remaining other types of monitoring samples relative to the third timestamp of the first type of monitoring sample;

[0792] Among them, the third timestamp information of the first type of monitoring samples includes at least one of the following:

[0793] The third signal measurement timestamp of the first monitoring sample in the first type of monitoring samples;

[0794] The identifier of the first monitoring sample in the first type of monitoring samples;

[0795] The third signal measurement timestamp of the last monitoring sample in the first type of monitoring samples;

[0796] The third signal measurement timestamp of the first monitoring sample in the first type of monitoring samples, and the relative time of the third signal measurement of the last monitoring sample relative to the third signal measurement timestamp of the first monitoring sample;

[0797] Among them, the third signal is a signal used for communication measurement, and the first type of monitoring samples is any type of monitoring samples in the first dataset.

[0798] Optionally, the device further includes a determination module for at least one of the following:

[0799] Determine whether to generate the first information based on the perception measurement quantity based on the first timestamp, the second timestamp, and the effective duration of the perception measurement quantity;

[0800] Determine whether to generate the first information based on the perception measurement quantity based on the fifth indication and the second timestamp;

[0801] Determine whether to generate the first information based on the perception measurement quantity based on whether the sixth indication is received;

[0802] Among them, the first timestamp is the timestamp related to the perception measurement quantity;

[0803] The second timestamp is the measurement timestamp of the last communication measurement result corresponding to the true value in the communication measurement results;

[0804] The fifth indication is used to indicate the failure time or failure time difference of the perception measurement quantity;

[0805] The sixth indication is used to indicate the failure of the perception measurement quantity.

[0806] Optionally, the first timestamp includes at least one of the following:

[0807] The measurement timestamp of the perception measurement quantity;

[0808] The reception timestamp of the perception measurement quantity.

[0809] Optionally, the determination module is further used for at least one of the following:

[0810] When the time corresponding to the second timestamp is earlier than the first time, determine to generate the first information based on the perception measurement quantity and the communication measurement related data; the first time is the time corresponding to the first timestamp plus the effective duration of the perception measurement quantity.

[0811] When the time corresponding to the second timestamp is earlier than the second time, determine to generate the first information based on the perception measurement quantity and the communication measurement related data; the second time is the expiration time of the perception measurement quantity determined based on the fifth indication.

[0812] When the first device does not receive the sixth indication, determine to generate the first information based on the perception measurement quantity and the communication measurement related data.

[0813] Optionally, the first information further includes: second information and third information, where the second information is used to characterize the characteristics of the first data set, and the third information is used to characterize the acquisition description information of the perception measurement quantity.

[0814] Optionally, the second information includes at least one of the following:

[0815] An indication of whether the monitored sample is associated with a perception measurement quantity;

[0816] The number of monitored samples associated with the perception measurement quantity;

[0817] The proportion of monitored samples associated with the perception measurement quantity;

[0818] The number of monitored samples associated with the valid perception measurement quantity;

[0819] The proportion of monitored samples associated with the valid perception measurement quantity;

[0820] Whether the number of monitored samples associated with the perception measurement quantity meets the minimum quantity threshold;

[0821] Whether the proportion of monitored samples associated with the perception measurement quantity meets the minimum proportion threshold;

[0822] The proportion of perception measurement quantities that meet the target timeliness among the perception measurement quantities associated with the monitored sample;

[0823] The number of perception measurement quantities that meet the target timeliness among the perception measurement quantities associated with the monitored sample;

[0824] The effective time information of each perception measurement quantity;

[0825] A first threshold, where the first threshold is the ratio of the number of valid perception measurement quantities associated with a monitored sample to the total number of perception measurement quantities associated with the monitored sample.

[0826] A second threshold, where the second threshold is a ratio of the number of sensed measurement quantities actually associated with a monitoring sample to the total number of supportable sensed measurement quantities associated with the monitoring sample.

[0827] Optionally, the third information includes at least one of the following:

[0828] A parameter item of the sensed measurement quantity associated with the monitoring sample;

[0829] An indication of the number of sensed measurement quantities associated with the monitoring sample;

[0830] A processing level of the sensed measurement quantity associated with the monitoring sample;

[0831] An indication of the source of the sensed measurement quantity associated with the monitoring sample;

[0832] An indication of the sending device of the sensed measurement quantity associated with the monitoring sample;

[0833] An indication of the receiving device of the sensed measurement quantity associated with the monitoring sample;

[0834] An indication of the type of the sensed measurement quantity associated with the monitoring sample.

[0835] Coordinates of the sensed measurement quantity associated with the monitoring sample;

[0836] A sensing mode of the sensed measurement quantity associated with the monitoring sample;

[0837] A sensing service of the sensed measurement quantity associated with the monitoring sample;

[0838] A use of the sensed measurement quantity associated with the monitoring sample;

[0839] A performance index of the sensed measurement quantity associated with the monitoring sample;

[0840] An indication that the sensed measurement quantity associated with the monitoring sample is a fixed sensed measurement quantity or a combination of sensed measurement quantities;

[0841] An indication that the sensed measurement quantity associated with the monitoring sample is a variable sensed measurement quantity or a combination of sensed measurement quantities;

[0842] An indication that the number of sensed measurement quantities associated with the monitoring sample is variable;

[0843] Configuration information of a fourth signal;

[0844] Wherein, the fourth signal is a signal corresponding to the sensed measurement quantity associated with the monitoring sample and used for sensing.

[0845] The device provided by the embodiment of the present application can implement each process implemented by the embodiment of the method for monitoring the inference performance of the AI unit as described above, and achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0846] Please refer to Figure 7 , Figure 7 FIG. 2 is a second structural diagram of a device for monitoring the inference performance of an AI unit provided by an embodiment of the present application. The device is applied to a second device. As Figure 7 shown, the device 700 for monitoring the inference performance of the AI unit includes:

[0847] A second receiving module 701, configured to receive first information sent by a first device;

[0848] A monitoring module 702, configured to monitor the inference performance of the AI unit based on the first information, and generate a first metric, where the first metric includes a monitoring metric for the inference performance of the AI unit;

[0849] Wherein, the first information is generated based on perception measurement-related data and communication measurement-related data, the first information includes a first data set, and the first data set includes monitoring samples for monitoring the inference performance of the AI unit.

[0850] Optionally, the device further includes a second sending module, configured to perform any one of the following:

[0851] Send the perception measurement-related data to the first device;

[0852] Send a first signal to the first device, where the first signal is used to obtain the perception measurement-related data, and the first signal is a signal used for perception.

[0853] Optionally, the second sending module is further configured to:

[0854] Send a first configuration to the first device, where the first configuration is used to indicate at least one of the following:

[0855] The identifier of the AI unit;

[0856] The reporting method of the first data set;

[0857] The quantity of the monitoring samples;

[0858] The reporting method of the monitoring samples;

[0859] Whether it includes a third indication, where the third indication is used to indicate the monitoring samples associated with the perception measurement quantities carried in the first data set;

[0860] Whether it includes a fourth indication for indicating the number or proportion of monitoring samples associated with the sensed measurement quantity carried in the first data set;

[0861] The first data set identifier, and the data set corresponding to the first data set identifier includes sensed data;

[0862] The configuration identifier of the sensed measurement quantity;

[0863] Information for indicating the threshold number of sensed measurement quantities;

[0864] Information for indicating the source of the sensed measurement quantity;

[0865] Information for indicating the type of the sensed measurement quantity;

[0866] Information for indicating the use of the sensed measurement quantity;

[0867] Information for indicating the sensing mode of the sensed measurement quantity;

[0868] Information for indicating the sensing requirement;

[0869] The configuration information of the first signal;

[0870] Information about the sending device of the first signal;

[0871] Information about the receiving device of the first signal;

[0872] Information about the sensing measurement link of the first signal.

[0873] Optionally, the second sending module is further configured to: determine the first configuration based on the monitoring requirement and send the first configuration to a third device;

[0874] Alternatively, the second receiving module is further configured to perform any one of the following:

[0875] Receive a third configuration sent by a third device and determine the first configuration;

[0876] Receive a second configuration sent by a third device and send feedback information to the third device, where the feedback information is used to indicate the difference between the first configuration and the second configuration or to indicate the desired first configuration.

[0877] The device provided in the embodiments of the present application can implement each process implemented by the monitoring method embodiments of the AI unit inference performance as described above and achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0878] Please refer to Figure 8 , Figure 8This is the third structural diagram of a monitoring device for the inference performance of an AI unit provided by an embodiment of the present application. The device is applied to a third device. As Figure 8 shown, the monitoring device 800 for the inference performance of the AI unit includes:

[0879] A third sending module 801, configured to send perception measurement-related data or a first signal to a first device, where the first signal is used to obtain the perception measurement-related data;

[0880] Among them, the perception measurement-related data is used for the first device to generate first information, the first information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of the AI unit, the first information is used for the second device to generate a first metric, and the first metric includes an AI unit inference performance monitoring metric.

[0881] Optionally, the third sending module is further configured to:

[0882] Send a fourth configuration to the first device, where the fourth configuration is used to indicate at least one of the following:

[0883] The identifier of the AI unit;

[0884] The reporting method of the first data set;

[0885] The number of the monitoring samples;

[0886] The reporting method of the monitoring samples;

[0887] Whether to include a third indication, where the third indication is used to indicate the monitoring samples associated with the perception measurement quantity carried in the first data set;

[0888] Whether to include a fourth indication, where the fourth indication is used to indicate the number or proportion of the monitoring samples associated with the perception measurement quantity carried in the first data set;

[0889] The first data set identifier, and the data set corresponding to the first data set identifier includes perception data;

[0890] The configuration identifier of the perception measurement quantity;

[0891] Information for indicating the number threshold of the perception measurement quantity;

[0892] Information for indicating the source of the perception measurement quantity;

[0893] Information for indicating the type of the perception measurement quantity;

[0894] Information for indicating the use of the perception measurement quantity;

[0895] Information for indicating a sensing mode of a sensed measurement quantity;

[0896] Information for indicating a sensing requirement;

[0897] Configuration information of the first signal;

[0898] Information of a transmitting device of the first signal;

[0899] Information of a receiving device of the first signal;

[0900] Information of a sensing measurement link of the first signal.

[0901] Optionally, the third transmitting module is further configured to: determine the third configuration, and send the third configuration to the second device;

[0902] Alternatively, the apparatus is further configured to perform any one of the following:

[0903] Receive the first configuration sent by the second device;

[0904] Send a second configuration to the second device, and receive feedback information sent by the second device, where the feedback information is used to indicate a difference between the first configuration and the second configuration, or is used to indicate a desired first configuration.

[0905] The apparatus provided in the embodiments of the present application can implement each process implemented by the embodiments of the method for monitoring the inference performance of the AI unit as described above, and achieve the same technical effects. To avoid repetition, details are not described herein again.

[0906] It should be noted that the above apparatus may be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal or other devices other than a terminal. Exemplarily, the terminal may include, but is not limited to, the types of the terminal 11 listed above, and other devices may be a server, a Network Attached Storage (NAS), etc. The embodiments of the present application do not make specific limitations.

[0907] As Figure 9 shown, the embodiments of the present application further provide a communication device 900, including a processor 901 and a memory 902. A program or instruction that can run on the processor 901 is stored on the memory 902. When the program or instruction is executed by the processor m01, each step of the embodiments of the above method for monitoring the inference performance of each AI unit is implemented, and the same technical effects can be achieved. To avoid repetition, details are not described herein again.

[0908] An embodiment of the present application further provides a terminal, including a processor and a communication interface, where the communication interface is coupled to the processor, and the processor is configured to run programs or instructions to implement the steps in the foregoing embodiment of the method for monitoring the inference performance of the AI unit. Each implementation process and implementation manner of the foregoing method embodiment can be applied to this terminal embodiment and can achieve the same technical effect. Specifically, Figure 10 It is a schematic diagram of the hardware structure of a terminal according to an embodiment of the present application.

[0909] The terminal 1000 includes, but is not limited to, at least some components such as a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010.

[0910] Those skilled in the art can understand that the terminal 1000 may further include a power source (such as a battery) for supplying power to each component. The power source can be logically connected to the processor 1010 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. Figure 10 The terminal structure shown does not limit the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0911] It should be understood that in the embodiment of the present application, the input unit 1004 may include a graphics processing unit (GPU) 10041 and a microphone 10042. The graphics processing unit 10041 processes image data of static pictures or videos obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include two parts: a touch detection device and a touch controller. The other input devices 10072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated here.

[0912] In an embodiment of the present application, after the radio frequency unit 1001 receives downlink data from a network-side device, it can be transmitted to the processor 1010 for processing; in addition, the radio frequency unit 1001 can send uplink data to the network-side device. Generally, the radio frequency unit 1001 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, etc.

[0913] The memory 1009 can be used to store software programs or instructions and various data. The memory 1009 mainly includes a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area can store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 1009 can include a volatile memory or a non-volatile memory 100. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synch link dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The memory 1009 in the embodiment of the present application includes, but is not limited to, these and any other suitable types of memories.

[0914] The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor may not be integrated into the processor 1010.

[0915] Wherein, when the terminal is the first device, the processor 1010 is configured to: obtain perception measurement-related data and communication measurement-related data, and generate first information based on the perception measurement-related data and the communication measurement-related data;

[0916] The radio frequency unit 1001 is configured to send the first information to the second device;

[0917] Wherein, the first information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of the AI unit, and the first information is used for the second device to generate a first metric, and the first metric includes an AI unit inference performance monitoring metric.

[0918] Alternatively, when the terminal is the second device, the radio frequency unit 1001 is configured to receive the first information sent by the first device;

[0919] The processor 1010 is configured to monitor the inference performance of the AI unit based on the first information and generate a first metric, and the first metric includes an AI unit inference performance monitoring metric;

[0920] Wherein, the first information is generated based on perception measurement-related data and communication measurement-related data, the first information includes a first data set, and the first data set includes monitoring samples for monitoring the inference performance of the AI unit.

[0921] Alternatively, when the terminal is the third device, the radio frequency unit 1001 is configured to send perception measurement-related data or a first signal to the first device, and the first signal is used to obtain the perception measurement-related data;

[0922] Wherein, the perception measurement-related data is used for the first device to generate first information, the first information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of the AI unit, and the first information is used for the second device to generate a first metric, and the first metric includes an AI unit inference performance monitoring metric.

[0923] It can be understood that the implementation processes of the implementation manners mentioned in this embodiment can refer to the relevant descriptions in the above method embodiment, and the terminal in this embodiment can achieve the same or corresponding technical effects. To avoid repetition, it will not be elaborated here.

[0924] An embodiment of the present application further provides a network-side device, including a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to run programs or instructions to implement the steps of the foregoing method embodiments. This embodiment of the network-side device corresponds to the foregoing method embodiments of the network-side device. Each implementation process and implementation manner of the foregoing method embodiments can be applied to this embodiment of the network-side device and can achieve the same technical effects.

[0925] Specifically, an embodiment of the present application further provides a network-side device. As Figure 11 shown, the network-side device 1100 includes: an antenna 111, a radio frequency device 112, a baseband device 113, a processor 114, and a memory 115. The antenna 111 is connected to the radio frequency device 112. In the uplink direction, the radio frequency device 112 receives information through the antenna 111 and sends the received information to the baseband device 113 for processing. In the downlink direction, the baseband device 113 processes the information to be sent and sends it to the radio frequency device 112. After processing the received information, the radio frequency device 112 sends it out through the antenna 111.

[0926] The method executed by the network-side device in the foregoing embodiments can be implemented in the baseband device 113. The baseband device 113 includes a baseband processor.

[0927] The baseband device 113 may include, for example, at least one baseband board. A plurality of chips are provided on the baseband board. As Figure 11 shown, one of the chips is, for example, a baseband processor, which is connected to the memory 115 through a bus interface to call the programs in the memory 115 and execute the operations of the network device shown in the foregoing method embodiments.

[0928] The network-side device may further include a network interface 116, which is, for example, a Common Public Radio Interface (CPRI).

[0929] Specifically, the network-side device 1100 in the embodiment of the present invention further includes: instructions or programs stored on the memory 115 and executable on the processor 114. The processor 114 calls the instructions or programs in the memory 115 to execute Figure 6 or Figure 7 or Figure 8 the methods executed by the foregoing modules shown and achieve the same technical effects. To avoid repetition, details are not described herein.

[0930] Specifically, an embodiment of the present application further provides a network-side device. As Figure 12As shown in the figure, the network-side device 1200 includes: a processor 1201, a network interface 1202, and a memory 1203. Among them, the network interface 1202 is, for example, a common public radio interface (CPRI).

[0931] Specifically, the network-side device 1200 in the embodiments of the present invention further includes: instructions or programs stored on the memory 1203 and executable on the processor 1201. The processor 1201 calls the instructions or programs in the memory 1203 to execute Figure 6 or Figure 7 or Figure 8 the methods executed by the modules shown in the figure, and achieves the same technical effects. To avoid repetition, they will not be elaborated here.

[0932] The embodiments of the present application further provide a readable storage medium. Programs or instructions are stored on the readable storage medium. When the programs or instructions are executed by a processor, the above Figure 2 、 Figure 4 or Figure 5 processes of the method embodiments are implemented, and the same technical effects can be achieved. To avoid repetition, they will not be elaborated here.

[0933] Among them, the processor is the processor in the terminal described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disks, or optical discs, etc. In some examples, the readable storage medium may be a non-transitory readable storage medium.

[0934] The embodiments of the present application further provide a chip. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the above Figure 2 、 Figure 4 or Figure 5 processes of the method embodiments, and the same technical effects can be achieved. To avoid repetition, they will not be elaborated here.

[0935] It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, system chip, chip system, or system-on-chip.

[0936] The embodiments of the present application further provide a computer program / program product. The computer program / program product is stored in a storage medium. The computer program / program product is executed by at least one processor to implement the above Figure 2 、 Figure 4 or Figure 5 processes of the method embodiments, and the same technical effects can be achieved. To avoid repetition, they will not be elaborated here.

[0937] The embodiment of the present application also provides a communication system, including: a first device, a second device and a third device. The first device can be used to execute the steps of the method described in the first aspect, the second device can be used to execute the steps of the method described in the second aspect, and the third device can be used to execute the steps of the method described in the third aspect.

[0938] It should be noted that in this document, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device including a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0939] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described example methods can be implemented by means of a computer software product plus a necessary general hardware platform, and of course, can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes several instructions for causing a terminal or a network-side device to execute the methods described in various embodiments of the present application.

[0940] The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms of embodiments without departing from the purpose of the present application and the scope protected by the claims. These embodiments are all within the protection scope of the present application.

Claims

1. A method for monitoring the inference performance of an artificial intelligence AI unit, characterized in that Including: The first device obtains perception measurement-related data and communication measurement-related data, and generates first information based on the perception measurement-related data and the communication measurement-related data; The first device sends the first information to the second device; Wherein, the first information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of the AI unit, and the first information is used for the second device to generate a first metric, and the first metric includes an AI unit inference performance monitoring metric.

2. The method according to claim 1, wherein The monitoring samples include at least one of the following: The input of the AI unit; The output of the AI unit; True value, which is used to indicate the actual measured value of the parameter corresponding to the output of the AI unit.

3. The method according to claim 2, wherein The input of the AI unit includes the perception measurement-related data, and the perception measurement-related data includes perception measurement quantities and at least one of the following: Indicator of the perception measurement quantity; Timestamp of the perception measurement quantity; Information of the sending device of the perception measurement quantity; Information of the receiving device of the perception measurement quantity; Coordinate information of the perception measurement quantity; Information for indicating the performance metric of the perception measurement quantity; Information for indicating the source of the perception measurement quantity; Information for indicating the type of the perception measurement quantity; Information for indicating the perception mode of the perception measurement quantity; Configuration information of the first signal; Information of the sending device of the first signal; Information of the receiving device of the first signal; Wherein, the first signal is a signal used for perception.

4. The method according to any one of claims 1 to 3, characterized in that, The first information further includes description information of the first data set, and the description information of the first data set includes at least one of the following: The identifier of the AI unit; The identifier of the monitoring sample; The number of the monitoring samples; The number of types of the monitoring samples; A first indication, which is used to indicate the perception measurement quantities known to the second device; A second indication, which is used to indicate the valid time of the monitoring samples; Perception measurement quantity description information corresponding to each type of the monitoring samples; The proportion of valid perception measurement quantities in each type of the monitoring samples; The valid time of the perception measurement quantity set; A third indication, which is used to indicate the monitoring samples associated with the perception measurement quantities carried in the first data set; A fourth indication, which is used to indicate the number or proportion of the monitoring samples associated with the perception measurement quantities carried in the first data set.

5. The method according to claim 4, characterized in that, The perception measurement quantity description information includes at least one of the following: The parameter item of the perception measurement quantity; The number of parameter items of the perception measurement quantity; The processing level of the perception measurement quantity; Information for indicating the sending device of the perception measurement quantity; Information for indicating the receiving device of the perception measurement quantity; Source information of the perception measurement quantity; Perception measurement quantity link identification information; Perception signal configuration identifier, which is used to indicate the perception signal corresponding to the perception measurement quantity; Perception service information; Data subscription identifier; Information for indicating the use of the perception measurement quantity; Information of the device corresponding to the perception measurement quantity; Coordinate information of the perception measurement quantity; Performance metric information corresponding to the perception measurement quantity.

6. The method according to claim 4, characterized in that, The first device obtains the perception measurement-related data, including any one of the following: The first device receives the perception measurement-related data sent by the second device; The first device receives a first signal sent by the second device or the third device, and obtains perception measurement-related data according to the first signal; The first device receives the perception measurement-related data sent by the third device, where the perception measurement-related data is obtained by the third device based on sensor device measurement or measurement based on a second signal, or is perception measurement-related data received from other perception devices; Wherein, the first signal or the second signal is a signal used for perception.

7. The method according to claim 6, wherein Before the first device receives the first signal sent by the second device or the third device, the method further includes: The first device receives a first configuration sent by the second device or a fourth configuration sent by the third device, where the first configuration or the fourth configuration is used to indicate at least one of the following: The identifier of the AI unit; The reporting method of the first data set; The number of monitoring samples; The reporting method of the monitoring samples; Whether to include the third indication; Whether to include the fourth indication; The first data set identifier, and the data set corresponding to the first data set identifier includes perception data; The configuration identifier of the perception measurement quantity; The information used to indicate the number threshold of the perception measurement quantity; The information used to indicate the source of the perception measurement quantity; The information used to indicate the type of the perception measurement quantity; The information used to indicate the use of the perception measurement quantity; The information used to indicate the perception mode of the perception measurement quantity; The information used to indicate the perception requirement; The configuration information of the first signal; The information of the sending device of the first signal; The information of the receiving device of the first signal; The information of the perception measurement link of the first signal.

8. The method according to claim 4, characterized in that The first indication includes at least one of the following: The identifier of the monitoring sample; The identifier of the resource used for communicating the measurement result; The reporting identifier of the communication measurement result; The identifier of the resource used for the perception measurement quantity; The reporting identifier of the perception measurement quantity; The reference sample identifier of the perception measurement quantity; The measurement timestamp indication of the perception measurement quantity; The parameter item indication of the perception measurement quantity; The perception measurement quantity identifier.

9. The method according to claim 4, characterized in that The second indication includes at least one of the following: The third signal measurement timestamp carried by each monitoring sample in the first data set; The third signal measurement timestamp carried by the first monitoring sample in the first data set, and the relative time of the other monitoring samples in the first data set except the first monitoring sample relative to the third signal measurement timestamp; The third signal measurement timestamp carried by the first monitoring sample in the first data set; The third signal measurement timestamp carried by the first monitoring sample in the first data set, and the relative time of the last monitoring sample relative to the timestamp; The third signal measurement timestamp carried by the last monitoring sample in the first data set; The third timestamp information of each type of monitoring sample in the first data set; The third timestamp information of the first type of monitoring sample in the first data set, and the relative time of the remaining other types of monitoring samples relative to the third timestamp of the first type of monitoring sample; Among them, the third timestamp information of the first type of monitoring samples includes at least one of the following: The third signal measurement timestamp of the first monitoring sample in the first type of monitoring samples; The identifier of the first monitoring sample in the first type of monitoring samples; The third signal measurement timestamp of the last monitoring sample in the first type of monitoring samples; The third signal measurement timestamp of the first monitoring sample in the first type of monitoring samples, and the relative time of the third signal measurement of the last monitoring sample relative to the third signal measurement timestamp of the first monitoring sample; Among them, the third signal is a signal used for communication measurement, and the first type of monitoring samples is any type of monitoring samples in the first dataset.

10. The method according to any one of claims 1-9, characterized in that, The perception measurement related data includes perception measurement quantities, and the method further includes any one of the following: The first device determines whether to generate the first information based on the perception measurement quantity based on the first timestamp, the second timestamp, and the effective duration of the perception measurement quantity; The first device determines whether to generate the first information based on the perception measurement quantity based on the fifth indication and the second timestamp; The first device determines whether to generate the first information based on whether the sixth indication is received; Among them, the first timestamp is the timestamp related to the perception measurement quantity; The second timestamp is the measurement timestamp of the last communication measurement result corresponding to the true value in the communication measurement result; The fifth indication is used to indicate the failure time or failure time difference of the perception measurement quantity; The sixth indication is used to indicate the failure of the perception measurement quantity.

11. The method according to claim 10, characterized in that, The first timestamp includes at least one of the following: The measurement timestamp of the perception measurement quantity; The reception timestamp of the perception measurement quantity.

12. The method according to claim 10 or 11, characterized in that The first device generates the first information based on the perception measurement related data and the communication measurement related data, including any one of the following: When the time corresponding to the second timestamp is earlier than the first time, the first device determines to generate the first information based on the perception measurement quantity and the communication measurement related data; the first time is the time corresponding to the first timestamp plus the effective duration of the perception measurement quantity; When the time corresponding to the second timestamp is earlier than the second time, the first device determines to generate the first information based on the perception measurement quantity and the communication measurement related data; The second time is the failure time of the perception measurement quantity determined based on the fifth indication; When the first device does not receive the sixth indication, the first device determines to generate the first information based on the perception measurement quantity and the communication measurement related data.

13. The method according to any one of claims 1-12, characterized in that, The first information further includes: second information and third information, where the second information is used to characterize the characteristics of the first dataset, and the third information is used to characterize the acquisition description information of the perception measurement quantity.

14. The method according to claim 13, wherein The second information includes at least one of the following: The indication of whether the monitoring sample is associated with the perception measurement quantity; The number of monitoring samples associated with the perception measurement quantity; The proportion of monitoring samples associated with the perception measurement quantity; The number of monitoring samples associated with the effective perception measurement quantity; The proportion of monitoring samples associated with valid perception measurement quantities; Whether the number of monitoring samples associated with perception measurement quantities meets the minimum quantity threshold; Whether the proportion of monitoring samples associated with perception measurement quantities meets the minimum proportion threshold; The proportion of perception measurement quantities that meet the target timeliness among the perception measurement quantities associated with the monitoring samples; The number of perception measurement quantities that meet the target timeliness among the perception measurement quantities associated with the monitoring samples; The effective time information of each perception measurement quantity; The first threshold, where the first threshold is the ratio of the number of valid perception measurement quantities associated with a monitoring sample to the total number of perception measurement quantities associated with the monitoring sample; The second threshold, where the second threshold is the ratio of the actual number of perception measurement quantities associated with a monitoring sample to the total number of the maximum supportable perception measurement quantities associated with the monitoring sample; 15. The method according to claim 13, wherein The third information includes at least one of the following: The parameter items of the perception measurement quantities associated with the monitoring sample; The indication of the number of perception measurement quantities associated with the monitoring sample; The processing level of the perception measurement quantities associated with the monitoring sample; The source indication of the perception measurement quantities associated with the monitoring sample; The sending device indication of the perception measurement quantities associated with the monitoring sample; The receiving device indication of the perception measurement quantities associated with the monitoring sample; The type indication of the perception measurement quantities associated with the monitoring sample; The coordinates of the perception measurement quantities associated with the monitoring sample; The perception mode of the perception measurement quantities associated with the monitoring sample; The perception service of the perception measurement quantities associated with the monitoring sample; The use of the perception measurement quantities associated with the monitoring sample; The performance indicators of the perception measurement quantities associated with the monitoring sample; The indication that the perception measurement quantities associated with the monitoring sample are fixed perception measurement quantities or a combination of perception measurement quantities; The indication that the perception measurement quantities associated with the monitoring sample are variable perception measurement quantities or a combination of perception measurement quantities; The indication that the number of perception measurement quantities associated with the monitoring sample is variable; The configuration information of the fourth signal; Wherein, the fourth signal is the signal corresponding to the perception measurement quantities associated with the monitoring sample and used for perception.

16. A method for monitoring the inference performance of an AI unit, characterized in that, Including: The second device receives the first information sent by the first device; The second device performs AI unit inference performance monitoring based on the first information and generates a first metric, where the first metric includes the AI unit inference performance monitoring metric; Wherein, the first information is generated based on perception measurement related data and communication measurement related data, the first information includes a first data set, and the first data set includes monitoring samples for performing AI unit inference performance monitoring.

17. The method according to claim 16, wherein The monitoring samples include at least one of the following: The input of the AI unit; The output of the AI unit; The true value, which is used to indicate the actual measured value of the parameter corresponding to the output of the AI unit.

18. The method according to any one of claims 16 - 17, characterized in that, The first information further includes the description information of the first data set, and the description information of the first data set includes at least one of the following: The identifier of the AI unit; The identifier of the monitoring sample; The number of the monitoring samples; The number of types of the monitoring samples; The first indication, which is used to indicate the perception measurement quantities known to the second device. A second indication for indicating the valid time of the monitored sample; Description information of the sensed measurement quantity corresponding to each type of the monitored sample; The proportion of valid sensed measurement quantities in each type of the monitored sample; The valid time of the sensed measurement quantity set; A third indication for indicating the monitored sample associated with the sensed measurement quantity carried in the first dataset; A fourth indication for indicating the number or proportion of the monitored samples associated with the sensed measurement quantity carried in the first dataset.

19. The method according to claim 18, wherein The second indication includes at least one of the following: The third signal measurement timestamp carried by each monitored sample in the first dataset; The third signal measurement timestamp carried by the first monitored sample in the first dataset, and the relative time of the other monitored samples in the first dataset except the first monitored sample relative to the third signal measurement timestamp; The third signal measurement timestamp carried by the first monitored sample in the first dataset; The third signal measurement timestamp carried by the first monitored sample in the first dataset, and the relative time of the last monitored sample relative to the timestamp; The third signal measurement timestamp carried by the last monitored sample in the first dataset; The third timestamp information of each type of monitored sample in the first dataset; The third timestamp information of the first type of monitored sample in the first dataset, and the relative time of the remaining other types of monitored samples relative to the third timestamp of the first type of monitored sample; Wherein, the third timestamp information of the first type of monitored sample includes at least one of the following: The third signal measurement timestamp of the first monitored sample in the first type of monitored sample; The identifier of the first monitored sample in the first type of monitored sample; The third signal measurement timestamp of the last monitored sample in the first type of monitored sample; The third signal measurement timestamp of the first monitored sample in the first type of monitored sample, and the relative time of the third signal measurement of the last monitored sample relative to the third signal measurement timestamp of the first monitored sample; Wherein, the third signal is a signal used for communication measurement, and the first type of monitored sample is any type of monitored sample in the first dataset.

20. The method according to claim 18, wherein Before the second device receives the first information sent by the first device, the method further includes any one of the following: The second device sends the sensed measurement related data to the first device; The second device sends a first signal to the first device, the first signal is used to obtain the sensed measurement related data, and the first signal is a signal used for sensing.

21. The method according to claim 20, characterized in that, Before the second device sends the first signal to the first device, the method further includes: The second device sends a first configuration to the first device, the first configuration is used to indicate at least one of the following: The identifier of the AI unit; The reporting method of the first dataset; The number of the monitored samples; The reporting method of the monitored samples; Whether to include the third indication; Whether to include the fourth indication; The first dataset identifier, and the dataset corresponding to the first dataset identifier includes sensing data; Configuration identifier of the sensed measurement quantity; Information for indicating the number threshold of the sensed measurement quantity; Information for indicating the source of the sensed measurement quantity; Information for indicating the type of the sensed measurement quantity; Information for indicating the use of the sensed measurement quantity; Information for indicating the sensing mode of the sensed measurement quantity; Information for indicating the sensing requirement; Configuration information of the first signal; Information of the sending device of the first signal; Information of the receiving device of the first signal; Information of the sensing measurement link of the first signal.

22. The method according to claim 21, wherein The method further includes any one of the following: The second device determines the first configuration based on the monitoring requirement and sends the first configuration to the third device; The second device receives the third configuration sent by the third device and determines the first configuration; The second device receives the second configuration sent by the third device and sends feedback information to the third device, where the feedback information is used to indicate the difference between the first configuration and the second configuration or to indicate the expected first configuration.

23. A method for monitoring the inference performance of an AI unit, characterized in that, Includes: The third device sends sensed measurement-related data or a first signal to the first device, where the first signal is used to obtain sensed measurement-related data; Wherein, the sensed measurement-related data is used for the first device to generate first information, the first information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of the AI unit, and the first information is used for the second device to generate a first metric, and the first metric includes an AI unit inference performance monitoring metric.

24. The method according to claim 23, wherein The method further includes: The third device sends a fourth configuration to the first device, and the fourth configuration is used to indicate at least one of the following: The identifier of the AI unit; The reporting method of the first data set; The quantity of the monitoring samples; The reporting method of the monitoring samples; Whether to include a third indication, where the third indication is used to indicate the monitoring samples associated with the sensed measurement quantity carried in the first data set; Whether to include a fourth indication, where the fourth indication is used to indicate the quantity or proportion of the monitoring samples associated with the sensed measurement quantity carried in the first data set; The first data set identifier, and the data set corresponding to the first data set identifier includes sensing data; Configuration identifier of the sensed measurement quantity; Information for indicating the number threshold of the sensed measurement quantity; Information for indicating the source of the sensed measurement quantity; Information for indicating the type of the sensed measurement quantity; Information for indicating the use of the sensed measurement quantity; Information for indicating the sensing mode of the sensed measurement quantity; Information for indicating the sensing requirement; Configuration information of the first signal; Information of the sending device of the first signal; Information of the receiving device of the first signal; Information of the sensing measurement link of the first signal.

25. The method according to claim 24, wherein The method further includes any one of the following: The third device determines the third configuration and sends the third configuration to the second device; The third device receives the first configuration sent by the second device; The third device sends a second configuration to the second device and receives feedback information sent by the second device. The feedback information is used to indicate the difference between the first configuration and the second configuration, or to indicate the desired first configuration.

26. A monitoring device for the inference performance of an AI unit, which is applied to a first device, is characterized in that The device includes: An acquisition module, configured to acquire perception measurement-related data and communication measurement-related data, and generate first information based on the perception measurement-related data and the communication measurement-related data; A first sending module, configured to send the first information to a second device; Wherein, the first information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of an AI unit, and the first information is used for the second device to generate a first metric, and the first metric includes an AI unit inference performance monitoring metric.

27. The device according to claim 26, characterized in that, The acquisition module is further configured to perform at least one of the following: Receive perception measurement-related data sent by the second device; Receive a first signal sent by the second device or a third device, and obtain perception measurement-related data according to the first signal; Receive perception measurement-related data sent by a third device, where the perception measurement-related data is obtained by the third device through sensor device measurement or based on a second signal measurement, or is perception measurement-related data received from other perception devices; Wherein, the first signal or the second signal is a signal used for perception.

28. The device according to claim 27, characterized in that, The device further includes: A first receiving module, configured to receive a first configuration sent by the second device or a fourth configuration sent by the third device, where the first configuration or the fourth configuration is used to indicate at least one of the following: The identifier of the AI unit; The reporting method of the first data set; The quantity of the monitoring samples; The reporting method of the monitoring samples; Whether to include a third indication, where the third indication is used to indicate monitoring samples associated with the perception measurement quantity carried in the first data set; Whether to include a fourth indication, where the fourth indication is used to indicate the quantity or proportion of monitoring samples associated with the perception measurement quantity carried in the first data set; The first data set identifier, and the data set corresponding to the first data set identifier includes perception data; The configuration identifier of the perception measurement quantity; Information for indicating the number threshold of the perception measurement quantity; Information for indicating the source of the perception measurement quantity; Information for indicating the type of the perception measurement quantity; Information for indicating the use of the perception measurement quantity; Information for indicating the perception mode of the perception measurement quantity; Information for indicating the perception requirement; The configuration information of the first signal; The information of the sending device of the first signal; The information of the receiving device of the first signal; The information of the perception measurement link of the first signal.

29. The device according to claim 26, characterized in that, The device further includes a determination module, configured to perform at least one of the following: Based on a first timestamp, a second timestamp, and the effective duration of the perception measurement quantity, determine whether to generate the first information based on the perception measurement quantity; Based on a fifth indication and a second timestamp, determine whether to generate the first information based on the perception measurement quantity; Based on whether a sixth indication is received, determine whether to generate the first information based on the perception measurement quantity; Wherein, the first timestamp is the timestamp related to the sensed measurement quantity; The second timestamp is the measurement timestamp of the last communication measurement result corresponding to the true value in the communication measurement result; The fifth indication is used to indicate the failure time or failure time difference of the sensed measurement quantity; The sixth indication is used to indicate the failure of the sensed measurement quantity.

30. The device according to claim 29, characterized in that, The determining module is further configured to perform at least one of the following: When the time corresponding to the second timestamp is earlier than the first time, determining to generate the first information based on the sensed measurement quantity and communication measurement related data; the first time is the time corresponding to the first timestamp superimposed with the effective duration of the sensed measurement quantity; When the time corresponding to the second timestamp is earlier than the second time, determining to generate the first information based on the sensed measurement quantity and communication measurement related data; The second time is the failure time of the sensed measurement quantity determined based on the fifth indication; When the first device does not receive the sixth indication, determining to generate the first information based on the sensed measurement quantity and communication measurement related data.

31. A monitoring device for the inference performance of an AI unit, which is applied to a second device, characterized in that, The device includes: A second receiving module, configured to receive the first information sent by the first device; A monitoring module, configured to monitor the inference performance of the AI unit based on the first information and generate a first metric, where the first metric includes an AI unit inference performance monitoring metric; Wherein, the first information is generated based on sensed measurement related data and communication measurement related data, the first information includes a first data set, and the first data set includes monitoring samples for performing AI unit inference performance monitoring.

32. The device according to claim 31, characterized in that, The device further includes a second sending module, configured to perform any one of the following: Send the sensed measurement related data to the first device; Send a first signal to the first device, where the first signal is used to obtain the sensed measurement related data, and the first signal is a signal used for sensing.

33. The device according to claim 32, characterized in that, The second sending module is further configured to: Send a first configuration to the first device, where the first configuration is used to indicate at least one of the following: The identifier of the AI unit; The reporting method of the first data set; The number of monitoring samples; The reporting method of the monitoring samples; Whether to include a third indication, where the third indication is used to indicate the monitoring samples associated with the sensed measurement quantity carried in the first data set; Whether to include a fourth indication, where the fourth indication is used to indicate the number or proportion of the monitoring samples associated with the sensed measurement quantity carried in the first data set; The first data set identifier, and the data set corresponding to the first data set identifier includes sensed data; The configuration identifier of the sensed measurement quantity; Information for indicating the number threshold of the sensed measurement quantity; Information for indicating the source of the sensed measurement quantity; Information for indicating the type of the sensed measurement quantity; Information for indicating the use of the sensed measurement quantity; Information for indicating the sensing mode of the sensed measurement quantity; Information for indicating the sensing requirement; The configuration information of the first signal; The information of the sending device of the first signal; The information of the receiving device of the first signal; The information of the sensing measurement link of the first signal.

34. A monitoring device for the inference performance of an AI unit, which is applied to a third device, is characterized in that The device includes: A third sending module, configured to send perception measurement related data or a first signal to a first device, where the first signal is used to obtain the perception measurement related data; Wherein, the perception measurement related data is used for the first device to generate first information, the first information includes a first data set, the first data set includes monitoring samples for monitoring the inference performance of an AI unit, and the first information is used for a second device to generate first metrics, the first metrics include AI unit inference performance monitoring metrics.

35. The device according to claim 34, characterized in that, The third sending module is further configured to: Send a fourth configuration to the first device, where the fourth configuration is used to indicate at least one of the following: The identifier of the AI unit; The reporting method of the first data set; The number of the monitoring samples; The reporting method of the monitoring samples; Whether to include a third indication, where the third indication is used to indicate the monitoring samples associated with the perception measurement quantity carried in the first data set; Whether to include a fourth indication, where the fourth indication is used to indicate the number or proportion of the monitoring samples associated with the perception measurement quantity carried in the first data set; The first data set identifier, and the data set corresponding to the first data set identifier includes perception data; The configuration identifier of the perception measurement quantity; Information for indicating the number threshold of the perception measurement quantity; Information for indicating the source of the perception measurement quantity; Information for indicating the type of the perception measurement quantity; Information for indicating the use of the perception measurement quantity; Information for indicating the perception mode of the perception measurement quantity; Information for indicating the perception requirement; The configuration information of the first signal; The information of the sending device of the first signal; The information of the receiving device of the first signal; The information of the perception measurement link of the first signal.

36. A communication device, characterized in that, Comprising a processor and a memory, where the memory stores a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the method for monitoring the inference performance of an AI unit according to any one of claims 1-25 are implemented.

37. A readable storage medium, characterized in that, The program or instruction is stored on the readable storage medium, and when the program or instruction is executed by the processor, the steps of the method for monitoring the inference performance of an AI unit according to any one of claims 1-25 are implemented.

Images