Method and device for monitoring reasoning performance of AI unit and communication equipment

Abstract

The invention discloses an AI unit reasoning performance monitoring method and device and communication equipment, and belongs to the technical field of communication.The AI unit reasoning performance monitoring method comprises the steps that first equipment receives first configuration sent by second equipment, the first configuration is used for indicating collection of monitoring samples, and the second equipment sends the monitoring samples to the first equipment; the monitoring sample is a sample for AI unit reasoning performance monitoring; the first device obtains sensing measurement related data and communication measurement related data, and generates first information according to the first configuration and based on the sensing measurement related data and the communication measurement related data; the first device sends the first information to a second device; wherein the first information comprises at least one of the first data set and a first index, the first data set comprises the monitoring sample, and the first index comprises an AI unit reasoning performance monitoring index.

Description

Technical Field

[0001] This application belongs to the field of communication technologies, and particularly relates to a method, apparatus, and communication device for monitoring the inference performance of an AI unit. Background Art

[0002] Currently, the network can obtain perception results by means of sensors or sensing measurement signals, and sensing measurement has become one of the main research directions in the current communication field. In addition, with the development of Artificial Intelligence (AI) technology, AI has also been applied to communication systems, such as AI-based beam prediction (for example, obtaining predicted beam information based on the inference of an AI unit, and the AI unit is also called an AI model), AI-based Channel State Information (CSI) prediction, AI-based positioning, etc. To ensure the accuracy of the output results of the AI unit, the inference performance of the AI unit is usually monitored. However, how to monitor the inference performance of the AI unit based on the perception results is not clear yet. Summary of the Invention

[0003] Embodiments of this application provide a method, apparatus, and communication device for monitoring the inference performance of an AI unit, which can solve the problem in the related art that the method for monitoring the inference performance of the AI unit based on the perception results is not clear.

[0004] In a first aspect, a method for monitoring the inference performance of an AI unit is provided, which is executed by a first device. The method includes:

[0005] The first device receives a first configuration sent by a second device, where the first configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit;

[0006] The first device obtains perception measurement-related data and communication measurement-related data, and generates first information based on the first configuration and based on the perception measurement-related data and the communication measurement-related data;

[0007] The first device sends the first information to the second device;

[0008] Wherein, the first information includes at least one of a first data set and a first metric. The first data set includes the monitoring samples, and the first metric includes an AI unit inference performance monitoring metric.

[0009] In a first aspect, a method for monitoring the inference performance of an AI unit is provided, which is executed by a second device. The method includes:

[0010] The second device sends a first configuration to the first device, where the first configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit;

[0011] The second device receives the first information sent by the first device;

[0012] Wherein, the first information includes at least one of a first data set and a first metric, the first data set includes the monitoring samples, and the first metric includes an AI unit inference performance monitoring metric.

[0013] In a third aspect, a method for monitoring the inference performance of an AI unit is provided, which is executed by a third device. The method includes:

[0014] The third device sends a third configuration to the first device, where the third configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit.

[0015] In a fourth aspect, a monitoring device for the inference performance of an AI unit is provided, which is applied to the first device. The device includes:

[0016] A first receiving module, configured to receive the first configuration sent by the second device, where the first configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit;

[0017] An obtaining module, configured to obtain perception measurement-related data and communication measurement-related data, and generate first information according to the first configuration and based on the perception measurement-related data and the communication measurement-related data;

[0018] A first sending module, configured to send the first information to the second device;

[0019] Wherein, the first information includes at least one of a first data set and a first metric, the first data set includes the monitoring samples, and the first metric includes an AI unit inference performance monitoring metric.

[0020] In a fifth aspect, a monitoring device for the inference performance of an AI unit is provided, which is applied to the second device. The device includes:

[0021] A second sending module, configured to send a first configuration to the first device, where the first configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit;

[0022] A second receiving module, configured to receive the first information sent by the first device;

[0023] Wherein, the first information includes at least one of a first data set and a first metric, the first data set includes the monitoring samples, and the first metric includes an AI unit inference performance monitoring metric.

[0024] In a sixth aspect, a monitoring device for the inference performance of an AI unit is provided, which is applied to a third device. The device includes:

[0025] A third sending module, configured to send a third configuration to a first device, where the third configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit.

[0026] In a seventh aspect, a communication device is provided. The communication device includes a processor and a memory. The memory stores a program or instruction that can run on the processor. When the program or instruction is executed by the processor, the steps of the method described in the first aspect, the second aspect, or the third aspect are implemented.

[0027] In an eighth aspect, a communication device is provided, including a processor and a communication interface. When the communication device is a first device, the communication interface is configured to receive a first configuration sent by a second device, where the first configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit; the processor is configured to obtain perception measurement-related data and communication measurement-related data, and generate first information based on the first configuration and the perception measurement-related data and the communication measurement-related data; the communication interface is further configured to send the first information to the second device; wherein, the first information includes at least one of a first data set and a first metric, the first data set includes the monitoring samples, and the first metric includes an AI unit inference performance monitoring metric.

[0028] Alternatively, when the communication device is a second device, the communication interface is configured to send a first configuration to the first device, where the first configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit, and receive the first information sent by the first device; wherein, the first information includes at least one of a first data set and a first metric, the first data set includes the monitoring samples, and the first metric includes an AI unit inference performance monitoring metric;

[0029] Alternatively, when the communication device is a third device, the communication interface is configured to send a third configuration to the first device, where the third configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit.

[0030] In a ninth aspect, a readable storage medium is provided, on which a program or instructions are stored. When the program or instructions are executed by a processor, the steps of the method described in the first aspect, or the steps of the method described in the second aspect, or the steps of the method described in the third aspect are implemented.

[0031] In a tenth aspect, a wireless communication system is provided, including: a first device, a second device, and a third device. The first device can be used to execute the steps of the method described in the first aspect, the second device can be used to execute the steps of the method described in the second aspect, and the third device can be used to execute the steps of the method described in the third aspect.

[0032] In an eleventh aspect, a chip is provided, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run a program or instructions to implement the steps of the method described in the first aspect, the second aspect, or the third aspect.

[0033] In a twelfth aspect, a computer program / program product is provided. The computer program / program product is stored in a storage medium, and the program / program product is executed by at least one processor to implement the steps of the method described in the first aspect, the second aspect, or the third aspect.

[0034] In an embodiment of the present application, the first device receives a first configuration sent by the second device. The first configuration is used to indicate the collection of monitoring samples for monitoring the inference performance of the AI unit. Then, after the first device obtains the perception measurement-related data and communication measurement-related data, it can generate a first piece of information according to the indication of the first configuration, combining the perception measurement-related data and the communication measurement-related data, and send the first piece of information to the second device. Among them, the first piece of information includes at least one of a first data set and a first metric. The first data set includes the monitoring samples, and the first metric includes the AI unit inference performance monitoring metric. The first piece of information is generated based on the perception measurement-related data and the communication measurement-related data. Furthermore, it enables the first device or the second device to integrate the perception measurement-related data in the monitoring of the AI unit inference performance. Compared with only using the communication measurement-related data to monitor the AI unit inference performance, the solution provided by the present application is more helpful for improving the inference accuracy of the AI unit to better implement the monitoring of the AI unit inference performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1a is a block diagram of a wireless communication system to which an embodiment of the present application can be applied;

[0036] Figure 1b is a schematic diagram of a perception mode to which an embodiment of the present application can be applied;

[0037] Figure 2 It is one of the flowcharts of a method for monitoring the inference performance of an AI unit provided by an embodiment of the present application;

[0038] Figures 3a - 3m It is a scenario diagram applicable to the method for monitoring the inference performance of an AI unit provided by an embodiment of the present application;

[0039] Figure 4 It is the second flowchart of a method for monitoring the inference performance of an AI unit provided by an embodiment of the present application;

[0040] Figure 5 It is the third flowchart of a method for monitoring the inference performance of an AI unit provided by an embodiment of the present application;

[0041] Figure 6 It is the first structural diagram of a device for monitoring the inference performance of an AI unit provided by an embodiment of the present application;

[0042] Figure 7 It is the second structural diagram of a device for monitoring the inference performance of an AI unit provided by an embodiment of the present application;

[0043] Figure 8 It is the third structural diagram of a device for monitoring the inference performance of an AI unit provided by an embodiment of the present application;

[0044] Figure 9 It is the structural diagram of a communication device provided by an embodiment of the present application;

[0045] Figure 10 It is the structural diagram of a terminal provided by an embodiment of the present application;

[0046] Figure 11 It is the structural diagram of a network-side device provided by an embodiment of the present application;

[0047] Figure 12 It is the structural diagram of another network-side device provided by an embodiment of the present application. Detailed implementation manners

[0048] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, rather than all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope protected by the present application.

[0049] The terms "first", "second", etc. in this application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances so that the embodiments of this application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same type, and do not limit the number of objects. For example, the first object can be one or multiple. In addition, "or" in this application means at least one of the connected objects. For example, "A or B" covers three scenarios, namely, Scenario 1: including A and not including B; Scenario 2: including B and not including A; Scenario 3: including both A and B. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0050] The term "indication" in this application can be either a direct indication (or an explicit indication) or an indirect indication (or an implicit indication). Among them, a direct indication can be understood as that the sender clearly informs the receiver of specific information, operations to be performed, or request results, etc. in the sent indication; an indirect indication can be understood as that the receiver determines the corresponding information according to the indication sent by the sender, or makes a judgment and determines the operations to be performed or request results, etc. according to the judgment result.

[0051] It is worth noting that the technology described in the embodiments of this application is not limited to Long Term Evolution (LTE) / LTE-Advanced (LTE-A) systems, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA), or other systems. The terms "system" and "network" in the embodiments of this application are often used interchangeably, and the described technology can be used in the systems and radio technologies mentioned above, as well as in other systems and radio technologies. The following description describes the New Radio (NR) system for example purposes, and uses NR terms in most of the following descriptions, but these technologies can also be applied to systems other than the NR system, such as the 6th generation (6 thGeneration, 6G) communication system.

[0052] FIG. 1 shows a block diagram of a wireless communication system to which embodiments of the present application can be applied. The wireless communication system includes a terminal 11 and a network-side device 12. Among them, the terminal 11 can be a mobile phone, a tablet personal computer, a laptop computer, a notebook computer, a personal digital assistant (PDA), a handheld computer, a netbook, an ultra-mobile personal computer (UMPC), a mobile internet device (MID), an augmented reality (AR), a virtual reality (VR) device, a robot, a wearable device, a flight vehicle, a vehicle user equipment (VUE), a shipborne device, a pedestrian user equipment (PUE), a smart home (home devices with wireless communication functions, such as refrigerators, TVs, washing machines, or furniture, etc.), a game console, a personal computer (PC), a teller machine, or a self-service machine, etc. Wearable devices include: smart watches, smart bracelets, smart earphones, smart glasses, smart jewelry (smart bracelets, smart bracelets, smart rings, smart necklaces, smart anklets, smart ankle chains, etc.), smart wristbands, smart clothing, etc. Among them, the vehicle user equipment can also be referred to as a vehicle terminal, a vehicle controller, a vehicle module, a vehicle component, a vehicle chip, or a vehicle unit, etc. It should be noted that the specific type of the terminal 11 is not limited in the embodiments of the present application. The network-side device 12 can include an access network device or a core network device. Among them, the access network device can also be referred to as a radio access network (RAN) device, a radio access network function, or a radio access network unit. The access network device can include a base station, a wireless local area network (WLAN) access point (AP), or a wireless fidelity (WiFi) node, etc.Among them, the base station can be referred to as Node B (NB), Evolved Node B (eNB), the next generation Node B (gNB), New Radio Node B (NR Node B), access point, Relay Base Station (RBS), Serving Base Station (SBS), Base Transceiver Station (BTS), radio base station, radio transceiver, Basic Service Set (BSS), Extended Service Set (ESS), home Node B (HNB), home evolved Node B, Transmission Reception Point (TRP), or some other suitable term in the art. As long as the same technical effect is achieved, the base station is not limited to specific technical terms. It should be noted that in the embodiments of this application, only the base station in the NR system is taken as an example for introduction, and the specific type of the base station is not limited.

[0053] The core network device may include, but is not limited to, at least one of the following: core network node, core network function, Mobility Management Entity (MME), Access and Mobility Management Function (AMF), Session Management Function (SMF), User Plane Function (UPF), Policy Control Function (PCF), Policy and Charging Rules Function (PCRF), Edge Application Server Discovery Function (EASDF), Unified Data Management (UDM), Unified Data Repository (UDR), Home Subscriber Server (HSS), Centralized network configuration (CNC), Location Management Function (LMF), Network Repository Function (NRF), Network Exposure Function (NEF), Local NEF (L-NEF), Binding Support Function (BSF), Application Function (AF), etc. It should be noted that in the embodiments of this application, only the core network devices in the NR system are taken as examples for introduction, and the specific types of core network devices are not limited.

[0054] For better understanding, the following explains the related concepts and principles that may be involved in the embodiments of this application.

[0055] Communication perception integration (also known as communication and sensing integration):

[0056] Future mobile communication systems, such as B5G systems or 6G systems, will not only have communication capabilities but also sensing capabilities. The sensing capabilities, that is, one or more devices with sensing capabilities, can sense information such as the orientation, distance, and speed of a target object through the transmission and reception of wireless signals, or detect, track, identify, image, etc. a target object, event, or environment. In the future, with the deployment of small base stations with high-frequency band and large bandwidth capabilities such as millimeter waves and terahertz in the 6G network, the sensing resolution will be significantly improved compared to centimeter waves, enabling the 6G network to provide more refined sensing services. A typical sensing function and application scenario are shown in Table 1.

[0057] Table 1

[0058]

[0059] Communication-sensing integration (abbreviated as communication-sensing integration) means that in the same system, through spectrum sharing and hardware sharing, the integration design of communication and sensing functions is realized. While the system is transmitting information, it can sense information such as orientation, distance, and speed, detect, track, and identify a target device or event. The communication system and the sensing system complement each other, achieving an improvement in overall performance and bringing a better service experience.

[0060] The integration of communication and radar belongs to a typical application of communication-sensing integration (communication-sensing fusion). In the past, radar systems and communication systems were strictly separated due to different research objects and focuses of attention, and the two systems were studied independently in most scenarios. In fact, both radar and communication systems are typical ways of information transmission, acquisition, processing, and exchange, and there are many similarities in terms of working principles, system architectures, and frequency bands. The design of the integration of communication and radar has great feasibility, which is mainly reflected in the following aspects: First, both the communication system and the sensing system are based on the electromagnetic wave theory, and use the emission and reception of electromagnetic waves to complete information acquisition and transmission; Second, both the communication system and the sensing system have structures such as antennas, transmitters, receivers, and signal processors, and there is a large overlap in hardware resources; With the development of technology, there is also more and more overlap in their working frequency bands; In addition, there are similarities in key technologies such as signal modulation, reception detection, and waveform design. The integration of communication and radar systems can bring many advantages, such as cost savings, size reduction, power consumption reduction, spectrum efficiency improvement, mutual interference reduction, etc., thus improving the overall performance of the system.

[0061] According to the different sensing signal sending nodes and receiving nodes, it is divided into 6 basic sensing modes, as Figure 1b shown, specifically including:

[0062] (1) Base station echo sensing. In this sensing mode, base station A sends a sensing signal and performs sensing measurements by receiving the echo of the sensing signal.

[0063] (2) Air interface sensing between base stations. At this time, base station B receives the sensing signal sent by base station A and performs sensing measurements.

[0064] (3) Uplink air interface sensing. At this time, base station A receives the sensing signal sent by terminal A and performs sensing measurements.

[0065] (4) Downlink air interface sensing. At this time, terminal B receives the sensing signal sent by base station B and performs sensing measurements.

[0066] (5) Terminal echo sensing. At this time, terminal A sends a sensing signal and performs sensing measurements by receiving the echo of the sensing signal.

[0067] (6) Sidelink sensing between terminals. At this time, terminal B receives the sensing signal sent by terminal A and performs sensing measurements.

[0068] It should be noted that Figure 1b each sensing mode takes a sensing signal sending node and a sensing signal receiving node as examples. In an actual system, according to different sensing use cases and sensing requirements, one or more different sensing modes can be selected, and there can be one or more sending nodes and receiving nodes for each sensing mode. Figure 1b The sensing targets in take people and vehicles as examples, and it is assumed that neither people nor vehicles carry or install signal transceiver devices. The sensing targets in the actual scenario will be more diverse.

[0069] Currently, the sensing results are usually obtained in the following ways.

[0070] 1. Obtain the sensing result by A sending and B receiving

[0071] By node A sending a sensing reference signal, node B receiving the sensing reference signal, and node B obtaining the sensing measurement quantity / sensing result. Among them, node A can be base station 1, node B can be the target UE, or a UE near the target UE, or another base station 2.

[0072] 2. Obtain the sensing result by self-sending and self-receiving

[0073] By node A sending a sensing reference signal, node A receiving the sensing reference signal, and node A obtaining the sensing measurement quantity / sensing result. Among them, node A can be the target UE, or the serving base station of the target UE, or another base station 2.

[0074] 3. The grid node obtains the sensing result through the sensor

[0075] The network node obtains the sensed measurement / sensed result through its own deployment or sensor-like sensing devices in the environment. Sensor-like sensing: A sensing method that performs sensing services through means other than the communication-sensing integrated system. Typical devices include: lidar, millimeter-wave radar, vision sensors (including: monocular vision, binocular vision, infrared sensors), inertial measurement unit (IMU), and various other sensors (rain gauges, thermometers, hygrometers, etc.).

[0076] With the development of AI technology, AI units have also been applied to communication systems. Currently, there are mainly the following use cases:

[0077] 1. AI-based beam prediction

[0078] In the AI-based beam prediction use case discussed in 5G, when monitoring the model, the key performance indicator (KPI) is generally the prediction accuracy of the strongest beam or the prediction error of the beam quality. Among them, the prediction accuracy of the strongest beam is calculated based on the predicted strongest beam identifier and the identifier of the strongest beam corresponding to the true value; the prediction error of the beam quality is calculated based on the predicted beam quality and the beam quality corresponding to the true value.

[0079] 2. AI-based CSI prediction

[0080] In the AI-based CSI prediction use case discussed in 5G, when monitoring the model, the KPI is generally the squared cosine similarity. Among them, the squared cosine similarity number is calculated based on the predicted channel matrix, precoding matrix indicator (PMI), channel eigenvector / eigenvalue, and the corresponding channel matrix, PMI, channel eigenvector / eigenvalue of the true value.

[0081] 3. AI-based positioning

[0082] In the AI-based positioning use case discussed in 5G, when monitoring the model, the KPIs are generally the input data distribution deviation, output data distribution deviation, position estimation error, and TOA estimation error. Among them, the input data distribution deviation is calculated based on the reference input data distribution and the data distribution input to the actual model; the output data distribution deviation is calculated based on the reference output data distribution and the data distribution input to the actual model; the position estimation error is calculated based on the predicted distance relative to the base station and the distance relative to the base station corresponding to the true value; the TOA estimation error is calculated based on the predicted time of arrival and the time of arrival corresponding to the true value.

[0083] Among them, the model input includes measurement results of positioning reference signals based on time-domain channels;

[0084] The model output is the predicted distance relative to the base station or the time of arrival (TOA).

[0085] The ground truth / label is the actually measured distance relative to the base station or the TOA.

[0086] The AI unit described in this application may also be referred to as an AI unit, a machine learning (ML) model, an ML unit, an AI structure, an AI function, an AI feature, a machine learning model, a neural network, a neural network function, a neural network function, etc. Alternatively, the AI unit / AI unit may also refer to a processing unit capable of implementing specific algorithms, formulas, processing flows, capabilities, etc. related to AI. Alternatively, the AI unit / AI unit may be a processing method, algorithm, function, module, or unit for a specific data set. Alternatively, the AI unit / AI unit may be a processing method, algorithm, function, module, or unit running on AI / ML-related hardware such as a Graphics Processing Unit (GPU), a Neural network Processing Unit (NPU), a Tensor Processing Unit (TPU), or an Application Specific Integrated Circuit (ASIC). This application does not make specific limitations in this regard. Optionally, the specific data set includes the input and / or output of the AI unit / AI unit.

[0087] Optionally, the identifier of the AI unit / AI unit may be an AI unit identifier, an AI structure identifier, an AI algorithm identifier, or the identifier of a specific data set associated with the AI unit / AI unit, or the identifier of a specific scenario, environment, channel characteristic, or device related to AI / ML, or the identifier of a function, feature, capability, or module related to AI / ML. This application does not make specific limitations in this regard.

[0088] The signal configuration information involved in this application includes at least one of the following:

[0089] (a) Signal resource identity (ID), used to distinguish different signal resource configurations;

[0090] (b) Signal usage, indicating that the signal is for communication (such as channel measurement, channel estimation, synchronization, carrying data information, etc.), a signal for sensing, or a signal for both communication and sensing. Specifically, it can also be a signal for which sensing service or a signal for which type of sensing service;

[0091] (c) Waveform, such as Orthogonal Frequency Division Multiplexing (OFDM), Single-carrier Frequency-Division Multiple Access (SC-FDMA), Orthogonal Time Frequency Space (OTFS), Chirp, Frequency Modulated Continuous Wave (FMCW), pulse signal, etc.;

[0092] (d) Subcarrier spacing, for example, the subcarrier spacing of an OFDM system is 30 KHz;

[0093] (e) Guard interval, which is the time interval between the moment when the signal ends transmission and the moment when the latest echo signal of the signal is received; this parameter is proportional to the maximum sensing distance; for example, it can be calculated by c / (2R max ), where R max is the maximum sensing distance (belonging to sensing requirement information). For example, for a self-transmitting and self-receiving sensing signal, R max represents the maximum distance from the sensing signal transceiver point to the signal emission point; in some cases, the cyclic prefix (CP) of an OFDM signal can serve as the minimum guard interval; c is the speed of light;

[0094] (f) Starting frequency domain position, that is, the starting frequency point, which can also be the starting Resource Element (RE), Resource Block (RB) index;

[0095] (g) Starting time domain position, that is, the starting time point, which can also be the starting symbol index, time slot index, frame index;

[0096] (h) Ending frequency domain position, that is, the ending frequency point, which can be represented by the ending RE, RB index;

[0097] (i) Ending time domain position, that is, the ending time point, which can be represented by the ending RE, RB index;

[0098] (j) Frequency-domain resource length, i.e., frequency-domain bandwidth, and the frequency-domain bandwidth is inversely proportional to the range resolution. The frequency-domain bandwidth B of each of the first signals satisfies B≥c / (2ΔR), where c is the speed of light and ΔR is the range resolution;

[0099] (k) Time-domain resource length, also known as the burst duration, and the time-domain resource length is inversely proportional to the Doppler resolution.

[0100] (l) Frequency-domain resource interval, which represents the interval between adjacent signal frequency-domain resource units, can be expressed by the number of resource elements (REs) or the number of resource blocks (RBs), or can be expressed by the density value Density. For example, Density = 1 means that there is one RE in each RB for carrying signals. The frequency-domain resource interval is inversely proportional to the maximum unambiguous range / delay. Among them, for an OFDM system, when the subcarriers are continuously mapped, the frequency-domain interval is equal to the subcarrier interval;

[0101] (m) Time-domain resource interval, which is the time interval between two adjacent signal resource units, and the time-domain resource interval is associated with the maximum unambiguous Doppler shift or the maximum unambiguous speed;

[0102] (n) Time-domain resource characteristics, such as periodic transmission, semi-persistent transmission, and aperiodic transmission;

[0103] (o) Signal power, for example, taking a value every 2 dBm from -20 dBm to 23 dBm;

[0104] (p) Sequence information, including sequence type information (ZC sequence, PN sequence, etc.), sequence generation method, sequence length, etc.;

[0105] (q) Signal direction, i.e., the angle information or beam information of signal transmission;

[0106] (r) Quasi co-location (QCL) relationship. For example, the sensing signal includes multiple resources, and each resource is QCL with a synchronization signal block (SSB). QCL includes Type A, B, C, or D.

[0107] (s) Antenna port information, such as the maximum number of antenna ports and the antenna port index.

[0108] (t) Cyclic prefix (CP) information, including CP type (such as normal cyclic prefix (NCP), extended cyclic prefix (ECP), or a newly designed CP dedicated to sensing measurement, etc.), CP length, etc.

[0109] The sensing requirements involved in this application include at least one of the following:

[0110] 1. Sensing service or sensing service type. The sensing service can be, for example, detecting the presence of a target, positioning, speed detection, distance detection, angle detection, acceleration detection, material analysis, component analysis, shape detection, category classification, radar cross section (RCS) detection, polarization scattering characteristic detection, fall detection, intrusion detection, quantity statistics, indoor positioning, gesture recognition, lip reading, gait recognition, expression recognition, face recognition, breathing monitoring, heart rate monitoring, pulse monitoring, humidity / brightness / temperature / atmospheric pressure monitoring, air quality monitoring, weather condition monitoring, environment reconstruction, terrain and landform, building / vegetation distribution detection, pedestrian or vehicle flow detection, crowd density, vehicle density detection, etc.; The sensing service type can classify multiple different sensing services according to certain characteristics. For example, it can be classified into detection-type sensing services (such as intrusion detection, fall detection), parameter estimation-type sensing services (distance, angle, speed calculation), recognition-type sensing services (action recognition, identity recognition), etc. according to function, or it can be, for example, target detection and tracking-type sensing services (including the presence of a target, target ranging / ranging / angle measurement / localization / trajectory tracking), environmental monitoring-type sensing services (including rainfall detection, flood monitoring), action detection-type sensing services (including gesture / action recognition, breathing / heartbeat detection, fall detection), etc. It can also be classified according to the sensing range (close-range sensing, medium-range sensing, long-range sensing), according to the sensing fineness (coarse-grained sensing, fine-grained sensing, etc.), according to power consumption / energy consumption, according to resource occupancy, etc.

[0111] 2. Sensing target area: It refers to the area where the sensing object may be located, or the area where imaging or environment reconstruction needs to be performed.

[0112] 3. Sensing object type: Classify the sensing objects according to the possible motion characteristics of the sensing objects. Each sensing object type contains information such as the motion speed, motion acceleration, and typical RCS of typical sensing objects.

[0113] 4. Sensing quality of service (QoS): Performance indicators for sensing the sensing target area or sensing objects, including at least one of the following:

[0114] a) Sensing resolution (which can be further divided into: ranging resolution, angle measurement resolution, speed measurement resolution, imaging resolution), etc.;

[0115] b) Sensing accuracy (which can be further divided into: ranging accuracy, angle measurement accuracy, speed measurement accuracy, positioning accuracy, etc.);

[0116] c) Perception range (which can be further divided into: ranging range, speed measurement range, angle measurement range, imaging range, etc.);

[0117] d) Perception delay (the time interval from the sending of the perception signal to the obtaining of the perception result, or the time interval from the initiation of the perception requirement to the obtaining of the perception result);

[0118] e) Perception update rate (the time interval between two adjacent executions of perception and the obtaining of the perception result);

[0119] f) Detection probability (the probability of being correctly detected when the perception object exists);

[0120] g) False alarm probability (the probability of erroneously detecting the perception target when the perception object does not exist);

[0121] h) The maximum number of perceivable targets.

[0122] The types of perception measurement quantities related to integrated communication and sensing involved in this application include at least one of the following:

[0123] First-level measurement quantities (received signal / raw channel information), including: complex results of received signal / channel response, amplitude / phase, I / Q channels and their operation results (operations include addition, subtraction, multiplication, division, matrix addition, subtraction, multiplication, matrix transpose, trigonometric relation operations, square root operations, power operations, etc., and threshold detection results, maximum / minimum extraction results, etc. of the above operation results; operations also include Fast Fourier Transform (FFT) / Inverse Fast Fourier Transform (IFFT), Discrete Fourier Transform (DFT) / Inverse Discrete Fourier Transform (IDFT), 2D-FFT, 3D-FFT, matched filtering, autocorrelation operation, wavelet transform, digital filtering, etc., and threshold detection results, maximum / minimum extraction results, etc. of the above operation results).

[0124] Second-level measurement quantities (basic measurement quantities), including: delay, Doppler, angle, intensity, and their multi-dimensional combined representations. For example, it can be a delay value, a Doppler value, or a delay power spectrum, a Doppler power spectrum, a speed power spectrum, an angle power spectrum, a delay-Doppler spectrum, a delay-angle spectrum, a Doppler-angle spectrum, a delay-Doppler-angle spectrum, etc.

[0125] Third-level measurement quantities (perception results / perception intermediate results), including:

[0126] Parameters for characterizing basic attributes / status, including: distance, speed, orientation, spatial position, acceleration;

[0127] Parameters for characterizing advanced attributes / status, including: whether the target exists, trajectory, action, expression, vital signs, quantity, imaging result, weather, air quality, shape, material, composition.

[0128] Parameters for characterizing the environmental reconstruction result, such as trajectory, etc.

[0129] Types of perception measurement quantities sensed by different types of sensors:

[0130] Perception measurement quantities related to lidar, including at least one of the following:

[0131] - Lidar point cloud data, and each point in the lidar point cloud data includes: X / Y / Z position information, and, additional information;

[0132] - The angle and distance of the target obtained from the lidar point cloud data;

[0133] - Visual features of the target identified from the lidar point cloud data, such as: people, vehicles, etc.;

[0134] - The number of targets identified from the lidar point cloud data.

[0135] The additional information in the described lidar point cloud data includes at least one of the following:

[0136] - Intensity: The echo intensity of the laser pulse generating the lidar point;

[0137] - Echo number: The echo number is the total number of echoes of a given pulse;

[0138] - Point classification: Each post-processed lidar point can have a classification defining the type of object reflecting the lidar pulse, and the lidar points can be divided into many categories, such as: ground, bare surface, top of the tree canopy, and water area, etc.;

[0139] - RGB: The RGB band can be used as an attribute of the lidar data, and this attribute usually comes from the image collected during the lidar measurement.

[0140] - GPS time: The GPS timestamp when the laser point is emitted from the aircraft.

[0141] - Scanning angle:

[0142] - Scanning direction: The traveling direction of the laser scanning mirror, with value 1 representing the positive scanning direction and value 0 representing the negative scanning direction.

[0143] Visual perception measurement quantities, including at least one of the following:

[0144] - Visual image;

[0145] - Luminance of image pixels;

[0146] - RGB values of image pixels;

[0147] - Visual features of the target recognized from the image, such as: people, vehicles, etc.

[0148] - Angle and distance of the target recognized from the image (especially for binocular vision);

[0149] - Number of targets recognized from the image.

[0150] Radar-related perception measurement quantities, including at least one of the following:

[0151] - Radar point cloud, each point in the point cloud includes at least one of: distance / speed / azimuth angle / elevation angle, or at least one of X / Y / Z / speed;

[0152] - Distance, speed, and angle of the recognized target;

[0153] - Radar imaging;

[0154] - Number of targets.

[0155] Inertial measurement unit-related perception measurement quantities, including at least one of the following:

[0156] - Acceleration: at least one of the three directions of X / Y / Z;

[0157] - Speed: at least one of the three directions of X / Y / Z;

[0158] - Angular velocity: at least one of the three axes of X / Y / Z.

[0159] Perception measurement quantities of a position sensor such as GNSS, including the position information of the device and the relative distance / angle information between the device and a specific object.

[0160] Other perception measurement quantities, including at least one of the following: whether the target exists, trajectory, action, expression, vital signs, quantity, imaging result, weather, air quality, shape, material, composition, etc.

[0161] Next, in conjunction with the accompanying drawings, the monitoring method for the inference performance of the AI unit provided by the embodiments of the present application will be described in detail through some embodiments and their application scenarios.

[0162] Please refer to Figure 2 , Figure 2It is one of the flowcharts of a method for monitoring the inference performance of an AI unit provided by an embodiment of the present application; the method is applied to a first device, and the first device may be a terminal or a network-side device. As Figure 2 shown, the method includes the following steps:

[0163] Step 201, the first device receives a first configuration sent by a second device.

[0164] Among them, the first configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit.

[0165] It can be understood that the second device sends a first configuration to the first device, and the first configuration is used to indicate the collection of monitoring samples for monitoring the inference performance of the AI unit. Furthermore, the first device can then collect the monitoring samples based on the indication of the first configuration.

[0166] Step 202, the first device obtains perception measurement-related data and communication measurement-related data, and generates first information according to the first configuration and based on the perception measurement-related data and the communication measurement-related data.

[0167] Among them, the perception measurement-related data includes but is not limited to perception reference signals, perception measurement quantities, perception performance, perception results, etc. The communication measurement-related data includes but is not limited to channel state information (CSI), beam measurement information, positioning reference signal measurement information, phase measurement information, etc. Specifically, for example, the reference signal received power (RSRP) of a beam, the reference signal received quality (RSRQ) of a beam, the signal-to-noise and interference ratio (SINR) of a beam, the RSRP of a cell channel, the RSRQ of a cell channel, the SINR of a cell channel, the received signal strength indication (RSSI) of a cell channel, the impulse response of a cell channel, the precoding matrix indicator (PMI), the rank indicator (RI), the channel quality indicator (CQI), the beam identifier, the subband identifier.

[0168] Exemplarily, taking the perception measurement related data including perception measurement quantities and the communication measurement related data including CSI measurement results as an example, the first device obtains the perception measurement quantities and the CSI measurement results, and generates first information based on the perception measurement quantities and the CSI measurement results according to the indication of the first configuration. Among them, the first information includes a first data set, and the first data set includes monitoring samples for monitoring the inference performance of the AI unit. For example, the first device can use the perception measurement quantities and the CSI measurement results as the input of the AI unit, and obtain the output of the AI unit. The input and output of the AI unit can be used as monitoring samples for monitoring the inference performance of the AI unit (for example, using the input distribution of the AI unit or the output distribution of the AI unit as model monitoring parameter items), that is, the first data set includes the input and output of the AI unit. Of course, the first data set may also include other possible parameters or contents.

[0169] Optionally, the monitoring samples include at least one of the following:

[0170] The input of the AI unit;

[0171] The output of the AI unit;

[0172] True value (also called label), which is used to indicate the actual measured value of the parameter corresponding to the output of the AI unit. For example, if the output of the AI unit is the predicted CSI, then the true value can be the actually measured CSI.

[0173] Exemplarily, the monitoring samples in the first data set can only include the input of the AI unit (for example, using the input distribution of the AI unit as a model monitoring parameter item), or the monitoring samples can also include the output and true value of the AI unit (for example, monitoring the inference accuracy or error of the AI unit using the output and true value of the AI unit), or the monitoring samples can also include the output of the AI unit (for example, using the output distribution of the AI unit as a model monitoring parameter item). Of course, the content included in the monitoring samples can also be other possible situations, which will not be specifically elaborated here.

[0174] Optionally, the input of the AI unit includes the perception measurement related data, and the perception measurement related data includes perception measurement quantities and at least one of the following:

[0175] Indications of perception measurement quantities, such as parameter items for indicating a predefined perception measurement quantity; the parameter items include at least one of radial Doppler, time delay, power, and angle;

[0176] Timestamps of perception measurement quantities, such as the measurement time of perception measurement quantities or the reception time of perception measurement quantities;

[0177] Information of the sending device of the sensed measurement quantity, such as the identity (ID) of the sending device of the sensed measurement quantity, the location of the sending device, the orientation of the sending device, the motion information of the sending device, etc.;

[0178] Information of the receiving device of the sensed measurement quantity, such as the ID of the receiving device of the sensed measurement quantity, the location of the receiving device, the orientation of the receiving device, the motion information of the receiving device, etc.;

[0179] Coordinate information of the sensed measurement quantity, such as whether the result of the sensed measurement quantity is based on the local coordinate system or the global coordinate system, the description information of the local coordinate system, such as the rotation angles relative to the global coordinate system: α (bearing angle), β (dip angle), and γ (tilt angle), etc.;

[0180] Information for indicating the performance indicators of the sensed measurement quantity, such as the resolution of the sensed measurement quantity (which can be time delay resolution, ranging resolution, angle measurement resolution, Doppler resolution, velocity measurement resolution, imaging resolution, etc., that is, the granularity of the reported measurement quantity value), signal-to-noise and interference ratio (SINR), sensed SINR (the ratio of the power of the path associated with the sensed target to the power of noise and interference), etc.;

[0181] Information for indicating the source of the sensed measurement quantity, such as whether the sensed measurement quantity is from the sensing reference signal or from sensor sensing, etc.;

[0182] Information for indicating the category of the sensed measurement quantity;

[0183] Information for indicating the sensing mode of the sensed measurement quantity, such as the self-transmitting and self-receiving sensing mode or the A-transmitting and B-receiving sensing mode, etc.;

[0184] Configuration information of the first signal, such as the configuration information of the sensing reference signal;

[0185] Information of the sending device of the first signal, such as the ID of the sending device of the sensing reference signal, the location of the sending device, the orientation of the sending device, the motion information of the sending device, etc.;

[0186] Information of the receiving device of the first signal, such as the ID of the receiving device of the sensing reference signal, the location of the receiving device, the orientation of the receiving device, the motion information of the receiving device, etc.;

[0187] Among them, the first signal is a signal for sensing, such as a sensing reference signal. For example, the first signal is a dedicated signal for sensing, such as a CSI reference signal (CSI-RS), a tracking reference signal (TRS), a sounding reference signal (SRS), or a synchronization signal, etc.

[0188] In the embodiments of the present application, when the monitoring samples of the first data set include the input of the AI unit, the input of the AI unit may include a sensing measurement quantity and at least one of the above items related to the sensing measurement quantity. By using the sensing measurement quantity and the information related thereto as the input of the AI unit and as the monitoring samples of the inference performance of the AI unit, it is more helpful to improve the inference accuracy of the AI unit. At this time, compared with only using communication measurement-related data to monitor the inference performance of the AI unit, the solution provided by the present application can better implement the monitoring steps of the inference performance of the AI unit.

[0189] 203. The first device sends the first information to the second device.

[0190] Among them, the first information includes at least one of a first data set and a first metric. The first data set includes the monitoring samples, and the first metric includes an AI unit inference performance monitoring metric.

[0191] Understandably, after the first device obtains the sensing measurement data and communication measurement related data, according to the received first configuration, it generates first information based on the sensing measurement related data and communication measurement related data, and sends the first information to the second device. Wherein, the first information includes at least one of a first data set and a first metric. For example, the first information only includes the first data set, and then the second device can calculate the AI unit inference performance monitoring metric based on the received first data set, that is, calculate the first metric. Or, the first information only includes the first metric, that is, the first metric is calculated by the first device based on the first data set. In this case, the second device can directly obtain the AI unit inference performance monitoring metric. Or, the first information includes the first data set and the first metric, that is, the first device calculates the first metric based on the first data set, and sends both the first metric and the first data set used to calculate the first metric to the second device. The second device can calculate a second metric based on the first data set. The second metric may be the same as or different from the first metric. For example, the first metric is the inference accuracy calculated by the first device, and the second metric is the prediction error calculated by the second device; or the second device can also directly adopt the first metric sent by the first device. Wherein, the first metric is used to characterize the AI unit inference performance or the monitoring result of the AI unit. For example, whether the AI unit inference performance is good or bad, which helps the second device make decisions on subsequent operations of the AI unit according to the first metric, such as whether to deactivate the AI unit, or switch to another AI unit, or trigger model training / fine-tuning to adjust the parameters or structure of the AI unit, etc.

[0192] In an embodiment of the present application, the first device receives the first configuration sent by the second device. The first configuration is used to indicate the collection of monitoring samples for monitoring the AI unit inference performance. Then, after the first device obtains the sensing measurement related data and communication measurement related data, it can, according to the indication of the first configuration, combine the sensing measurement related data and communication measurement related data to generate first information, and send the first information to the second device. Wherein, the first information includes at least one of a first data set and a first metric. The first data set includes the monitoring samples, and the first metric includes the AI unit inference performance monitoring metric. And the first information is generated based on the sensing measurement related data and communication measurement related data. Thus, it enables the first device or the second device to integrate the sensing measurement related data in the monitoring of the AI unit inference performance. Compared with only using the communication measurement related data to monitor the AI unit inference performance, the solution provided in the present application can better realize the monitoring of the AI unit inference performance.

[0193] Optionally, in the embodiments of the present application, the first configuration includes at least one of the following:

[0194] AI unit identifier;

[0195] AI unit monitoring purpose identifier;

[0196] A first number, which is used to indicate the number of groups of perception measurement quantities to be monitored;

[0197] A first interval, which is related to the transmission interval of a first signal, and the first signal is a signal used for perception (such as a perception reference signal). For example, the first interval is used to indicate the transmission period of the actually transmitted perception reference signal;

[0198] A second interval, which is the baseline of the default valid interval or the valid interval to be monitored. For example, the second interval is related to the default transmission interval of the first signal corresponding to the perception measurement quantity to be monitored by the AI unit, where the perception measurement quantity to be monitored by the AI unit may refer to using the perception measurement quantity as the input of the AI unit;

[0199] A third interval, which is used to indicate the usage interval of the perception measurement quantity to be monitored. For example, the third interval is used to indicate the usage period of the perception measurement quantity to be monitored. The perception measurement quantity to be monitored may refer to using the perception measurement quantity as the input of the AI unit to obtain a first data set, and monitoring the inference performance monitoring index of the AI unit based on the first data set;

[0200] A fourth interval, which is used to indicate the usage or transmission interval of the perception measurement quantity in the group of perception measurement quantities to be monitored. For example, the fourth interval is used to indicate the transmission period of the perception measurement quantity in the group of perception measurement quantities to be monitored. The group of perception measurement quantities may refer to multiple different perception measurement quantities or sets of perception measurement quantities;

[0201] A first coefficient, which is the ratio of the second interval to the first interval;

[0202] A second coefficient, which is the ratio of the first interval to the second interval;

[0203] A third coefficient, which is the ratio of the third interval to the first interval;

[0204] A fourth coefficient, which is the ratio of the first interval to the third interval;

[0205] A first duration, where the first duration is related to a reference time and a relative duration. The reference time is determined by the time when a sensed measurement quantity is obtained, and the relative duration is a duration relative to the time when the sensed measurement quantity is obtained. For example, the reference time may be the time when the sensed measurement quantity is obtained, or it may also be the time when the sensed measurement quantity is obtained plus the second interval.

[0206] A first indication, where the first indication is used to indicate the collection method of the monitoring samples. For example, the first indication is used to indicate different collection methods such as whether the monitoring samples are collected based on a period or based on a fixed number of samples.

[0207] An indication of the reporting method of the monitoring samples. For example, the reporting method may include: reporting all the monitoring samples associated with a certain sensed measurement quantity together, and the monitoring samples reported at one time are associated with the same sensed measurement quantity or the same sensed measurement quantity control group, and reporting according to a fixed number of samples (the monitoring samples may be associated with different sensed measurement quantities).

[0208] An indication of the number of the monitoring samples.

[0209] A first threshold, where the first threshold is used to indicate a relevant threshold for determining the effective duration of the sensed measurement quantity.

[0210] An indication of the first metric, such as whether to report the first metric.

[0211] Information for indicating the reason for the second device to trigger the AI unit inference performance monitoring. Among them, the reasons for the second device to trigger the monitoring may include at least one of the following: the services of the serving cell and neighboring cells meet a preset relationship (such as the number of consecutive measurements meets a preset number, the measurement period meets a preset period, the service difference meets a preset difference, etc.), the second device receives a third indication message sent by a third device (the third indication message may indicate a change in the number of sensed measurement quantities or a change in the sensing performance), the second device finds that the number of sensed measurement quantities it can obtain has changed or the sensing performance has changed, the device configuration of the second device has changed, and the environment monitored by the second device has changed.

[0212] It should be noted that the above interval may be a time-domain period, or a frequency-domain density, or a beam interval or density, or an angular-domain interval or density, which is not limited here.

[0213] For example, at the initial stage of AI model inference, a sensing reference signal transmission period (second interval) is specified by network configuration or protocol agreement. However, over time, it is necessary to monitor whether this default period (second interval) is appropriate and whether it is necessary to adjust the second interval. Therefore, the AI unit can also use the sensing measurement quantity as the model input of the AI unit based on another period (third interval) to observe the inference performance of the AI unit under different usage periods of the sensing measurement quantity.

[0214] In the embodiments of the present application, the first device receives the first configuration sent by the second device, and the first configuration includes at least one of the above contents. Furthermore, the first device can more clearly know how to collect and report the monitoring samples based on the contents included in the first configuration, which is also more helpful for the first device to generate the first information based on the first configuration.

[0215] Optionally, the collection method of the monitoring samples includes at least one of the following:

[0216] Period-based sample collection, for example, collecting monitoring samples every preset period;

[0217] Fixed-sample-number-based collection, for example, collecting monitoring samples every time a fixed number of samples is reached;

[0218] Sliding-window-based sample collection, for example, collecting monitoring samples within the sliding window range based on the sliding of the sliding window;

[0219] Immediate sample collection, that is, collecting monitoring samples in real time, for example, collecting monitoring samples every time a sensing measurement quantity is obtained (which may include at least one monitoring sample);

[0220] The specific determination of the sample collection range can be through:

[0221] Method 1. Determined based on the first duration. The first duration is determined by the reference time and the relative duration. At this time, the reference time is the time when the sensing measurement quantity is obtained. The relative duration includes the first relative duration and the second relative duration. The first relative duration is the end time of monitoring sample collection relative to the reference time, and the second relative duration is the time to start collecting monitoring samples in advance relative to the reference time. For example, the first relative duration is 100 ms, the second relative duration is 0, or not configured, and the default is 0. Then, starting from the moment t when a sensing measurement quantity is obtained, monitoring samples are collected within t + 100 ms.

[0222] Method 2. Determined based on the first quantity. For example, if the first quantity is configured as 5, then starting from the moment t when a sensing measurement quantity is obtained, 5 samples are continuously collected.

[0223] Near-term sample collection, where the near-term sample collection is used to indicate obtaining the monitoring sample for the resources associated with the third interval, and the third interval is used to indicate the transmission interval of the sensed measurement quantity to be monitored. The near-term sample collection is also used to indicate collecting the monitoring sample within the transmission interval of the sensed measurement quantity to be monitored. For example, if the usage period of the sensed measurement quantity indicated by the third interval is 1 s, then a monitoring sample (including at least one monitoring sample) can be collected at the time of obtaining the sensed measurement quantity plus 1 s. The specific determination of the sample collection range can be through:

[0224] Method 1. Determine based on the third interval. For example, at the moment of the time of obtaining the sensed measurement quantity + 1 s, a sample collection is performed;

[0225] Method 2. Determine based on the third interval and the first duration. The first duration is determined by the reference time and the relative duration. At this time, the reference time is the time of the sensed measurement quantity + the third interval. The relative duration can include the first relative duration and the second relative duration. The first relative duration is the end time of the monitoring sample collection relative to the reference time, and the second relative duration is the time to start collecting the monitoring sample in advance relative to the reference time. For example, if the first relative duration is 50 ms and the second relative duration is 100 ms, then the monitoring sample collection is performed within the time range of (the time of obtaining the sensed measurement quantity + 1 s - 100 ms, the time of obtaining the sensed measurement quantity + 1 s + 50 ms). Here, -100 and +50 ms are the relative duration indications in the first duration indication, and the time of obtaining the sensed measurement quantity + 1 s is the reference time indication in the first duration.

[0226] Method 3: Determine based on the third interval and the first quantity. For example, configure the first quantity to be 5, then start continuously collecting 5 samples at the time of obtaining the sensed measurement quantity + 1 s. Or, assume the time to collect one sample is t1, then start continuously collecting 5 samples from the time of obtaining the sensed measurement quantity + 1 s to 5 * t1 (corresponding to Figure 3e (c)).

[0227] In the embodiments of this application, the first configuration may include a first indication for indicating the collection method of the monitoring sample. The collection method of the monitoring sample includes at least one of the above, so that the collection method of the monitoring sample by the first device is more flexible, facilitating monitoring different intervals of the sensed measurement quantity in different ways.

[0228] Optionally, the monitoring purposes of the AI unit include at least one of the following:

[0229] AI unit performance;

[0230] Timeliness of the sensed measurement quantity;

[0231] Validity of the perceived measurement quantity;

[0232] Comparison of different configurations of the perceived measurement quantity.

[0233] In an embodiment of the present application, the monitoring sample includes the input of the AI unit. The input of the AI unit may include the perceived measurement quantity, that is, the first device can use the perceived measurement quantity as part of the monitoring sample to monitor the inference performance of the AI unit. Furthermore, the monitoring purpose of the AI unit may include monitoring the timeliness and validity of the perceived measurement quantity, as well as comparing different configurations of the perceived measurement quantity. Thus, the first device or the second device can know the influence of the perceived measurement quantity on the monitoring of the AI model inference performance according to the calculated first index, which is more helpful for making decisions on subsequent operations of the AI unit or for making decisions on configuration adjustments related to the perceived measurement quantity.

[0234] Optionally, in an embodiment of the present application, the first information further includes at least one of the following:

[0235] The identifier of the AI unit;

[0236] The first number, which is used to indicate the number of control groups of the perceived measurement quantity to be monitored (including or not including the group corresponding to the second interval);

[0237] The first sample set, which includes the monitoring samples of the control groups of the perceived measurement quantity to be monitored (including or not including the second sample set);

[0238] The second sample set, which includes the monitoring samples corresponding to the second interval. The second interval is the default valid interval or the baseline of the valid interval to be monitored. For example, the second interval is related to the default transmission interval of the first signal corresponding to the perceived measurement quantity to be monitored by the AI unit. The first signal is the signal used for perception, such as the perception reference signal. The second sample set may be the monitoring samples within the default transmission interval of the perception reference signal corresponding to the perceived measurement quantity to be monitored by the AI unit;

[0239] The results of the control groups of the perceived measurement quantity to be monitored (including the group corresponding to the second interval);

[0240] The effective duration of the perceived measurement quantity;

[0241] The fifth coefficient, which is related to the second interval and the effective duration of the perceived measurement quantity. For example, the fifth coefficient is the quotient of the effective duration of the perceived measurement quantity determined by monitoring and the second interval;

[0242] The description information of the first data set, that is, the information used to describe the first data set and the monitored samples included therein, such as the identifier of the monitored sample, the number of monitored samples, etc.

[0243] In the embodiments of the present application, the above content also includes at least one item in the first information. After receiving the first information, the second device can more clearly know the content related to the monitored sample and the sensed measurement quantity according to the content carried in the first information, thereby being more helpful for monitoring the inference performance of the AI unit.

[0244] Optionally, the description information of the first data set includes at least one of the following:

[0245] The identifier of the AI unit;

[0246] The identifier of the monitored sample;

[0247] The number of monitored samples;

[0248] The number of types of monitored samples;

[0249] A second indication, the second indication is used to indicate the sensed measurement quantity known to the second device, and the sensed measurement quantity known to the second device can be the sensed measurement quantity that the first device has sent to the second device, or the sensed measurement quantity that the second device has stored, such as the sensed measurement quantity obtained from other devices or the sensed measurement quantity obtained based on sensor sensing;

[0250] A third indication, the third indication is used to indicate the valid time of the monitored sensed measurement quantity;

[0251] The description information of the sensed measurement quantity corresponding to each type of monitored sample;

[0252] The proportion of valid sensed measurement quantities in each type of monitored sample;

[0253] The valid time of the sensed measurement quantity set;

[0254] A fourth indication, the fourth indication is used to indicate the monitored sample associated with the sensed measurement quantity carried in the first data set;

[0255] A fifth indication, the fifth indication is used to indicate the number or proportion of monitored samples associated with the sensed measurement quantity carried in the first data set.

[0256] Optionally, the description information of the sensed measurement quantity includes at least one of the following:

[0257] The parameter item of the sensed measurement quantity;

[0258] The number of parameter items of the sensed measurement quantity;

[0259] Processing level of the sensed measurement quantity;

[0260] Information indicating the sending device of the sensed measurement quantity, such as the sending device ID, the sending device location, the sending device orientation, the sending device motion information, etc.;

[0261] Information indicating the receiving device of the sensed measurement quantity, such as the receiving device ID, the receiving device location, the receiving device orientation, the receiving device motion information, etc.;

[0262] Source information of the sensed measurement quantity, such as whether the sensed measurement quantity is from a sensed reference signal or sensor sensing;

[0263] Sensed measurement quantity link identification information, used to distinguish which sensing link or which sensing mode the sensing measurement result comes from, such as a monostatic sensing mode or a bistatic sensing mode, etc.;

[0264] Signal configuration identification, the signal configuration identification is used to indicate the signal corresponding to the sensed measurement quantity;

[0265] Sensing service information, such as the sensing service ID, the sensing service type ID, etc.;

[0266] Data subscription identification;

[0267] Information indicating the use of the sensed measurement quantity, such as for communication, sensing, communication and sensing, AI inference, AI unit training, etc.;

[0268] Information of the device corresponding to the sensed measurement quantity, such as the device ID, the device location, the device orientation, the device motion information, etc.;

[0269] Coordinate information of the sensed measurement quantity, used to illustrate whether the sensed measurement quantity is the result based on the local coordinate system or the global coordinate system, and the description information in the local coordinate system, such as the rotation angles relative to the global coordinate system: α (bearing angle), β (down tilt angle), and γ (tilt angle), etc.;

[0270] Performance index information corresponding to the sensed measurement quantity, such as resolution (which can be time delay resolution, ranging resolution, angle measurement resolution, Doppler resolution, speed measurement resolution, imaging resolution, etc., that is, the granularity of the reported measurement quantity value), SINR, sensing SINR (the ratio of the power of the path associated with the sensing target to the power of noise and interference), etc.;

[0271] In the embodiment of the present application, the first information sent by the first device to the second device includes the first data set, the description information of the first data set, and the description information of the sensed measurement quantity. Furthermore, the second device can better learn the information content related to the monitored samples and the sensed measurement quantities associated with the monitored samples in the first data set based on the description information of the first data set and the description information of the sensed measurement quantity. Thus, the second device can better monitor the inference performance of the AI unit based on this information content, thereby avoiding the impact caused by the deterioration of the AI unit inference on the network.

[0272] Optionally, in the embodiment of the present application, the first device obtaining the relevant data of the sensed measurement quantity may include any one of the following:

[0273] The first device receives the sensed measurement-related data sent by the second device;

[0274] The first device receives the first signal sent by the second device or the third device and obtains the sensed measurement-related data according to the first signal;

[0275] The first device receives the sensed measurement-related data sent by the third device, where the sensed measurement-related data is obtained by the third device through sensor device measurement or based on the second signal measurement, or is the sensed measurement-related data received from other sensing devices;

[0276] Among them, the first signal or the second signal is a signal used for sensing.

[0277] Table 2

[0278]

[0279] Exemplarily, please refer to Table 2 above. In the embodiments of the present application, the first device may also be referred to as a data set collection device, that is, collecting or obtaining the first data set. For example, the first device may be a target UE, a serving base station, etc.; the functions of the first device include at least one of the following: receiving a first configuration, sending a first message. The second device is a control device for monitoring the AI unit. Optionally, it may also be referred to as a metric calculation device, that is, calculating the first metric, that is, the inference performance monitoring metric of the AI unit. For example, the second device may be a serving base station, a core network element (such as an AMF, an LMF), etc.; the functions of the second device include at least one of the following: sending a first configuration, receiving a first message, calculating a monitoring metric, sending a sensing measurement quantity. The third device may be a device for sending sensing measurement-related data or a first signal. For example, the third device may be a serving base station, a neighboring base station, a UE near the target UE, etc. For better understanding, the following takes the sensing measurement-related data including sensing measurement quantities, the first device as the target UE, the second device as the serving base station, and the third device as the neighboring base station or a UE near the target UE as an example to illustrate the above three cases.

[0280] For example, in one implementation, the target UE (the first device) may receive the sensing measurement quantity sent by the serving base station (the third device). The sensing measurement quantity may be obtained by the serving base station through sensing by a sensor device, or may also be obtained according to a sensing reference signal sent by other devices (such as other UEs). Then, the target UE generates the first message based on the sensing measurement quantity and the communication measurement-related data, and sends the first message to the serving base station (the second device). At this time, the second device and the third device are co-located in the serving base station.

[0281] Alternatively, in another implementation, the target UE (the first device) receives the sensing reference signal sent by the serving base station (the third device) or the neighboring base station (the third device), obtains the sensing measurement quantity according to the sensing reference signal, and then generates the first message based on the sensing measurement quantity and the communication measurement-related data, and sends the first message to the serving base station (the second device). At this time, the second device and the third device are co-located in the serving base station.

[0282] Alternatively, in yet another implementation, the target UE (the first device) may receive the sensing measurement quantity sent by a nearby UE (the second UE, which is the third device at this time). The sensing measurement quantity may be obtained by the second UE according to the sensing reference signal sent by the serving base station (the third device). Further, the target UE generates the first message based on the sensing measurement quantity and the communication measurement-related data, and sends the first message to the serving base station (the second device).

[0283] It should be noted that the specific interaction processes between the above different devices will be described in detail in the subsequent embodiments, and will not be listed in detail here. In the embodiments of the present application, the first device can obtain perception measurement quantities in different ways, so that the solution provided by the embodiments of the present application can be applicable to different scenarios based on the differences of the first device.

[0284] Optionally, when the first device receives a first signal sent by a third device, the method further includes any one of the following:

[0285] The second device determines the first configuration based on the monitoring requirements and sends the first configuration to the third device;

[0286] The second device receives a third configuration sent by the third device. The third configuration may be the same as or different from the first configuration. For example, the third configuration may include some or all of the content of the first configuration. The second device may determine the first configuration based on the third configuration;

[0287] The second device receives a second configuration sent by the third device and sends feedback information to the third device. The feedback information is used to indicate the difference between the first configuration and the second configuration, or to indicate the desired first configuration.

[0288] Exemplarily, the first device is a target UE, the second device is a serving base station, and the third device is a neighboring base station. When the target UE receives a perception reference signal sent by the neighboring base station and obtains perception measurement quantities according to the perception reference signal, the first configuration can also be interacted or negotiated between the serving base station and the neighboring base station. For example, the serving base station determines the first configuration according to the monitoring requirements and sends the first configuration to the neighboring base station, and may further receive feedback from the neighboring base station on the first configuration, such as consent or the negotiated configuration; or, the third device determines the third configuration (the same as or different from the first configuration) by itself and then sends the third configuration to the serving base station; or, the serving base station receives a second configuration sent by the neighboring base station. The serving base station may determine the first configuration according to the second configuration and send feedback information to the neighboring base station based on the monitoring requirements. The feedback information is used to indicate the difference between the second configuration and the first configuration, or to indicate the desired first configuration of the serving base station to negotiate with the neighboring base station to obtain the desired first configuration. In the embodiments of the present application, the determination method of the first configuration is made more flexible.

[0289] In the embodiments of the present application, the first device sending the first information to the second device includes:

[0290] In the case of a first event being triggered, the first device sends the first information to the second device, and the first information includes an identifier of the first event; wherein, the first event includes at least one of the following:

[0291] The first metric meets a preset threshold;

[0292] The first device receives second indication information sent by a third device, and the second indication information is used to indicate that the number of sensed measurement quantities changes or the sensing performance changes;

[0293] The moving speed of the first device changes;

[0294] The first device can obtain that the number of sensed measurement quantities changes or the sensing performance changes.

[0295] Among them, the first event may be a predefined event. In the embodiments of the present application, in the case of a first event being triggered, the first device sends the first information to the second device, that is, reports at least one of the first data set and the first metric to the second device, so that the first device reports the first information only when the first event is triggered. Moreover, the first information carries the identifier of the first event, so that the second device can know that the first device reports the first information this time due to the trigger of the first event, that is, can know the reason for reporting the first information.

[0296] For better understanding, the AI unit inference performance monitoring method provided by the present application is explained below through specific embodiments.

[0297] Embodiment 1

[0298] As Figure 3a shown, the UE is the first device, and the serving base station is the second device (or co-located with the third device), and the method includes the following steps:

[0299] Step 11a. The serving base station sends a first configuration to the UE, and the first configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit;

[0300] Step 12a. The serving base station sends a first signal (such as a sensing reference signal) to the UE;

[0301] Step 13a. The UE receives the first signal sent by the serving base station, obtains the data related to sensing measurement (such as sensing measurement quantity), and generates the first information including at least one of the first data set and the first metric by combining the data related to sensing measurement and the data related to communication measurement (such as CSI measurement result). The first data set includes the monitoring samples for monitoring the inference performance of the AI unit, and one monitoring sample may include at least one of the AI unit input, the AI unit output, and the ground truth (or label); the first metric includes the monitoring metric for the inference performance of the AI unit.

[0302] Step 14a. The UE sends the first information to the serving base station.

[0303] Step 15a. The serving base station determines the monitoring metric for the inference performance of the AI unit (the first metric) based on the first information. Further, the serving base station may make relevant decisions on the subsequent operations of the AI unit based on the first metric, such as deactivating the AI unit, switching to another AI unit, or triggering model training / fine-tuning to adjust the parameters and / or structure of the AI unit, etc.

[0304] Wherein, the content included in the first configuration may refer to the description in the previous method embodiments and will not be elaborated here.

[0305] Optionally, in this embodiment, in another alternative scenario, the UE may also receive the first signal sent by the second base station. As Figure 3b shown, the UE is the first device, the serving base station is the second device, and the second base station is the third device. The method includes the following processes:

[0306] Step 10b. The second base station sends the third configuration to the serving base station, and the serving base station may adjust and determine the first configuration based on the third configuration.

[0307] Step 11b. The serving base station sends the first configuration to the UE, and the first configuration is used to indicate the collection of monitoring samples, and the monitoring samples are the samples for monitoring the inference performance of the AI unit.

[0308] Step 12b. The second base station sends the first signal (such as a sensing reference signal) to the UE.

[0309] Step 13b. The UE receives the first signal sent by the second base station, obtains the data related to sensing measurement (such as sensing measurement quantity), and generates the first information including at least one of the first data set and the first metric by combining the data related to sensing measurement and the data related to communication measurement (such as CSI measurement result). The first data set includes the monitoring samples for monitoring the inference performance of the AI unit. A monitoring sample may include at least one of the input of the AI unit, the output of the AI unit, and the ground truth (or label). The first metric includes the monitoring metric for the inference performance of the AI unit.

[0310] Step 14b. The UE sends the first information to the serving base station.

[0311] Step 15b. The serving base station determines the monitoring metric for the inference performance of the AI unit (the first metric) based on the first information. Further, the serving base station may make relevant decisions on the subsequent operations of the AI unit based on the first metric, such as adjusting the structure and / or parameters of the AI unit.

[0312] In this case, the negotiation of the first configuration between the serving base station and the second base station may be carried out in the following ways:

[0313] Way 1. The serving base station determines the first configuration based on the monitoring requirements, sends the first configuration to the second base station, and receives the feedback from the second base station on the first configuration, such as agreement or the negotiated configuration.

[0314] Way 2. The serving base station receives the third configuration sent by the second base station and determines the first configuration.

[0315] Way 3. The serving base station receives the second configuration sent by the second base station and sends feedback information to the second base station based on the monitoring requirements. The feedback information is used to indicate the desired first configuration or the difference between the first configuration and the second configuration.

[0316] In this embodiment, for what information the first configuration includes under different monitoring purposes of the AI unit, the following will be specifically described through several cases.

[0317] Case 1: The monitoring purpose of the AI unit is the timeliness of the sensing measurement quantity (the effective duration to be monitored is less than the default effective duration)

[0318] Exemplarily, assume that the default effective interval of the network, that is, the configuration period of the perception measurement quantity corresponding to the timeliness of the currently considered perception measurement quantity is N. However, the current inference performance is not good, and it is guessed that the default effective duration of the perception measurement quantity is set too long. Then, a shorter usage period of the perception measurement quantity can be monitored. Specifically, when actually configuring the perception measurement quantity, a more intensive perception measurement quantity can be configured, such as a period of 1 / 2N or 1 / 4N. Further, when monitoring the inference performance of the AI model, inferences and monitoring are respectively performed for the perception measurement quantities with periods of N, 1 / 2N, and 1 / 4N, and the obtained AI model inference performance monitoring metrics are classified and recorded.

[0319] As Figure 3c shown, in the figure, (c) represents the sample collection method of the second interval (second period) collected according to the period N. In one effective period, a total of 4 monitoring samples are collected. At this time, it can be approximately understood that the duration of 4 samples is the default effective duration of the perception measurement quantity. This is used as the monitoring benchmark. In the figure, (a) attempts to collect samples according to the period 1 / 4N. In 4 effective periods, a total of 4 monitoring samples are collected. The perception measurement quantity is only provided for the use of the subsequent one sample, that is, the effective duration is shortened to 1 / 4 of the default effective duration.

[0320] In the figure, (b) attempts to collect samples according to the period 1 / 2N. In 2 effective periods, a total of 4 monitoring samples are collected. The perception measurement quantity is only provided for the use of the subsequent 2 samples, that is, the effective duration is shortened to 1 / 2 of the default effective duration.

[0321] Alternatively, the collection of monitoring samples can also be carried out according to the near-term sample collection method. Assume that the configuration period of the perception measurement quantity corresponding to the timeliness of the perception measurement quantity currently known by the network is N. Then, when actually configuring the perception measurement quantity, a more intensive perception measurement quantity can be configured, such as periods of 0.75N, 0.5N, and 0.25N. Then, when monitoring the inference performance of the AI model, inferences and monitoring are respectively performed for the perception measurement quantities with periods of N, 0.75N, 0.5N, and 0.25N, and the obtained AI model inference performance monitoring metrics are classified and recorded.

[0322] As Figure 3d shown, in the figure, (a) represents the near-term sample collection made according to the second interval (second period). Each sample is collected at a time when the time from the acquisition time of the perception measurement quantity is the second period N. Based on this method, continuous sample collection can be performed. In the figure, (b) represents the near-term sample collection made according to 0.75 of the second period (0.75N). Each sample is collected at a time when the time from the acquisition time of the perception measurement quantity is 0.75N. Based on this method, continuous sample collection can be performed.

[0323] In this case, the UE can report in the following two ways:

[0324] Reporting method 1: Reporting monitored sample data (i.e., the first data set)

[0325] In this method, the following contents need to be included in the first configuration:

[0326] The first interval, which is related to the transmission interval of the first signal. For example, the first interval is the actual transmission period of the sensed reference signal;

[0327] The second interval, which is the default valid interval or the baseline of the valid interval to be monitored. For example, the second interval is related to the default transmission interval of the first signal corresponding to the sensed measurement quantity to be monitored by the AI unit;

[0328] The third interval, which is used to indicate the transmission interval of the sensed measurement quantity to be monitored;

[0329] The first coefficient, which is the ratio of the second interval to the first interval;

[0330] The second coefficient, which is the ratio of the first interval to the second interval;

[0331] The third coefficient, which is the ratio of the third interval to the first interval;

[0332] The fourth coefficient, which is the ratio of the first interval to the third interval;

[0333] The first number, which is used to indicate the number of the sensed measurement quantity control groups to be monitored;

[0334] The first indication, which is used to indicate the collection method of the monitored samples, such as the periodic sample collection method as described above Figure 3c in the above is the periodic sample collection method, Figure 3d in the above is the near-term sample collection method;

[0335] The identification of the AI unit monitoring purpose, such as the AI unit monitoring purpose in this embodiment is the timeliness of the sensed measurement quantity.

[0336] In addition, in this reporting method, the following contents also need to be included in the first information:

[0337] The identification of the AI unit;

[0338] The first number, which is used to indicate the number of the sensed measurement quantity control groups to be monitored;

[0339] The first sample set, the first sample set includes monitoring samples of the control group of the perception measurement quantity to be monitored. If the first data is greater than 2, there are multiple first sample sets;

[0340] The second sample set, the second sample set includes the monitoring samples corresponding to the second interval.

[0341] Wherein, the first sample set or the second sample set includes the following: input of the AI unit, output of the AI unit, true value, monitoring sample identifier, number of monitoring samples, number of types of monitoring samples, third indication (used to indicate the effective time of the perception measurement quantity to be monitored), description information of the perception measurement quantity corresponding to each type of the monitoring samples, effective time of the perception measurement quantity set, measured value of the perception measurement quantity in each type of the monitoring samples, fifth indication (used to indicate the number or proportion of the monitoring samples associated with the perception measurement quantity carried in the first sample set or the second sample set).

[0342] It should be noted that the content included in the description information of the perception measurement quantity may be referred to the description in the previous method embodiments, and will not be elaborated here.

[0343] Reporting method 2: Report the first indicator (i.e., the AI unit inference performance monitoring indicator)

[0344] Under this reporting method, the first configuration may include the following content:

[0345] The first interval, the first interval is related to the transmission interval of the first signal. For example, the first interval is the transmission period of the actually transmitted perception reference signal;

[0346] The second interval, the second interval is the default effective interval or the baseline of the effective interval to be monitored. For example, the second interval is related to the default transmission interval of the first signal corresponding to the perception measurement quantity to be monitored by the AI unit;

[0347] The first coefficient, the first coefficient is the ratio of the second interval to the first interval;

[0348] The second coefficient, the second coefficient is the ratio of the first interval to the second interval;

[0349] The third coefficient, the third coefficient is the ratio of the third interval to the first interval, and the third interval is used to indicate the transmission interval of the perception measurement quantity to be monitored;

[0350] The fourth coefficient, the fourth coefficient is the ratio of the first interval to the third interval;

[0351] The first number, the first number is used to indicate the number of the control groups of the perception measurement quantity to be monitored;

[0352] The first indication, which is used to indicate the collection method of the monitored sample;

[0353] The monitoring purpose identifier of the AI unit. For example, in this embodiment, the monitoring purpose of the AI unit is to sense the timeliness of measurement quantities;

[0354] The indication of the first metric, that is, the indication of the AI unit inference performance monitoring metric. For example, the prediction accuracy of the strongest beam identifier, the prediction error of beam quality, the square of the cosine similarity of the predicted channel / PMI / feature vector, the TOA prediction error, the distance prediction error, etc.;

[0355] The first threshold, which is used to indicate the relevant threshold for determining the effective duration of the sensed measurement quantity. For example, when the first metric drops to the first threshold, the corresponding time is the effective duration of the sensed measurement quantity.

[0356] In addition, in this reporting method, the following contents also need to be included in the first information:

[0357] The identifier of the AI unit;

[0358] The first number, which is used to indicate the number of the control groups of the sensed measurement quantities to be monitored;

[0359] The results of the control groups of the sensed measurement quantities to be monitored;

[0360] The effective duration of the sensed measurement quantity;

[0361] The description information of the first data set. For the specific content, please refer to the description in the foregoing method embodiment.

[0362] In this embodiment, the monitoring metric results of the control groups of the sensed measurement quantities to be monitored can be indicated in the following manner:

[0363] Method 1: Direct indication, as shown in Table 3 below.

[0364] Table 3

[0365] Perceived measurement quantity configuration period Monitoring metric result: Prediction accuracy of the strongest beam identifier N 90％ 1.25N 88％ 1.5N 86％ 2N 80％

[0366] Method 2: Difference method, as shown in Table 4 below.

[0367] Table 4

[0368] Perceived measurement quantity configuration period Monitoring metric result: Prediction accuracy of the strongest beam identifier N 90％ Perceived measurement quantity configuration period Monitoring metric result: Difference from the monitoring metric result of configuration period N 1.25N 2 (Indicated accuracy is 90% - 2% = 88%) 1.5N 4 (Indicated accuracy is 90% - 4% = 86%) 2N 10 (Indicated accuracy is 90% - 10% = 80%)

[0369] It should be noted that the above list is only for illustration. Through the above two methods, the monitoring metric results corresponding to different sensed measurement quantity configuration periods can be intuitively obtained.

[0370] Scenario 2: The monitoring purpose of the AI unit is to sense the timeliness of the measurement quantity (the effective duration to be monitored is greater than the default effective duration).

[0371] Exemplarily, assume that the configured period of the sensed measurement quantity corresponding to the currently known timeliness of the sensed measurement quantity in the network is N, and the inference accuracy rate has always been very high. It is speculated that the default effective duration of the sensed measurement quantity is set too short. Then, a longer usage period of the sensed measurement quantity can be monitored. Specifically, when actually configuring the sensed measurement quantity, a sensed measurement quantity with a more relaxed period can be configured, such as a period of 1.25N, 1.5N, or 2N. Then, when monitoring the inference performance of the AI unit, inferences and monitoring are respectively performed on the sensed measurement quantities with periods of N, 1.25N, 1.5N, and 2N, and the obtained AI model inference performance monitoring metrics are classified and recorded.

[0372] As Figure 3e shown in the monitoring sample collection method, in the figure, (a) represents the monitoring sample collection method collected according to the default effective period N of the sensed measurement quantity. Within one effective period N, a total of 4 monitoring samples are collected, that is, obtaining one sensed measurement quantity can provide subsequent 4 samples for the AI unit to perform inferences. In the figure, (b) represents the monitoring sample collection method collected according to the period of 1.25N, that is, obtaining one sensed measurement quantity can provide subsequent 5 samples for the AI unit to perform inferences. Within one effective period of 1.25N, a total of 5 monitoring samples are collected. In the figure, (c) represents the monitoring sample collection method collected according to the period of 1.25N + the fixed number of samples (the aforementioned first quantity). First, obtaining one sensed measurement quantity can provide subsequent 5 samples for the AI unit to perform inferences. In order to keep the number of monitoring samples consistent with that in figure (a), that is, the same as the number of monitoring samples collected in figure (a), only 4 samples closer to the expiration time of the sensed measurement quantity are retained for collection.

[0373] Or, as Figure 3f shown in the monitoring sample collection method, in the figure, (a) represents the near-term sample collection performed according to the default effective period N of the sensed measurement quantity, and each sample is collected at a time N from the acquisition time of the sensed measurement quantity; continuous sample collection can be performed based on this method. In the figure, (b) represents the near-term sample collection performed according to the period of 1.25N, and each sample is collected at a time 1.25N from the acquisition time of the sensed measurement quantity; continuous sample collection can be performed based on this method.

[0374] In this Scenario 2, the UE can report in the following two ways:

[0375] Reporting method 1: Report the monitoring sample data (i.e., the first data set).

[0376] In this mode, the following contents need to be included in the first configuration:

[0377] A first interval, where the first interval is related to the transmission interval of the first signal. For example, the first interval is the actual transmission period of the sensed reference signal;

[0378] A second interval, where the second interval is the baseline of the default valid interval or the valid interval to be monitored. For example, the second interval is related to the default transmission interval of the first signal corresponding to the sensed measurement quantity to be monitored by the AI unit;

[0379] A third interval, where the third interval is used to indicate the transmission interval of the sensed measurement quantity to be monitored;

[0380] A first number, where the first number is used to indicate the number of control groups of the sensed measurement quantity to be monitored;

[0381] A first coefficient, where the first coefficient is the ratio of the second interval to the first interval;

[0382] A second coefficient, where the second coefficient is the ratio of the first interval to the second interval;

[0383] A third coefficient, where the third coefficient is the ratio of the third interval to the first interval;

[0384] A fourth coefficient, where the fourth coefficient is the ratio of the first interval to the third interval;

[0385] A fifth coefficient, where the fifth coefficient is the ratio of the third interval to the second interval;

[0386] A sixth coefficient, where the sixth coefficient is the ratio of the second interval to the third interval;

[0387] A first indication, where the first indication is used to indicate the collection method of the monitoring samples;

[0388] An AI unit monitoring purpose identifier. For example, in this embodiment, the AI unit monitoring purpose is the timeliness of the sensed measurement quantity.

[0389] In addition, in this mode, the following contents also need to be included in the first information:

[0390] The identifier of the AI unit;

[0391] A first number, where the first number is used to indicate the number of control groups of the sensed measurement quantity to be monitored;

[0392] A first sample set, where the first sample set includes the monitoring samples of the control groups of the sensed measurement quantity to be monitored. If the first data is greater than 2, there are multiple first sample sets;

[0393] The second sample set, where the second sample set includes the monitoring samples corresponding to the second interval;

[0394] The first indication, which is used to indicate the collection method of the monitoring samples. For example, Figure 3f the collection method corresponding to (a) is the collection of approaching-expiry samples according to the default effective period N of the perceived measurement quantity, and Figure 3f the collection method corresponding to (b) is the collection of approaching-expiry samples according to a 1.25N period.

[0395] Among them, the first sample set or the second sample set includes the following: the input of the AI unit, the output of the AI unit, the true value, the monitoring sample identifier, the number of monitoring samples, the number of types of monitoring samples, the third indication (used to indicate the effective time of the perceived measurement quantity to be monitored), the description information of the perceived measurement quantity corresponding to each type of the monitoring samples, the effective time of the set of perceived measurement quantities, the measured value of the perceived measurement quantity in each type of the monitoring samples, the fifth indication (used to indicate the number or proportion of the monitoring samples associated with the perceived measurement quantity carried in the first sample set or the second sample set).

[0396] It should be noted that the content included in the description information of the perceived measurement quantity can be referred to the description in the previous method embodiments, and will not be elaborated here.

[0397] Reporting method 2: Report the first indicator (i.e., the AI unit inference performance monitoring indicator)

[0398] Under this reporting method, the first configuration may include the following content:

[0399] The first interval, which is related to the transmission interval of the first signal. For example, the first interval is the transmission period of the actually transmitted perceived reference signal;

[0400] The second interval, which is the default effective interval or the baseline of the effective interval to be monitored. For example, the second interval is related to the default transmission interval of the first signal corresponding to the perceived measurement quantity to be monitored by the AI unit;

[0401] The first coefficient, which is the ratio of the second interval to the first interval;

[0402] The second coefficient, which is the ratio of the first interval to the second interval;

[0403] The third coefficient, which is the ratio of the third interval to the first interval, and the third interval is used to indicate the transmission interval of the perceived measurement quantity to be monitored;

[0404] The fourth coefficient, which is the ratio of the first interval to the third interval;

[0405] The first number, which is used to indicate the number of control groups of the perceived measurement quantity to be monitored;

[0406] The first indication, which is used to indicate the collection method of the monitoring samples;

[0407] The identification of the monitoring purpose of the AI unit. For example, in this embodiment, the monitoring purpose of the AI unit is the timeliness of the perceived measurement quantity;

[0408] The indication of the first index, that is, the indication of the monitoring index of the AI unit's inference performance. For example, the prediction accuracy of the strongest beam identification, the prediction error of the beam quality, the square of the cosine similarity of the predicted channel / PMI / feature vector, the TOA prediction error, the distance prediction error, etc.

[0409] In addition, in this case, the first information also needs to include the following content:

[0410] The identification of the AI unit;

[0411] The first number, which is used to indicate the number of control groups of the perceived measurement quantity to be monitored;

[0412] The results of the control groups of the perceived measurement quantity to be monitored;

[0413] The description information of the first data set. For the specific content, reference can be made to the description in the foregoing method embodiment.

[0414] In this case, the first information directly includes the effective duration of the perceived measurement quantity obtained by monitoring, which can be obtained in the manner shown in Figure 3g As shown. As Figure 3g shown, in (a) of the figure, it means that the perceived measurement quantity is used immediately, and thus a batch of monitoring data is obtained; in (b) of the figure, the perceived measurement quantity is used as the input of the AI unit with a second period of 1.25, and the obtained monitoring data is obtained.

[0415] In this case, the monitoring sample collection methods include the immediate sample collection method and the near-term sample collection method. Taking the AI-based beam prediction as an example, assuming that the monitoring index of the AI unit's inference performance is the prediction accuracy of the strongest beam identification, the monitoring indexes under different usage periods of each perceived measurement quantity can be calculated as shown in Table 5 below.

[0416] Table 5

[0417] Perceived measurement quantity configuration period Monitoring metric: Prediction accuracy of the strongest beam identifier Immediate sample collection 95％ Second period 90％ 1.25 * Second period 88％ 1.5 * Second period 86％ 2 * Second period 80％

[0418] Assume that the first threshold is 10%. Then the effective duration is 95% - 10% = 85% of the monitoring metrics corresponding to the instant sample collection. The smallest configuration period greater than 85% is 1.5 * the second period. Therefore, the effective duration of the sensed measurement quantity is 1.5 * the second period. It can be understood that the above list is only for illustration and does not constitute a limitation to this application.

[0419] In this way, the first configuration may include the following:

[0420] A first interval, where the first interval is related to the transmission interval of the first signal. For example, the first interval is the transmission period of the actually transmitted sensing reference signal;

[0421] A second interval, where the second interval is the default effective interval or the baseline of the effective interval to be monitored. For example, the second interval is related to the default transmission interval of the first signal corresponding to the sensed measurement quantity to be monitored by the AI unit;

[0422] A fourth interval, where the fourth interval is used to indicate the transmission interval of the sensed measurement quantity in the control group of the sensed measurement quantity to be monitored;

[0423] A first number, where the first number is used to indicate the number of the control groups of the sensed measurement quantity to be monitored;

[0424] A first coefficient, where the first coefficient is the ratio of the second interval to the first interval;

[0425] A second coefficient, where the second coefficient is the ratio of the first interval to the second interval;

[0426] A third coefficient, where the third coefficient is the ratio of the third interval to the first interval;

[0427] A fourth coefficient, where the fourth coefficient is the ratio of the first interval to the third interval;

[0428] A first indication, where the first indication is used to indicate the collection method of the monitoring samples;

[0429] The identification of the monitoring purpose of the AI unit. For example, in this embodiment, the monitoring purpose of the AI unit is the timeliness of the sensed measurement quantity;

[0430] The indication of the first metric, that is, the indication of the inference performance monitoring metric of the AI unit. For example, the prediction accuracy of the strongest beam identification, the prediction error of the beam quality, the square of the cosine similarity of the predicted channel / PMI / feature vector, the TOA prediction error, the distance prediction error, etc.;

[0431] A first threshold, where the first threshold is used to indicate the relevant threshold for determining the effective duration of the sensed measurement quantity. Different from the foregoing situation, in this case, the first configuration includes the first threshold (that is, specific to this case).

[0432] In addition, the first information needs to include the following content:

[0433] The identifier of the AI unit;

[0434] The first number, which is used to indicate the number of the control group of the perception measurement quantity to be monitored;

[0435] The effective duration of the perception measurement quantity;

[0436] The fifth coefficient, which is related to the second interval and the effective duration of the perception measurement quantity. For example, the fifth coefficient is the quotient of the effective duration of the perception measurement quantity and the second interval;

[0437] The description information of the first data set.

[0438] Case 3: The monitoring purpose of the AI unit is the effectiveness of the perception measurement quantity

[0439] In this case, it can be to monitor the inference performance of the AI unit separately twice. One time is to monitor the inference performance of the AI unit using the perception measurement quantity, and the other time is to monitor the inference performance of the AI unit without using the perception measurement quantity.

[0440] As Figure 3h shown, (a) in the figure represents the collection of monitoring samples performed periodically. Using the perception measurement quantity and the CSI measurement quantity as the inputs of the AI model to monitor the inference performance of the AI unit. Within 1 effective cycle, a total of 4 monitoring samples are collected; (b) in the figure represents the collection of monitoring samples performed periodically. Using the CSI measurement quantity as the input of the AI model to monitor the inference performance of the AI unit (excluding the perception measurement quantity). Within 1 effective cycle, a total of 4 monitoring samples are collected.

[0441] In this case, the UE can report in the following two ways:

[0442] Reporting method 1: Reporting the monitoring sample data (i.e., the first data set)

[0443] In this method, the first configuration needs to include the following content:

[0444] The identifier of the AI unit monitoring purpose. For example, in this embodiment, the monitoring purpose of the AI unit is the effectiveness of the perception measurement quantity;

[0445] The second interval, which is the default effective interval or the baseline of the effective interval to be monitored. For example, the second interval is related to the default transmission interval of the first signal corresponding to the perception measurement quantity to be monitored by the AI unit;

[0446] The first indication, which is used to indicate the collection method of the monitoring samples.

[0447] In addition, under this reporting method, the first information needs to include the following content:

[0448] The identifier of the AI unit;

[0449] The first number, which is used to indicate the number of the control groups of the sensed measurement quantities to be monitored;

[0450] The first sample set, which includes the monitoring samples of the control groups of the sensed measurement quantities to be monitored. If the first data is greater than 2, there are multiple first sample sets;

[0451] The second sample set, which includes the monitoring samples corresponding to the second interval.

[0452] Among them, the first sample set or the second sample set includes the following content: the input of the AI unit, the output of the AI unit, the true value, the monitoring sample identifier, the number of monitoring samples, the number of types of monitoring samples, the third indication (used to indicate the valid time of the sensed measurement quantity to be monitored), the description information of the sensed measurement quantity corresponding to each type of the monitoring samples, the valid time of the sensed measurement quantity set, the measured value of the sensed measurement quantity in each type of the monitoring samples, the fifth indication (used to indicate the number or proportion of the monitoring samples associated with the sensed measurement quantity carried in the first sample set or the second sample set).

[0453] It should be noted that the content included in the description information of the sensed measurement quantity can be referred to the description in the previous method embodiments and will not be elaborated here.

[0454] Reporting method 2: Report the first indicator (i.e., the AI unit inference performance monitoring indicator)

[0455] Under this reporting method, the first configuration may include the following content:

[0456] The first interval, which is related to the sending interval of the first signal. For example, the first interval is the sending period of the actually sent sensed reference signal;

[0457] The second interval, which is the default valid interval or the baseline of the valid interval to be monitored. For example, the second interval is related to the default sending interval of the first signal corresponding to the sensed measurement quantity to be monitored by the AI unit;

[0458] The first coefficient, which is the ratio of the second interval to the first interval;

[0459] The second coefficient, which is the ratio of the first interval to the second interval;

[0460] The first number, which is used to indicate the number of control groups of the perception measurement quantity to be monitored;

[0461] The first indication, which is used to indicate the collection method of the monitoring sample;

[0462] The identification of the monitoring purpose of the AI unit. For example, in this embodiment, the monitoring purpose of the AI unit is the effectiveness of the perception measurement quantity;

[0463] The indication of the first metric, that is, the indication of the inference performance monitoring metric of the AI unit. For example, the prediction accuracy of the strongest beam identification, the prediction error of the beam quality, the square of the cosine similarity of the predicted channel / PMI / feature vector, the TOA prediction error, the distance prediction error, etc.

[0464] In addition, in this reporting method, the following content also needs to be included in the first information:

[0465] The identification of the AI unit;

[0466] The first number, which is used to indicate the number of control groups of the perception measurement quantity to be monitored;

[0467] The results of the control groups of the perception measurement quantity to be monitored;

[0468] The description information of the first data set. For specific content, reference can be made to the description in the foregoing method embodiments.

[0469] Case 4: The monitoring purpose of the AI unit is the effectiveness of different combinations of perception measurement quantities (that is, the comparison of different configurations of the perception measurement quantity)

[0470] Such as Figure 3i As shown, in the figure, (a) represents the monitoring samples collected by using the first group of perception measurement quantities and CSI measurement quantities as the input of the AI unit for monitoring the inference performance of the AI unit. In one effective period, a total of 4 monitoring samples are collected; (b) in the figure represents the monitoring samples collected by using the second group of perception measurement quantities and CSI measurement quantities as the input of the AI unit for monitoring the inference performance of the AI unit. In one effective period, a total of 3 monitoring samples are collected; (c) in the figure represents the monitoring samples collected by using the first group of perception measurement quantities and CSI measurement quantities as the input of the AI unit for monitoring the inference performance of the AI unit. In one effective period, and in order to align the number of samples with the control group, only 3 monitoring samples are collected.

[0471] In this case, the following content needs to be included in the first configuration:

[0472] The identification of the monitoring purpose of the AI unit. For example, in this embodiment, the monitoring purpose of the AI unit is the comparison of different configurations of the perception measurement quantity;

[0473] Fourth interval, which is used to indicate the transmission interval of the sensed measurement quantity in the control group of the sensed measurement quantity to be monitored;

[0474] First indication, which is used to indicate the collection method of the monitoring sample.

[0475] In addition, in this reporting method, the first information needs to include the following content:

[0476] The identifier of the AI unit;

[0477] First number, which is used to indicate the number of control groups of the sensed measurement quantity to be monitored;

[0478] First sample set, which includes the monitoring samples of the control group of the sensed measurement quantity to be monitored. If the first data is greater than 2, there are multiple first sample sets;

[0479] Second sample set, which includes the monitoring samples corresponding to the second interval.

[0480] Among them, the first sample set or the second sample set includes the following content: the input of the AI unit, the output of the AI unit, the true value, the monitoring sample identifier, the number of monitoring samples, the number of types of monitoring samples, the third indication (used to indicate the valid time of the sensed measurement quantity to be monitored), the description information of the sensed measurement quantity corresponding to each type of the monitoring sample, the valid time of the sensed measurement quantity set, the measured value of the sensed measurement quantity in each type of the monitoring sample, the fifth indication (used to indicate the number or proportion of the monitoring samples associated with the sensed measurement quantity carried in the first sample set or the second sample set).

[0481] It should be noted that the content included in the description information of the sensed measurement quantity may be referred to the description in the previous method embodiments and will not be elaborated here.

[0482] Embodiment 2

[0483] As Figure 3j shown, the UE is the first device, and the serving base station is the second device (or co-located with the third device). The method includes the following steps:

[0484] Step 21a. The serving base station sends the first configuration to the target UE;

[0485] Step 22a. The serving base station sends the sensor-based sensed measurement quantity to the target UE;

[0486] Step 23a. The target UE generates first information including a first data set or a monitoring metric (i.e., the first metric) by combining the perception measurement quantity and the CSI measurement result. The first data set includes monitoring samples for monitoring the inference performance of the AI unit. A monitoring sample may include at least one of the AI unit input, the AI unit output, and the ground truth (or label).

[0487] Step 24a. The target UE sends the first information to the serving base station.

[0488] Step 25a. The serving base station determines the AI unit inference performance monitoring metric (i.e., the first metric) based on the first information. Further, the serving base station adjusts the perception reference signal configuration for the perception reference signal receiving device (such as a UE) based on the first metric.

[0489] Optionally, the following scenarios may also have a similar process as above:

[0490] Scenario 1: Base station 1 sends a perception reference signal, base station 2 receives the perception reference signal, base station 2 obtains the perception measurement quantity based on the perception reference signal, sends the perception measurement quantity to base station 1, and base station 1 then sends the perception measurement quantity to the target UE. The subsequent process is the same as steps 23a to 25a above.

[0491] Scenario 2: Base station 1 sends a perception reference signal, UE2 receives the perception reference signal, UE2 obtains the perception measurement quantity based on the perception reference signal, sends the perception measurement quantity to base station 1, and base station 1 then sends the perception measurement quantity to the target UE. The subsequent process is the same as steps 23a to 25a above.

[0492] Scenario 3: Base station 1 sends a perception reference signal, base station 1 receives the perception reference signal, base station 1 obtains the perception measurement quantity based on the perception reference signal, and sends the perception measurement quantity to the target UE. The subsequent process is the same as steps 23a to 25a above.

[0493] Scenario 4: Base station 2 sends a perception reference signal, base station 2 receives the perception reference signal, base station 2 obtains the perception measurement quantity based on the perception reference signal, sends the perception measurement quantity to base station 1, and base station 1 then sends the perception measurement quantity to the target UE. The subsequent process is the same as steps 23a to 25a above.

[0494] Scenario 5: UE2 sends a perception reference signal, UE2 receives the perception reference signal, UE2 obtains the perception measurement quantity based on the perception reference signal, sends the perception measurement quantity to base station 1, and base station 1 then sends the perception measurement quantity to the target UE. The subsequent process is the same as steps 23a to 25a above.

[0495] Scenario 6: As Figure 3kAs shown, the UE is the first device, the serving base station is the second device (or co-located with the third device), the second base station is the third device, and the second UE is the third device. The serving base station of the second UE is the second base station, and the method includes the following steps:

[0496] Step 21b. Base station 2 (i.e., the second base station in the figure) sends a sensing reference signal;

[0497] Step 22b. UE2 (i.e., the second UE in the figure) receives the sensing reference signal, and UE2 obtains a sensing measurement based on the sensing reference signal;

[0498] Step 23b. UE2 sends the sensing measurement to base station 2;

[0499] Step 24b. Base station 2 sends the sensing measurement to base station 1 (i.e., the serving base station in the figure), and base station 1 then sends the sensing measurement to the target UE. The subsequent process is the same as steps 23a to 25a above.

[0500] Scenario 7: As Figure 3l As shown, the UE is the first device, the serving base station is the second device (or co-located with the third device), the second base station is the third device, and the second UE is the third device. The serving base station of the second UE is the first base station, and the method includes the following steps:

[0501] Step 21c. Base station 2 (i.e., the second base station in the figure) sends a sensing reference signal;

[0502] Step 22c. UE2 (i.e., the second UE in the figure) receives the sensing reference signal, and UE2 obtains a sensing measurement based on the sensing reference signal;

[0503] Step 23c. UE2 sends the sensing measurement to base station 1 (i.e., the serving base station in the figure), and base station 1 then sends the sensing measurement to the target UE. The subsequent process is the same as steps 23a to 25a above.

[0504] Embodiment 3:

[0505] As Figure 3m As shown, the UE is the first device, the serving base station is the second device (or co-located with the third device), the second base station is the third device, and the method includes the following steps:

[0506] Step 31. The first base station (serving base station) sends a first configuration to the UE;

[0507] Step 32. The first base station sends a sensing reference signal to the second base station;

[0508] Step 33. The second base station obtains a sensing measurement based on the sensing reference signal;

[0509] Step 34. The second base station sends the sensing measurement quantity to the UE; (At this time, the second base station is the secondary cell base station of the UE)

[0510] Step 35. The target UE combines the sensing measurement quantity and the CSI measurement result to generate first information including a first data set or a monitoring metric (i.e., the first metric), where the first data set includes monitoring samples for monitoring the inference performance of the AI unit, and one monitoring sample may include at least one of the AI unit input, the AI unit output, and the true value (or label);

[0511] Step 36. The target UE sends the first information to the first base station;

[0512] Step 37. The first base station determines the AI unit inference performance monitoring metric (i.e., the first metric) based on the first information. Further, the first base station adjusts the sensing reference signal configuration sent to the second base station based on the first metric.

[0513] Optionally, the following scenarios may also have a similar process as above:

[0514] Scenario 8: Base station 1 sends a sensing reference signal, UE2 receives the sensing reference signal and obtains the sensing measurement quantity, and UE2 sends the sensing measurement quantity to the target UE. The subsequent process is the same as steps 35 to 37 above.

[0515] Embodiment 4

[0516] For the self-transmitting and self-receiving sensing mode, there may be the following scenarios:

[0517] Scenario 9: Base station 2 sends a sensing reference signal, base station 2 receives the sensing reference signal and obtains the sensing measurement quantity, and base station 2 sends the sensing measurement quantity to the target UE. The subsequent process is the same as steps 35 to 37 above.

[0518] Scenario 10: UE2 sends a sensing reference signal, UE2 receives the sensing reference signal and obtains the sensing measurement quantity, and UE2 sends the sensing measurement quantity to the target UE. The subsequent process is the same as steps 35 to 37 above.

[0519] Scenario 11: The target UE sends a sensing reference signal, the target UE receives the sensing reference signal and obtains the sensing measurement quantity. The subsequent process is the same as steps 35 to 37 above.

[0520] Among them, before the above device sends the sensing reference signal, the serving base station (base station 1) sends a first configuration to the target UE. In addition, after the serving base station determines the AI unit inference performance monitoring metric based on the first information, it may send a sensing measurement adjustment request to the sensing measurement signal sending device or the sensor sensing device indicated in the first information.

[0521] Please refer to Figure 4 , Figure 4 which is a flowchart of a method for monitoring the inference performance of an AI unit provided by an embodiment of the present application. The method is applied to a second device. As Figure 4 shown, the method includes the following steps:

[0522] Step 401, the second device sends a first configuration to the first device;

[0523] wherein, the first configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit;

[0524] Step 402, the second device receives the first information sent by the first device;

[0525] wherein, the first information includes at least one of a first data set and a first metric. The first data set includes the monitoring samples, and the first metric includes an AI unit inference performance monitoring metric.

[0526] Optionally, the first configuration includes at least one of the following:

[0527] AI unit identifier;

[0528] AI unit monitoring purpose identifier;

[0529] A first number, which is used to indicate the number of control groups of sensed measurement quantities to be monitored;

[0530] A first interval, which is related to the transmission interval of a first signal, and the first signal is a signal used for sensing;

[0531] A second interval, which is the default valid interval or the baseline of the valid interval to be monitored;

[0532] A third interval, which is used to indicate the transmission interval of the sensed measurement quantity to be monitored;

[0533] A fourth interval, which is used to indicate the transmission interval of the sensed measurement quantity in the control group of the sensed measurement quantity to be monitored;

[0534] A first coefficient, which is the ratio of the second interval to the first interval;

[0535] A second coefficient, which is the ratio of the first interval to the second interval;

[0536] A third coefficient, which is the ratio of the third interval to the first interval;

[0537] The fourth coefficient, which is the ratio of the first interval to the third interval;

[0538] A first duration, which is related to a reference time and a relative duration. The reference time is determined by the time when a perception measurement quantity is obtained, and the relative duration is the duration relative to the time when the perception measurement quantity is obtained;

[0539] A first indication for indicating the collection method of the monitoring sample;

[0540] An indication of the reporting method of the monitoring sample;

[0541] An indication of the number of the monitoring samples;

[0542] A first threshold for indicating a relevant threshold for determining the effective duration of a perception measurement quantity;

[0543] An indication of the first metric;

[0544] Information for indicating the reason for the second device to trigger the inference performance monitoring of the AI unit.

[0545] Optionally, the collection method of the monitoring sample includes at least one of the following:

[0546] Period-based sample collection;

[0547] Collection based on a fixed number of samples;

[0548] Sliding window-based sample collection;

[0549] Immediate sample collection;

[0550] Near-term sample collection, which is used to indicate obtaining the monitoring sample from the resources associated with the third interval.

[0551] Optionally, the monitoring purposes of the AI unit include at least one of the following:

[0552] AI unit performance;

[0553] Timeliness of perception measurement quantity;

[0554] Validity of perception measurement quantity;

[0555] Comparison of different configurations of perception measurement quantity.

[0556] Optionally, the monitoring sample includes at least one of the following:

[0557] The input of the AI unit;

[0558] The output of the AI unit;

[0559] True value, which is used to indicate the actual measured value of the parameter corresponding to the output of the AI unit.

[0560] Optionally, the input of the AI unit includes perception measurement-related data, and the perception measurement-related data includes perception measurement quantities and at least one of the following:

[0561] Indicator of the perception measurement quantity;

[0562] Timestamp of the perception measurement quantity;

[0563] Information of the sending device of the perception measurement quantity;

[0564] Information of the receiving device of the perception measurement quantity;

[0565] Coordinate information of the perception measurement quantity;

[0566] Information for indicating the performance index of the perception measurement quantity;

[0567] Information for indicating the source of the perception measurement quantity;

[0568] Information for indicating the category of the perception measurement quantity;

[0569] Information for indicating the perception mode of the perception measurement quantity;

[0570] Configuration information of the first signal;

[0571] Information of the sending device of the first signal;

[0572] Information of the receiving device of the first signal;

[0573] Wherein, the first signal is a signal used for perception.

[0574] Optionally, the first information further includes at least one of the following:

[0575] Identity of the AI unit;

[0576] First number, which is used to indicate the number of the control groups of the perception measurement quantities to be monitored;

[0577] First sample set, which includes the monitoring samples of the control groups of the perception measurement quantities to be monitored;

[0578] Second sample set, which includes the monitoring samples corresponding to the second interval, and the second interval is related to the sending interval of the first signal corresponding to the perception measurement quantity to be monitored by the AI unit, and the first signal is a signal used for perception;

[0579] Results of the control groups of the perception measurement quantities to be monitored;

[0580] The effective duration of the sensed measurement quantity;

[0581] A fifth coefficient, where the fifth coefficient is related to the second interval and the effective duration of the sensed measurement quantity;

[0582] The description information of the first data set.

[0583] Optionally, the description information of the first data set includes at least one of the following:

[0584] The identifier of the AI unit;

[0585] The identifier of the monitored sample;

[0586] The number of the monitored samples;

[0587] The number of types of the monitored samples;

[0588] A second indication, where the second indication is used to indicate the sensed measurement quantity known to the second device;

[0589] A third indication, where the third indication is used to indicate the effective time of the monitored sensed measurement quantity;

[0590] The description information of the sensed measurement quantity corresponding to each type of the monitored sample;

[0591] The proportion of the effective sensed measurement quantity in each type of the monitored sample;

[0592] The effective time of the sensed measurement quantity set;

[0593] A fourth indication, where the fourth indication is used to indicate the monitored sample associated with the sensed measurement quantity carried in the first data set;

[0594] A fifth indication, where the fifth indication is used to indicate the number or proportion of the monitored samples associated with the sensed measurement quantity carried in the first data set.

[0595] Optionally, the sensed measurement quantity description information includes at least one of the following:

[0596] The parameter item of the sensed measurement quantity;

[0597] The number of the parameter items of the sensed measurement quantity;

[0598] The processing level of the sensed measurement quantity;

[0599] The information used to indicate the sending device of the sensed measurement quantity;

[0600] The information used to indicate the receiving device of the sensed measurement quantity;

[0601] The source information of the sensed measurement quantity;

[0602] Perceived measurement quantity link identification information;

[0603] Signal configuration identification, which is used to indicate the signal corresponding to the perceived measurement quantity;

[0604] Perceived service information;

[0605] Data subscription identification;

[0606] Information used to indicate the purpose of the perceived measurement quantity;

[0607] Information of the device corresponding to the perceived measurement quantity;

[0608] Coordinate information of the perceived measurement quantity;

[0609] Performance index information corresponding to the perceived measurement quantity.

[0610] Optionally, the method further includes any one of the following:

[0611] The second device determines the first configuration based on the monitoring requirement and sends the first configuration to the third device;

[0612] The second device receives the third configuration sent by the third device;

[0613] The second device receives the second configuration sent by the third device and sends feedback information to the third device, where the feedback information is used to indicate the difference between the first configuration and the second configuration, or is used to indicate the desired first configuration.

[0614] Optionally, when the first information includes the first data set, the method further includes:

[0615] The second device monitors the inference performance of the AI unit based on the first information and determines the first metric.

[0616] Optionally, the method further includes any one of the following:

[0617] The second device adjusts the perceived measurement configuration sent to the first device based on the first metric;

[0618] When the first information includes an indication of the perceived measurement signal sending device or a sensor sensing device indication, the second device sends a perceived measurement adjustment request to the perceived measurement signal sending device or the sensor sensing device indicated in the first information based on the first metric.

[0619] Optionally, the perceived measurement configuration includes at least one of the following:

[0620] Identification of the desired first data set;

[0621] Configuration identifier of the expected perception measurement quantity;

[0622] Fifth interval of the perception measurement quantity;

[0623] Quantity threshold of the perception measurement quantity;

[0624] Information for indicating the source of the perception measurement quantity;

[0625] Information for indicating the category of the perception measurement quantity;

[0626] Perception mode of the perception measurement quantity;

[0627] Indication of the perception requirement;

[0628] Configuration information of the first signal;

[0629] Transmitting device of the first signal;

[0630] Receiving device of the first signal;

[0631] Wherein, the first signal is a signal used for perception.

[0632] It should be noted that the specific implementation process and related concepts involved in the embodiments of the present application can be referred to the description in the foregoing method embodiments, and will not be elaborated in this embodiment.

[0633] The method provided by the embodiments of the present application enables the second device to integrate perception measurement-related data in the monitoring of the inference performance of the AI unit. Compared with only using communication measurement-related data to monitor the inference performance of the AI unit, the solution provided by the present application can better monitor the inference performance of the AI unit.

[0634] Please refer to Figure 5 , Figure 5 is a flowchart of a method for monitoring the inference performance of an AI unit provided by the embodiments of the present application. The method is applied to a third device. As Figure 5 shown, the method includes the following steps:

[0635] Step 501, the third device sends a third configuration to the first device, and the third configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit.

[0636] It should be noted that the third configuration may be the same as or different from the above first configuration. For example, the third configuration may include some or all of the content of the first configuration.

[0637] Optionally, the first configuration includes at least one of the following:

[0638] AI unit identifier;

[0639] AI unit monitoring purpose identifier;

[0640] The first number, which is used to indicate the number of the control group of the sensed measurement quantities to be monitored;

[0641] The first interval, which is related to the transmission interval of the first signal, and the first signal is the signal used for sensing;

[0642] The second interval, which is the default effective interval or the baseline of the effective interval to be monitored;

[0643] The third interval, which is used to indicate the transmission interval of the sensed measurement quantities to be monitored;

[0644] The fourth interval, which is used to indicate the transmission interval of the sensed measurement quantities in the control group of the sensed measurement quantities to be monitored;

[0645] The first coefficient, which is the ratio of the second interval to the first interval;

[0646] The second coefficient, which is the ratio of the first interval to the second interval;

[0647] The third coefficient, which is the ratio of the third interval to the first interval;

[0648] The fourth coefficient, which is the ratio of the first interval to the third interval;

[0649] The first duration, which is related to the reference time and the relative duration. The reference time is determined by the time when the sensed measurement quantity is obtained, and the relative duration is the duration relative to the time when the sensed measurement quantity is obtained;

[0650] The first indication, which is used to indicate the collection method of the monitoring sample;

[0651] The indication of the reporting method of the monitoring sample;

[0652] The indication of the number of the monitoring samples;

[0653] The first threshold, which is used to indicate the relevant threshold for determining the effective duration of the sensed measurement quantity;

[0654] The indication of the first index;

[0655] The information used to indicate the reason for the second device to trigger the AI unit inference performance monitoring.

[0656] Optionally, the method further includes:

[0657] The third device sends perception measurement related data or a first signal to the first device, and the first signal is used to obtain perception measurement related data;

[0658] Wherein, the perception measurement related data is used for the first device to generate first information, and the first information includes at least one of a first data set and a first metric. The first data set includes the monitoring samples, and the first metric includes an AI unit inference performance monitoring metric.

[0659] Optionally, the method further includes any one of the following:

[0660] The third device determines the third configuration and sends the third configuration to the second device;

[0661] The third device receives the first configuration sent by the second device and determines the third configuration;

[0662] The third device sends a second configuration to the second device and receives feedback information sent by the second device. The feedback information is used to indicate the difference between the first configuration and the second configuration, or is used to indicate the desired first configuration. It should be noted that the specific implementation process and related concepts involved in the embodiments of the present application can refer to the description in the foregoing method embodiments, and will not be elaborated in this embodiment.

[0663] In the embodiments of the present application, perception measurement related data is integrated in the monitoring of the AI unit inference performance. Compared with only using communication measurement related data to monitor the AI unit inference performance, the solution provided by the present application can better implement the monitoring of the AI unit inference performance.

[0664] For the method for monitoring the AI unit inference performance provided by the embodiments of the present application, the execution subject may be a monitoring device for the AI unit inference performance. In the embodiments of the present application, taking the monitoring device for the AI unit inference performance to execute the method for monitoring the AI unit inference performance as an example, the monitoring device for the AI unit inference performance provided by the embodiments of the present application is described.

[0665] Please refer to Figure 6 , Figure 6 which is one of the structural diagrams of a monitoring device for the AI unit inference performance provided by the embodiments of the present application. The device is applied to the first device. As Figure 6 shown, the monitoring device 600 for the AI unit inference performance includes:

[0666] A first receiving module 601, configured to receive a first configuration sent by a second device, where the first configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the AI unit inference performance;

[0667] An acquisition module 602, configured to acquire perception measurement-related data and communication measurement-related data, and generate first information according to the first configuration and based on the perception measurement-related data and the communication measurement-related data;

[0668] A first sending module 603, configured to send the first information to a second device;

[0669] Wherein, the first information includes at least one of the first data set and the first metric, the first data set includes the monitoring samples, and the first metric includes an AI unit inference performance monitoring metric.

[0670] Optionally, the first configuration includes at least one of the following:

[0671] An AI unit identifier;

[0672] An AI unit monitoring purpose identifier;

[0673] A first number, which is used to indicate the number of groups of perception measurement quantities to be monitored;

[0674] A first interval, which is related to the sending interval of a first signal, and the first signal is a signal used for perception;

[0675] A second interval, which is the default effective interval or the baseline of the effective interval to be monitored;

[0676] A third interval, which is used to indicate the sending interval of the perception measurement quantity to be monitored;

[0677] A fourth interval, which is used to indicate the sending interval of the perception measurement quantity in the group of perception measurement quantities to be monitored;

[0678] A first coefficient, which is the ratio of the second interval to the first interval;

[0679] A second coefficient, which is the ratio of the first interval to the second interval;

[0680] A third coefficient, which is the ratio of the third interval to the first interval;

[0681] A fourth coefficient, which is the ratio of the first interval to the third interval;

[0682] A first duration, which is related to a reference time and a relative duration, the reference time is determined by the time when the perception measurement quantity is obtained, and the relative duration is the duration relative to the time when the perception measurement quantity is obtained;

[0683] A first indication, which is used to indicate the collection method of the monitoring samples;

[0684] The reporting method indication of the monitored sample;

[0685] The indication of the number of monitored samples;

[0686] The first threshold, which is used to indicate the relevant threshold for determining the effective duration of the sensed measurement quantity;

[0687] The indication of the first indicator;

[0688] Information used to indicate the reason for the second device to trigger the AI unit inference performance monitoring.

[0689] Optionally, the collection method of the monitored sample includes at least one of the following:

[0690] Period-based sample collection;

[0691] Collection based on a fixed number of samples;

[0692] Sliding window-based sample collection;

[0693] Immediate sample collection;

[0694] Near-term sample collection, which is used to indicate obtaining the monitored sample from the resources associated with the third interval.

[0695] Optionally, the AI unit monitoring purposes include at least one of the following:

[0696] AI unit performance;

[0697] Timeliness of the sensed measurement quantity;

[0698] Validity of the sensed measurement quantity;

[0699] Comparison of different configurations of the sensed measurement quantity.

[0700] Optionally, the monitored sample includes at least one of the following:

[0701] The input of the AI unit;

[0702] The output of the AI unit;

[0703] True value, which is used to indicate the actual measured value of the parameter corresponding to the output of the AI unit.

[0704] Optionally, the input of the AI unit includes the sensed measurement-related data, and the sensed measurement-related data includes the sensed measurement quantity and at least one of the following:

[0705] The indication of the sensed measurement quantity;

[0706] The timestamp of the sensed measurement quantity;

[0707] Information of the sending device for the sensed measurement quantity;

[0708] Information of the receiving device for the sensed measurement quantity;

[0709] Coordinate information of the sensed measurement quantity;

[0710] Information for indicating the performance index of the sensed measurement quantity;

[0711] Information for indicating the source of the sensed measurement quantity;

[0712] Information for indicating the category of the sensed measurement quantity;

[0713] Information for indicating the sensing mode of the sensed measurement quantity;

[0714] Configuration information of the first signal;

[0715] Information of the sending device of the first signal;

[0716] Information of the receiving device of the first signal;

[0717] Wherein, the first signal is a signal used for sensing.

[0718] Optionally, the first information further includes at least one of the following:

[0719] The identifier of the AI unit;

[0720] A first number, which is used to indicate the number of the control groups of the sensed measurement quantities to be monitored;

[0721] A first sample set, which includes the monitoring samples of the control groups of the sensed measurement quantities to be monitored;

[0722] A second sample set, which includes the monitoring samples corresponding to a second interval, and the second interval is related to the sending interval of the first signal corresponding to the sensed measurement quantity to be monitored by the AI unit, and the first signal is a signal used for sensing;

[0723] The results of the control groups of the sensed measurement quantities to be monitored;

[0724] The effective duration of the sensed measurement quantity;

[0725] A fifth coefficient, which is related to the second interval and the effective duration of the sensed measurement quantity;

[0726] The description information of the first data set.

[0727] Optionally, the description information of the first data set includes at least one of the following:

[0728] The identifier of the AI unit;

[0729] The identifier of the monitored sample;

[0730] The quantity of the monitored sample;

[0731] The number of types of the monitored sample;

[0732] A second indication for indicating the quantity of sensed measurements known to the second device;

[0733] A third indication for indicating the valid time of the sensed measurements to be monitored;

[0734] The description information of the sensed measurements corresponding to each type of the monitored sample;

[0735] The proportion of valid sensed measurements in each type of the monitored sample;

[0736] The valid time of the set of sensed measurements;

[0737] A fourth indication for indicating the monitored samples associated with the sensed measurements carried in the first dataset;

[0738] A fifth indication for indicating the quantity or proportion of the monitored samples associated with the sensed measurements carried in the first dataset.

[0739] Optionally, the description information of the sensed measurements includes at least one of the following:

[0740] The parameter items of the sensed measurements;

[0741] The number of the parameter items of the sensed measurements;

[0742] The processing level of the sensed measurements;

[0743] The information for indicating the sending device of the sensed measurements;

[0744] The information for indicating the receiving device of the sensed measurements;

[0745] The source information of the sensed measurements;

[0746] The link identification information of the sensed measurements;

[0747] A signal configuration identifier for indicating the signal corresponding to the sensed measurements;

[0748] The sensed service information;

[0749] The data subscription identifier;

[0750] The information for indicating the use of the sensed measurements;

[0751] Information of the device corresponding to the sensed measurement quantity;

[0752] Coordinate information of the sensed measurement quantity;

[0753] Performance index information corresponding to the sensed measurement quantity.

[0754] Optionally, the obtaining module is further configured to perform any one of the following:

[0755] Receive the sensed measurement-related data sent by the second device;

[0756] Receive the first signal sent by the second device or the third device, and obtain the sensed measurement-related data according to the first signal;

[0757] Receive the sensed measurement-related data sent by the third device, where the sensed measurement-related data is obtained by the third device through measurement by a sensor device or based on a second signal, or is received from other sensing devices;

[0758] Wherein, the first signal or the second signal is a signal used for sensing.

[0759] Optionally, the first sending module 603 is further configured to:

[0760] In the case of a first event being triggered, send the first information to the second device, where the first information includes an identifier of the first event; wherein, the first event includes at least one of the following:

[0761] The first metric meets a preset threshold;

[0762] The first device receives second indication information sent by the third device, where the second indication information is used to indicate that the quantity of the sensed measurement quantity changes or the sensing performance changes;

[0763] The moving speed of the first device changes;

[0764] The quantity of the sensed measurement quantity that the first device can obtain changes or the sensing performance changes.

[0765] The device provided by the embodiments of the present application can implement each process implemented by the embodiments of the monitoring method for the inference performance of the AI unit as described above, and achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0766] Please refer to Figure 7 , Figure 7 is the second structural diagram of a monitoring device for the inference performance of an AI unit provided by the embodiments of the present application. The device is applied to the second device. As Figure 7As shown, the monitoring device 700 for the inference performance of the AI unit includes:

[0767] A second sending module 701, configured to send a first configuration to a first device, where the first configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit;

[0768] A second receiving module 702, configured to receive first information sent by the first device;

[0769] Wherein, the first information includes at least one of a first data set and a first metric, the first data set includes the monitoring samples, and the first metric includes an AI unit inference performance monitoring metric.

[0770] Optionally, the first configuration includes at least one of the following:

[0771] AI unit identifier;

[0772] AI unit monitoring purpose identifier;

[0773] A first number, which is used to indicate the number of groups of perception measurement quantities to be monitored;

[0774] A first interval, which is related to the sending interval of a first signal, and the first signal is a signal used for perception;

[0775] A second interval, which is the default valid interval or the baseline of the valid interval to be monitored;

[0776] A third interval, which is used to indicate the sending interval of the perception measurement quantity to be monitored;

[0777] A fourth interval, which is used to indicate the sending interval of the perception measurement quantity in the group of perception measurement quantities to be monitored;

[0778] A first coefficient, which is the ratio of the second interval to the first interval;

[0779] A second coefficient, which is the ratio of the first interval to the second interval;

[0780] A third coefficient, which is the ratio of the third interval to the first interval;

[0781] A fourth coefficient, which is the ratio of the first interval to the third interval;

[0782] A first duration, which is related to a reference time and a relative duration, the reference time is determined by the time when the perception measurement quantity is obtained, and the relative duration is the duration relative to the time when the perception measurement quantity is obtained;

[0783] The first indication, which is used to indicate the collection method of the monitored sample;

[0784] The indication of the reporting method of the monitored sample;

[0785] The indication of the quantity of the monitored sample;

[0786] The first threshold, which is used to indicate the relevant threshold for determining the effective duration of the sensed measurement quantity;

[0787] The indication of the first metric;

[0788] Information used to indicate the reason for triggering the AI unit inference performance monitoring by the second device.

[0789] Optionally, the collection method of the monitored sample includes at least one of the following:

[0790] Period-based sample collection;

[0791] Collection based on a fixed number of samples;

[0792] Sliding window-based sample collection;

[0793] Immediate sample collection;

[0794] Near-term sample collection, which is used to indicate obtaining the monitored sample from the resources associated with the third interval.

[0795] Optionally, the monitoring purposes of the AI unit include at least one of the following:

[0796] AI unit performance;

[0797] Timeliness of the sensed measurement quantity;

[0798] Validity of the sensed measurement quantity;

[0799] Comparison of different configurations of the sensed measurement quantity.

[0800] Optionally, the monitored sample includes at least one of the following:

[0801] The input of the AI unit;

[0802] The output of the AI unit;

[0803] The true value, which is used to indicate the actual measured value of the parameter corresponding to the output of the AI unit.

[0804] Optionally, the input of the AI unit includes sensed measurement-related data, and the sensed measurement-related data includes the sensed measurement quantity and at least one of the following:

[0805] Indicator of the perceived measurement quantity;

[0806] Timestamp of the perceived measurement quantity;

[0807] Information on the sending device of the perceived measurement quantity;

[0808] Information on the receiving device of the perceived measurement quantity;

[0809] Coordinate information of the perceived measurement quantity;

[0810] Information for indicating the performance index of the perceived measurement quantity;

[0811] Information for indicating the source of the perceived measurement quantity;

[0812] Information for indicating the category of the perceived measurement quantity;

[0813] Information for indicating the perception mode of the perceived measurement quantity;

[0814] Configuration information of the first signal;

[0815] Information on the sending device of the first signal;

[0816] Information on the receiving device of the first signal;

[0817] Wherein, the first signal is a signal used for perception.

[0818] Optionally, the first information further includes at least one of the following:

[0819] The identifier of the AI unit;

[0820] A first number, which is used to indicate the number of the control groups of the perceived measurement quantities to be monitored;

[0821] A first sample set, which includes the monitoring samples of the control groups of the perceived measurement quantities to be monitored;

[0822] A second sample set, which includes the monitoring samples corresponding to a second interval, and the second interval is related to the sending interval of the first signal corresponding to the perceived measurement quantity to be monitored by the AI unit, and the first signal is a signal used for perception;

[0823] The results of the control groups of the perceived measurement quantities to be monitored;

[0824] The effective duration of the perceived measurement quantity;

[0825] A fifth coefficient, which is related to the second interval and the effective duration of the perceived measurement quantity;

[0826] The description information of the first data set.

[0827] Optionally, the description information of the first data set includes at least one of the following:

[0828] The identifier of the AI unit;

[0829] The identifier of the monitoring sample;

[0830] The quantity of the monitoring samples;

[0831] The number of types of the monitoring samples;

[0832] A second indication for indicating the perception measurement quantities known to the second device;

[0833] A third indication for indicating the valid time of the monitored perception measurement quantities;

[0834] The perception measurement quantity description information corresponding to each type of the monitoring samples;

[0835] The proportion of valid perception measurement quantities in each type of the monitoring samples;

[0836] The valid time of the perception measurement quantity set;

[0837] A fourth indication for indicating the monitoring samples associated with the perception measurement quantities carried in the first data set;

[0838] A fifth indication for indicating the quantity or proportion of the monitoring samples associated with the perception measurement quantities carried in the first data set.

[0839] Optionally, the perception measurement quantity description information includes at least one of the following:

[0840] The parameter items of the perception measurement quantity;

[0841] The number of the parameter items of the perception measurement quantity;

[0842] The processing level of the perception measurement quantity;

[0843] The information for indicating the sending device of the perception measurement quantity;

[0844] The information for indicating the receiving device of the perception measurement quantity;

[0845] The source information of the perception measurement quantity;

[0846] The perception measurement quantity link identification information;

[0847] A signal configuration identifier for indicating the signal corresponding to the perception measurement quantity;

[0848] The perception service information;

[0849] Data subscription identifier;

[0850] Information for indicating the use of the sensed measurement quantity;

[0851] Information about the device corresponding to the sensed measurement quantity;

[0852] Coordinate information of the sensed measurement quantity;

[0853] Performance index information corresponding to the sensed measurement quantity.

[0854] Optionally, the second sending module is further configured to:

[0855] Determine the first configuration based on the monitoring requirements and send the first configuration to the third device;

[0856] Alternatively, the second receiving module is further configured to perform any one of the following:

[0857] Receive a third configuration sent by the third device;

[0858] Receive a second configuration sent by the third device and send feedback information to the third device, where the feedback information is used to indicate the difference between the first configuration and the second configuration, or to indicate the desired first configuration.

[0859] Optionally, when the first information includes the first data set, the apparatus further includes:

[0860] A determination module, configured to perform AI unit inference performance monitoring based on the first information and determine the first metric.

[0861] Optionally, the apparatus further includes:

[0862] An adjustment module, configured to adjust the sensed measurement configuration sent to the first device based on the first metric;

[0863] Alternatively, the second sending module is further configured to: when the first information includes an indication of a sensed measurement signal sending device or a sensor sensing device, send a sensed measurement adjustment request to the sensed measurement signal sending device or the sensor sensing device indicated in the first information based on the first metric.

[0864] Optionally, the sensed measurement configuration includes at least one of the following:

[0865] Identifier of the desired first data set;

[0866] Configuration identifier of the desired sensed measurement quantity;

[0867] Fifth interval of the sensed measurement quantity;

[0868] The number threshold of the sensed measurement quantity;

[0869] Information for indicating the source of the sensed measurement quantity;

[0870] Information for indicating the category of the sensed measurement quantity;

[0871] The sensing mode of the sensed measurement quantity;

[0872] The indication of the sensing requirement;

[0873] The configuration information of the first signal;

[0874] The sending device of the first signal;

[0875] The receiving device of the first signal;

[0876] Wherein, the first signal is a signal used for sensing.

[0877] The device provided by the embodiment of the present application can implement each process implemented by the embodiment of the monitoring method for the inference performance of the AI unit as described above, and achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0878] Please refer to Figure 8 , Figure 8 is the third structure diagram of a monitoring device for the inference performance of an AI unit provided by the embodiment of the present application. The device is applied to a third device. As Figure 8 shown, the monitoring device 800 for the inference performance of the AI unit includes:

[0879] The third sending module 801 is configured to send a third configuration to the first device, and the third configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit.

[0880] Optionally, the third configuration includes at least one of the following:

[0881] AI unit identifier;

[0882] AI unit monitoring purpose identifier;

[0883] The first number, and the first number is used to indicate the number of the control groups of the sensed measurement quantities to be monitored;

[0884] The first interval, and the first interval is related to the sending interval of the first signal, and the first signal is a signal used for sensing;

[0885] The second interval, and the second interval is the default valid interval or the baseline of the valid interval to be monitored;

[0886] The third interval, and the third interval is used to indicate the sending interval of the sensed measurement quantities to be monitored;

[0887] Fourth interval, where the fourth interval is used to indicate the transmission interval of the sensed measurement quantity in the control group of the sensed measurement quantities to be monitored;

[0888] First coefficient, where the first coefficient is the ratio of the second interval to the first interval;

[0889] Second coefficient, where the second coefficient is the ratio of the first interval to the second interval;

[0890] Third coefficient, where the third coefficient is the ratio of the third interval to the first interval;

[0891] Fourth coefficient, where the fourth coefficient is the ratio of the first interval to the third interval;

[0892] First duration, where the first duration is related to a reference time and a relative duration. The reference time is determined by the time when the sensed measurement quantity is obtained, and the relative duration is the duration relative to the time when the sensed measurement quantity is obtained;

[0893] First indication, where the first indication is used to indicate the collection method of the monitoring sample;

[0894] Indication of the reporting method of the monitoring sample;

[0895] Indication of the number of the monitoring samples;

[0896] First threshold, where the first threshold is used to indicate the relevant threshold for determining the effective duration of the sensed measurement quantity;

[0897] Indication of the first metric;

[0898] Information for indicating the reason for the second device to trigger the inference performance monitoring of the AI unit.

[0899] Optionally, the third sending module is further configured to:

[0900] Send sensed measurement-related data or a first signal to the first device, where the first signal is used to obtain sensed measurement-related data;

[0901] Wherein, the sensed measurement-related data is used for the first device to generate first information, and the first information includes at least one of a first data set and a first metric. The first data set includes the monitoring samples, and the first metric includes the inference performance monitoring metric of the AI unit.

[0902] Optionally, the third sending module is further configured to perform any one of the following:

[0903] The third device determines the third configuration and sends the third configuration to the second device;

[0904] The third device sends a second configuration to the second device and receives feedback information sent by the second device, where the feedback information is used to indicate the difference between the first configuration and the second configuration, or is used to indicate the desired first configuration.

[0905] Alternatively, the apparatus further includes a third receiving module, configured to receive the first configuration sent by the second device and determine the third configuration.

[0906] The apparatus provided in the embodiments of the present application can implement each process implemented by the embodiments of the above-mentioned AI unit inference performance monitoring method and achieve the same technical effects. To avoid repetition, details are not described here again.

[0907] The above-mentioned apparatus in the embodiments of the present application may be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal or other devices other than a terminal. Exemplarily, the terminal may include, but is not limited to, the types of the above-mentioned terminal 11, and other devices may be a server, a Network Attached Storage (NAS), etc. The embodiments of the present application do not make specific limitations.

[0908] As Figure 9 shown, the embodiments of the present application further provide a communication device 900, including a processor 901 and a memory 902. A program or instruction that can run on the processor 901 is stored on the memory 902. When the program or instruction is executed by the processor m01, it implements each step of the above-mentioned embodiments of the AI unit inference performance monitoring method and can achieve the same technical effects. To avoid repetition, details are not described here again.

[0909] The embodiments of the present application further provide a terminal, including a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to run a program or instruction to implement the steps in the above-mentioned embodiments of the AI unit inference performance monitoring method. Each implementation process and implementation manner of the above-mentioned method embodiments can be applied to this terminal embodiment and can achieve the same technical effects. Specifically, Figure 10 FIG. is a schematic diagram of the hardware structure of a terminal for implementing the embodiments of the present application.

[0910] The terminal 1000 includes, but is not limited to, at least some components such as a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010.

[0911] Those skilled in the art can understand that the terminal 1000 may further include a power source (such as a battery) for powering each component. The power source can be logically connected to the processor 1010 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. Figure 10 The terminal structure shown in Figure 10 does not limit the terminal. The terminal may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0912] It should be understood that in the embodiments of the present application, the input unit 1004 may include a Graphics Processing Unit (GPU) 10041 and a microphone 10042. The GPU 10041 processes the image data of static pictures or videos obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include two parts: a touch detection device and a touch controller. The other input devices 10072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, which will not be elaborated here.

[0913] In the embodiments of the present application, after receiving downlink data from a network-side device, the radio frequency unit 1001 can transmit it to the processor 1010 for processing; in addition, the radio frequency unit 1001 can send uplink data to the network-side device. Generally, the radio frequency unit 1001 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, etc.

[0914] The memory 1009 can be used to store software programs or instructions as well as various data. The memory 1009 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area may store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 1009 may include a volatile memory or a non-volatile memory 100. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synch link dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The memory 1009 in the embodiments of the present application includes but is not limited to these and any other suitable types of memories.

[0915] The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 1010 either.

[0916] Among them, when the terminal is a first device, the radio frequency unit 1001 is used to receive a first configuration sent by a second device, and the first configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit;

[0917] The processor 1010 is used to obtain perception measurement-related data and communication measurement-related data, and generate first information according to the first configuration and based on the perception measurement-related data and the communication measurement-related data;

[0918] A radio frequency unit 1001, configured to send the first information to a second device;

[0919] Wherein, the first information includes at least one of the first data set and the first metrics, the first data set includes the monitoring samples, and the first metrics include AI unit inference performance monitoring metrics.

[0920] Alternatively, when the terminal is the second device, the radio frequency unit 1001 is configured to send a first configuration to the first device, the first configuration is used to indicate collection of monitoring samples, the monitoring samples are samples for monitoring AI unit inference performance, and receive the first information sent by the first device;

[0921] Wherein, the first information includes at least one of the first data set and the first metrics, the first data set includes the monitoring samples, and the first metrics include AI unit inference performance monitoring metrics.

[0922] Alternatively, when the terminal is the third device, the radio frequency unit 1001 is configured to send a third configuration to the first device, the third configuration is used to indicate collection of monitoring samples, the monitoring samples are samples for monitoring AI unit inference performance.

[0923] It can be understood that the implementation processes of the implementation manners mentioned in this embodiment can refer to the relevant descriptions in the foregoing method embodiments. The terminal in this embodiment can achieve the same or corresponding technical effects. To avoid repetition, details are not described herein again.

[0924] An embodiment of the present application further provides a network-side device, including a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction to implement the steps of the foregoing method embodiments. This network-side device embodiment corresponds to the foregoing network-side device method embodiment. Each implementation process and implementation manner of the foregoing method embodiment can be applied to this network-side device embodiment, and the same technical effects can be achieved.

[0925] Specifically, an embodiment of the present application further provides a network-side device. As Figure 11 shown, the network-side device 1100 includes: an antenna 111, a radio frequency device 112, a baseband device 113, a processor 114, and a memory 115. The antenna 111 is connected to the radio frequency device 112. In the uplink direction, the radio frequency device 112 receives information through the antenna 111 and sends the received information to the baseband device 113 for processing. In the downlink direction, the baseband device 113 processes the information to be sent and sends it to the radio frequency device 112. The radio frequency device 112 processes the received information and then sends it out through the antenna 111.

[0926] In the above embodiments, the method executed by the network-side device may be implemented in the baseband device 113, and the baseband device 113 includes a baseband processor.

[0927] The baseband device 113 may include, for example, at least one baseband board, on which a plurality of chips are provided, such as Figure 11 As shown, one of the chips is, for example, a baseband processor, which is connected to the memory 115 through a bus interface to call the program in the memory 115 and execute the operations of the network device shown in the above method embodiments.

[0928] The network-side device may further include a network interface 116, and the interface is, for example, a Common Public Radio Interface (CPRI).

[0929] Specifically, the network-side device 1100 in the embodiments of the present invention further includes: instructions or programs stored on the memory 115 and executable on the processor 114. The processor 114 calls the instructions or programs in the memory 115 to execute Figure 6 or Figure 7 or Figure 8 the methods executed by the modules shown, and achieves the same technical effects. To avoid repetition, they are not described herein again.

[0930] Specifically, the embodiments of the present application further provide a network-side device. As Figure 12 shown, the network-side device 1200 includes: a processor 1201, a network interface 1202, and a memory 1203. Among them, the network interface 1202 is, for example, a common public radio interface (CPRI).

[0931] Specifically, the network-side device 1200 in the embodiments of the present invention further includes: instructions or programs stored on the memory 1203 and executable on the processor 1201. The processor 1201 calls the instructions or programs in the memory 1203 to execute Figure 6 or Figure 7 or Figure 8 the methods executed by the modules shown, and achieves the same technical effects. To avoid repetition, they are not described herein again.

[0932] The embodiments of the present application further provide a readable storage medium, on which a program or instructions are stored. When the program or instructions are executed by a processor, the processes of the above Figure 2 , Figure 4 or Figure 5 method embodiments are implemented, and the same technical effects can be achieved. To avoid repetition, they are not described herein again.

[0933] Among them, the processor is the processor in the terminal described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disks, or optical discs, etc. In some examples, the readable storage medium may be a non-transitory readable storage medium.

[0934] Another embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the above Figure 2 , Figure 4 or Figure 5 each process of the method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0935] It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, system chip, chip system, or system-on-chip, etc.

[0936] Another embodiment of the present application provides a computer program / program product, which is stored in a storage medium, and the computer program / program product is executed by at least one processor to implement the above Figure 2 , Figure 4 or Figure 5 each process of the method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0937] The embodiments of the present application further provide a communication system, including: a first device, a second device, and a third device. The first device can be used to execute the steps of the method described in the first aspect, the second device can be used to execute the steps of the method described in the second aspect, and the third device can be used to execute the steps of the method described in the third aspect.

[0938] It should be noted that in this document, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the existence of additional identical elements in the process, method, article or device comprising such element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, but may also include performing functions in a substantially simultaneous manner or in the reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted or combined. Additionally, the features described with reference to certain examples may be combined in other examples.

[0939] From the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of a computer software product plus a necessary general hardware platform, and of course, can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes several instructions for causing a terminal or a network-side device to execute the methods described in the various embodiments of the present application.

[0940] The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms of embodiments without departing from the purpose of the present application and the scope protected by the claims. These embodiments are all within the protection scope of the present application.

Claims

1. A method for monitoring the inference performance of an artificial intelligence AI unit, characterized in that, Including: The first device receives a first configuration sent by the second device, where the first configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit; The first device obtains perception measurement-related data and communication measurement-related data, and generates first information according to the first configuration and based on the perception measurement-related data and the communication measurement-related data; The first device sends the first information to the second device; Wherein, the first information includes at least one of a first data set and a first metric, the first data set includes the monitoring samples, and the first metric includes an AI unit inference performance monitoring metric.

2. The method according to claim 1, characterized in that, The first configuration includes at least one of the following: AI unit identifier; AI unit monitoring purpose identifier; A first number, which is used to indicate the number of control groups of perception measurement quantities to be monitored; A first interval, which is related to the transmission interval of a first signal, and the first signal is a signal used for perception; A second interval, which is a default valid interval or a baseline of the valid interval to be monitored; A third interval, which is used to indicate the transmission interval of the perception measurement quantity to be monitored; A fourth interval, which is used to indicate the transmission interval of the perception measurement quantity in the control group of the perception measurement quantity to be monitored; A first coefficient, which is the ratio of the second interval to the first interval; A second coefficient, which is the ratio of the first interval to the second interval; A third coefficient, which is the ratio of the third interval to the first interval; A fourth coefficient, which is the ratio of the first interval to the third interval; A first duration, which is related to a reference time and a relative duration, the reference time is determined by the time of obtaining the perception measurement quantity, and the relative duration is the duration relative to the time of obtaining the perception measurement quantity; A first indication, which is used to indicate the collection method of the monitoring samples; The reporting method indication of the monitoring samples; The indication of the number of the monitoring samples; A first threshold, which is used to indicate the relevant threshold for determining the valid duration of the perception measurement quantity; The indication of the first metric; Information used to indicate the reason for the second device to trigger the AI unit inference performance monitoring.

3. The method according to claim 2, wherein The collection method of the monitoring samples includes at least one of the following: Period-based sample collection; Collection based on a fixed number of samples; Sliding window-based sample collection; Immediate sample collection; Near-term sample collection, which is used to indicate obtaining the monitoring samples from the resources associated with the third interval.

4. The method according to claim 2, wherein The AI unit monitoring purposes include at least one of the following: AI unit performance; Timeliness of perception measurement quantity; Validity of perception measurement quantity; Comparison of different configurations of perception measurement quantity.

5. The method according to claim 1, wherein The monitoring samples include at least one of the following: The input of the AI unit; The output of the AI unit; True value, which is used to indicate the actual measured value of the parameter corresponding to the output of the AI unit.

6. The method according to claim 5, characterized in that The input of the AI unit includes the perception measurement-related data, and the perception measurement-related data includes perception measurement quantities and at least one of the following: Indicator of the perceived measurement quantity; Timestamp of the perceived measurement quantity; Information of the sending device of the perceived measurement quantity; Information of the receiving device of the perceived measurement quantity; Coordinate information of the perceived measurement quantity; Information for indicating the performance index of the perceived measurement quantity; Information for indicating the source of the perceived measurement quantity; Information for indicating the category of the perceived measurement quantity; Information for indicating the perception mode of the perceived measurement quantity; Configuration information of the first signal; Information of the sending device of the first signal; Information of the receiving device of the first signal; Wherein, the first signal is a signal used for perception.

7. The method according to claim 1, characterized in that, The first information further includes at least one of the following: The identifier of the AI unit; The first number, which is used to indicate the number of control groups of the perceived measurement quantity to be monitored; The first sample set, which includes the monitoring samples of the control groups of the perceived measurement quantity to be monitored; The second sample set, which includes the monitoring samples corresponding to the second interval, and the second interval is related to the sending interval of the first signal corresponding to the perceived measurement quantity to be monitored by the AI unit, and the first signal is a signal used for perception; The results of the control groups of the perceived measurement quantity to be monitored; The effective duration of the perceived measurement quantity; The fifth coefficient, which is related to the second interval and the effective duration of the perceived measurement quantity; The description information of the first data set.

8. The method according to claim 7, wherein The description information of the first data set includes at least one of the following: The identifier of the AI unit; The identifier of the monitoring sample; The number of the monitoring samples; The number of types of the monitoring samples; The second indication, which is used to indicate the perceived measurement quantity known to the second device; The third indication, which is used to indicate the effective time of the monitored perceived measurement quantity; The description information of the perceived measurement quantity corresponding to each type of the monitoring samples; The proportion of the effective perceived measurement quantity in each type of the monitoring samples; The effective time of the set of perceived measurement quantities; The fourth indication, which is used to indicate the monitoring samples associated with the perceived measurement quantity carried in the first data set; The fifth indication, which is used to indicate the number or proportion of the monitoring samples associated with the perceived measurement quantity carried in the first data set.

9. The method according to claim 8, wherein The perceived measurement quantity description information includes at least one of the following: The parameter items of the perceived measurement quantity; The number of the parameter items of the perceived measurement quantity; The processing level of the perceived measurement quantity; Information for indicating the sending device of the perceived measurement quantity; Information for indicating the receiving device of the perceived measurement quantity; The source information of the perceived measurement quantity; The perceived measurement quantity link identification information; The signal configuration identifier, which is used to indicate the signal corresponding to the perceived measurement quantity; The perception service information; The data subscription identifier; Information for indicating the use of the perceived measurement quantity; Information of the device corresponding to the perceived measurement quantity; Coordinate information of the perceived measurement quantity; Performance index information corresponding to the perceived measurement quantity.

10. The method according to any one of claims 1-9, characterized in that, The first device obtains perception measurement related data, including any one of the following: The first device receives the perception measurement related data sent by the second device; The first device receives the first signal sent by the second device or the third device, and obtains the perception measurement related data according to the first signal; The first device receives the perception measurement related data sent by the third device. The perception measurement related data is obtained by the third device based on the measurement of the sensor device or the measurement based on the second signal, or is the perception measurement related data received from other perception devices; Wherein, the first signal or the second signal is a signal used for perception.

11. The method according to any one of claims 1 to 10, characterized in that The first device sends the first information to the second device, including: In the case of the trigger of the first event, the first device sends the first information to the second device, and the first information includes the identifier of the first event; wherein, the first event includes at least one of the following: The first metric meets a preset threshold; The first device receives the second indication information sent by the third device, and the second indication information is used to indicate that the quantity of the perception measurement changes or the perception performance changes; The moving speed of the first device changes; The quantity of the perception measurement that the first device can obtain changes or the perception performance changes.

12. A method for monitoring the inference performance of an AI unit, characterized in that, Including: The second device sends the first configuration to the first device, and the first configuration is used to indicate the collection of the monitoring samples, and the monitoring samples are samples for monitoring the inference performance of the AI unit; The second device receives the first information sent by the first device; Wherein, the first information includes at least one of the first data set and the first metric. The first data set includes the monitoring samples, and the first metric includes the monitoring metric of the AI unit inference performance.

13. The method according to claim 12, characterized in that, The first configuration includes at least one of the following: AI unit identifier; AI unit monitoring purpose identifier; The first number, which is used to indicate the number of the perception measurement control groups to be monitored; The first interval, which is related to the sending interval of the first signal, and the first signal is a signal used for perception; The second interval, which is the default valid interval or the baseline of the valid interval to be monitored; The third interval, which is used to indicate the sending interval of the perception measurement to be monitored; The fourth interval, which is used to indicate the sending interval of the perception measurement in the perception measurement control group to be monitored; The first coefficient, which is the ratio of the second interval to the first interval; The second coefficient, which is the ratio of the first interval to the second interval; The third coefficient, which is the ratio of the third interval to the first interval; The fourth coefficient, which is the ratio of the first interval to the third interval; The first duration, which is related to the reference time and the relative duration. The reference time is determined by the time when the perception measurement is obtained, and the relative duration is the duration relative to the time when the perception measurement is obtained; The first indication, which is used to indicate the collection method of the monitoring samples; The reporting method indication of the monitoring samples; The indication of the quantity of the monitoring samples; The first threshold, which is used to indicate the relevant threshold for determining the valid duration of the perception measurement; The indication of the first metric; The information used to indicate the reason for the second device to trigger the AI unit inference performance monitoring.

14. The method according to claim 13, wherein The collection method of the monitoring samples includes at least one of the following: Period-based sample collection; Collection based on a fixed number of samples; Sliding window-based sample collection; Immediate sample collection; Near-term sample collection, where the near-term sample collection is used to indicate obtaining the monitoring sample from the resources associated with the third interval.

15. The method according to claim 13, wherein The monitoring purposes of the AI unit include at least one of the following: AI unit performance; Timeliness of the sensed measurement quantity; Validity of the sensed measurement quantity; Comparison of different configurations of the sensed measurement quantity.

16. The method according to claim 12, characterized in that, The monitoring sample includes at least one of the following: The input of the AI unit; The output of the AI unit; True value, where the true value is used to indicate the actual measured value of the parameter corresponding to the output of the AI unit.

17. The method according to claim 16, wherein The input of the AI unit includes sensed measurement-related data, and the sensed measurement-related data includes the sensed measurement quantity and at least one of the following: Indicator of the sensed measurement quantity; Timestamp of the sensed measurement quantity; Information of the sending device of the sensed measurement quantity; Information of the receiving device of the sensed measurement quantity; Coordinate information of the sensed measurement quantity; Information indicating the performance index of the sensed measurement quantity; Information indicating the source of the sensed measurement quantity; Information indicating the category of the sensed measurement quantity; Information indicating the sensing mode of the sensed measurement quantity; Configuration information of the first signal; Information of the sending device of the first signal; Information of the receiving device of the first signal; Wherein, the first signal is a signal used for sensing.

18. The method according to claim 12, wherein The first information further includes at least one of the following: The identifier of the AI unit; First number, where the first number is used to indicate the number of control groups of sensed measurement quantities to be monitored; First sample set, where the first sample set includes the monitoring samples of the control groups of sensed measurement quantities to be monitored; Second sample set, where the second sample set includes the monitoring samples corresponding to the second interval, and the second interval is related to the sending interval of the first signal corresponding to the sensed measurement quantity to be monitored by the AI unit, and the first signal is a signal used for sensing; Results of the control groups of sensed measurement quantities to be monitored; Effective duration of the sensed measurement quantity; Fifth coefficient, where the fifth coefficient is related to the second interval and the effective duration of the sensed measurement quantity; Description information of the first data set.

19. The method according to claim 18, characterized in that, The description information of the first data set includes at least one of the following: The identifier of the AI unit; The identifier of the monitoring sample; The number of the monitoring samples; The number of types of the monitoring samples; Second indication, where the second indication is used to indicate the sensed measurement quantities known to the second device; Third indication, where the third indication is used to indicate the effective time of the monitored sensed measurement quantity; Description information of the sensed measurement quantity corresponding to each type of the monitoring sample; Proportion of the effective sensed measurement quantity in each type of the monitoring sample; Effective time of the set of sensed measurement quantities; Fourth indication, where the fourth indication is used to indicate the monitoring samples associated with the sensed measurement quantity carried in the first data set; Fifth indication, where the fifth indication is used to indicate the number or proportion of the monitoring samples associated with the sensed measurement quantity carried in the first data set.

20. The method according to claim 19, characterized in that, The sensed measurement quantity description information includes at least one of the following: Parameter items of the sensed measurement quantity; The number of parameter items of the sensed measurement quantity; Processing level of the sensed measurement quantity; Information of the sending device for indicating the perceived measurement quantity; Information of the receiving device for indicating the perceived measurement quantity; Source information of the perceived measurement quantity; Perceived measurement quantity link identification information; Signal configuration identification, which is used to indicate the signal corresponding to the perceived measurement quantity; Perceived service information; Data subscription identification; Information for indicating the use of the perceived measurement quantity; Information of the device corresponding to the perceived measurement quantity; Coordinate information of the perceived measurement quantity; Performance index information corresponding to the perceived measurement quantity.

21. The method according to any one of claims 12-20, characterized in that, The method further includes any one of the following: The second device determines the first configuration based on the monitoring requirements and sends the first configuration to the third device; The second device receives the third configuration sent by the third device; The second device receives the second configuration sent by the third device and sends feedback information to the third device, where the feedback information is used to indicate the difference between the first configuration and the second configuration, or to indicate the desired first configuration.

22. The method according to any one of claims 12 - 20, characterized in that, In the case where the first information includes the first data set, the method further includes: The second device performs AI unit inference performance monitoring based on the first information and determines the first index.

23. The method according to any one of claims 12-22, characterized in that, The method further includes any one of the following: The second device adjusts the perceived measurement configuration sent to the first device based on the first index; In the case where the first information includes an indication of the perceived measurement signal sending device or the sensor perception device, the second device sends a perceived measurement adjustment request to the perceived measurement signal sending device or the sensor perception device indicated in the first information based on the first index.

24. The method according to claim 23, wherein The perceived measurement configuration includes at least one of the following: Identification of the desired first data set; Configuration identification of the desired perceived measurement quantity; The fifth interval of the perceived measurement quantity; Quantity threshold of the perceived measurement quantity; Information for indicating the source of the perceived measurement quantity; Information for indicating the category of the perceived measurement quantity; Perception mode of the perceived measurement quantity; Indication of the perception requirement; Configuration information of the first signal; Sending device of the first signal; Receiving device of the first signal; Wherein, the first signal is a signal used for perception.

25. A method for monitoring the inference performance of an AI unit, characterized in that, Includes: The third device sends a third configuration to the first device, and the third configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for AI unit inference performance monitoring.

26. The method according to claim 25, wherein The third configuration includes at least one of the following: AI unit identification; AI unit monitoring purpose identification; The first number, which is used to indicate the number of perceived measurement quantity control groups to be monitored; The first interval, which is related to the sending interval of the first signal, and the first signal is a signal used for perception; The second interval, which is the default effective interval or the baseline of the effective interval to be monitored; The third interval, which is used to indicate the sending interval of the perceived measurement quantity to be monitored; The fourth interval, which is used to indicate the sending interval of the perceived measurement quantity in the perceived measurement quantity control group to be monitored; The first coefficient, which is the ratio of the second interval to the first interval; The second coefficient, where the second coefficient is the ratio of the first interval to the second interval; The third coefficient, where the third coefficient is the ratio of the third interval to the first interval; The fourth coefficient, where the fourth coefficient is the ratio of the first interval to the third interval; The first duration, where the first duration is related to a reference time and a relative duration. The reference time is determined by the time when a perception measurement quantity is obtained, and the relative duration is the duration relative to the time when the perception measurement quantity is obtained; The first indication, which is used to indicate the collection method of the monitoring sample; The indication of the reporting method of the monitoring sample; The indication of the number of the monitoring samples; The first threshold, which is used to indicate the relevant threshold for determining the effective duration of the perception measurement quantity; The indication of the first metric; Information used to indicate the reason for the second device to trigger the AI unit inference performance monitoring.

27. The method according to claim 25, wherein The method further includes: The third device sends perception measurement-related data or a first signal to the first device, and the first signal is used to obtain perception measurement-related data; Wherein, the perception measurement-related data is used by the first device to generate first information, and the first information includes at least one of a first data set and a first metric. The first data set includes the monitoring samples, and the first metric includes the AI unit inference performance monitoring metric.

28. The method according to claim 25, wherein The method further includes any one of the following: The third device determines the third configuration and sends the third configuration to the second device; The third device receives the first configuration sent by the second device and determines the third configuration; The third device sends a second configuration to the second device and receives feedback information sent by the second device. The feedback information is used to indicate the difference between the first configuration and the second configuration, or to indicate the desired first configuration.

29. A monitoring device for the inference performance of an AI unit, which is applied to a first device, is characterized in that The apparatus includes: A first receiving module, configured to receive a first configuration sent by a second device. The first configuration is used to indicate the collection of monitoring samples, and the monitoring samples are samples for AI unit inference performance monitoring; An obtaining module, configured to obtain perception measurement-related data and communication measurement-related data, and generate first information based on the first configuration and based on the perception measurement-related data and the communication measurement-related data; A first sending module, configured to send the first information to the second device; Wherein, the first information includes at least one of a first data set and a first metric. The first data set includes the monitoring samples, and the first metric includes the AI unit inference performance monitoring metric.

30. The device according to claim 29, characterized in that, The obtaining module is further configured to perform any one of the following: Receive perception measurement-related data sent by the second device; Receive a first signal sent by the second device or the third device, and obtain perception measurement-related data according to the first signal; Receive perception measurement-related data sent by the third device, where the perception measurement-related data is obtained by the third device based on sensor device measurement or based on a second signal measurement, or is perception measurement-related data received from other perception devices; Wherein, the first signal or the second signal is a signal used for perception.

31. A monitoring device for the inference performance of an AI unit, applied to a second device, characterized in that, The apparatus includes: A second sending module, configured to send a first configuration to a first device, where the first configuration is used to indicate collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of an AI unit; A second receiving module, configured to receive first information sent by the first device; Wherein, the first information includes at least one of a first data set and a first metric, the first data set includes the monitoring samples, and the first metric includes an AI unit inference performance monitoring metric.

32. The device according to claim 31, characterized in that, The second sending module is further configured to: Determine the first configuration based on a monitoring requirement and send the first configuration to a third device; Alternatively, the second receiving module is further configured to perform any one of the following: Receive a third configuration sent by the third device; Receive a second configuration sent by the third device and send feedback information to the third device, where the feedback information is used to indicate a difference between the first configuration and the second configuration, or is used to indicate a desired first configuration.

33. The device according to claim 31, characterized in that, When the first information includes the first data set, the apparatus further includes: A determination module, configured to monitor the inference performance of the AI unit based on the first information and determine the first metric.

34. The device according to claim 31, characterized in that, The apparatus further includes: An adjustment module, configured to adjust a perception measurement configuration sent to the first device based on the first metric; Alternatively, the second sending module is further configured to: when the first information includes an indication of a perception measurement signal sending device or a sensor perception device, send a perception measurement adjustment request to the perception measurement signal sending device or the sensor perception device indicated in the first information based on the first metric.

35. A monitoring device for the inference performance of an AI unit, which is applied to a third device, is characterized in that The apparatus includes: A third sending module, configured to send a third configuration to a first device, where the third configuration is used to indicate collection of monitoring samples, and the monitoring samples are samples for monitoring the inference performance of an AI unit.

36. The device according to claim 35, characterized in that, The third sending module is further configured to: Send perception measurement related data or a first signal to the first device, where the first signal is used to obtain perception measurement related data; Wherein, the perception measurement related data is used by the first device to generate first information, the first information includes at least one of a first data set and a first metric, the first data set includes the monitoring samples, and the first metric includes an AI unit inference performance monitoring metric.

37. A communication device, characterized in that, Including a processor and a memory, where the memory stores a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the method for monitoring the inference performance of an AI unit according to any one of claims 1-28 are implemented.

38. A readable storage medium, characterized in that, A program or instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the steps of the method for monitoring the inference performance of an AI unit according to any one of claims 1-28 are implemented.

Images