Method and terminal for testing or monitoring artificial intelligence (AI) model for positioning

Abstract

The invention discloses a testing or monitoring method for an artificial intelligence AI model for positioning and a terminal, and belongs to the technical field of communication, and the testing or monitoring method for the artificial intelligence AI model for positioning in the embodiment of the invention comprises the steps that the terminal obtains a reference value and an input value; the terminal executes a first operation, and the first operation comprises one of the following steps: inputting the input value into an AI model which is locally deployed by the terminal and is used for positioning, obtaining an output value of the AI model, taking the output value as a test / monitoring value, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value; and taking the input value as a test / monitoring value, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of communication, and particularly relates to a test or monitoring method of an artificial intelligence (AI) model for positioning and a terminal. BACKGROUND

[0002] With the gradual maturity of artificial intelligence (AI) technology, more and more researches on AI / machine learning (ML) based air interface technology are carried out. For example, an AI / ML model can be used for positioning of a terminal. In order to meet the AI positioning requirement, before deployment of an AI / ML model for positioning, a related technology needs to complete consistency test in a laboratory. However, for a device with AI function that has been deployed in a field, the AI / ML model in the device may become unsuitable for actual positioning requirement. For example, the AI / ML model in the device is updated, the field environment changes, and the like. At this time, it is not known whether the AI / ML model in the device can meet the consistency test requirement. SUMMARY

[0003] Embodiments of the present application provide a test or monitoring method of an artificial intelligence (AI) model for positioning and a terminal, which can test or monitor a deployed AI model for positioning.

[0004] In a first aspect, a test or monitoring method of an artificial intelligence (AI) model for positioning is provided, which is executed by a terminal, and the method comprises:

[0005] The terminal acquires a reference value and an input value;

[0006] The terminal performs a first operation, and the first operation comprises one of the following:

[0007] The input value is input into an AI model for positioning deployed locally by the terminal, an output value of the AI model is acquired, the output value is taken as a test / monitoring value, and a test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value;

[0008] The input value is taken as a test / monitoring value, and a test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value.

[0009] In a second aspect, a test or monitoring method of an artificial intelligence (AI) model for positioning is provided, which is executed by a network side device, and the method comprises:

[0010] The network side device acquires a reference value and an input value;

[0011] The network side device performs a third operation, and the third operation comprises one of the following:

[0012] sending the reference value and the input value to a terminal, the reference value and the input value being used to determine a test / monitoring result of an AI model for positioning locally deployed by the terminal;

[0013] sending the input value to a terminal and receiving an output value from the terminal, the output value being obtained by inputting the input value into an AI model for positioning locally deployed by the terminal, determining a test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the output value;

[0014] inputting the input value into an AI model for positioning locally deployed by the network-side device, obtaining an output value of the AI model, and determining a test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the output value;

[0015] determining a test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the input value.

[0016] In a third aspect, a device for testing or monitoring an artificial intelligence (AI) model for positioning is provided, and includes:

[0017] a first obtaining unit configured to obtain a reference value and an input value;

[0018] a first execution unit configured to perform a first operation, the first operation including one of the following:

[0019] inputting the input value into an AI model for positioning locally deployed by the terminal, obtaining an output value of the AI model, taking the output value as a test / monitoring value, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value;

[0020] taking the input value as a test / monitoring value, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value.

[0021] In a fourth aspect, a device for testing or monitoring an artificial intelligence (AI) model for positioning is provided, and includes:

[0022] a second obtaining unit configured to obtain a reference value and an input value;

[0023] a second execution unit configured to perform a third operation, the third operation including one of the following:

[0024] sending the reference value and the input value to a terminal, the reference value and the input value being used to determine a test / monitoring result of an AI model for positioning locally deployed by the terminal;

[0025] sending the input value to the terminal, and receiving an output value from the terminal, the output value being obtained by inputting the input value into an AI model for positioning locally deployed in the terminal, determining a test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the output value;

[0026] inputting the input value into an AI model for positioning locally deployed in the network-side device, obtaining an output value of the AI model, determining a test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the output value;

[0027] determining a test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the input value.

[0028] In a fifth aspect, a terminal is provided, which includes a processor and a memory, the memory storing programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement steps of the method according to the first aspect.

[0029] In a sixth aspect, a terminal is provided, which includes a processor and a communication interface, and the processor is configured to obtain a reference value and an input value, and perform a first operation, the first operation including one of: inputting the input value into an AI model for positioning locally deployed in the terminal, obtaining an output value of the AI model, and taking the output value as a test / monitoring value, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value; and taking the input value as a test / monitoring value, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value.

[0030] In a seventh aspect, a network-side device is provided, which includes a processor and a memory, the memory storing programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement steps of the method according to the second aspect.

[0031] In an eighth aspect, a network-side device is provided, including a processor and a communication interface, wherein the processor is configured to obtain a reference value and an input value; and perform a third operation, the third operation including one of the following: sending the reference value and the input value to a terminal, the reference value and the input value being used to determine a test / monitoring result of an AI model for positioning locally deployed by the terminal; sending the input value to the terminal and receiving an output value from the terminal, the output value being obtained by inputting the input value into the AI model for positioning locally deployed by the terminal, determining the test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the output value; inputting the input value into an AI model for positioning locally deployed by the network-side device, obtaining an output value of the AI model, determining the test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the output value; and determining the test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the input value.

[0032] In a ninth aspect, a readable storage medium is provided, the readable storage medium storing a program or instructions, the program or instructions being executed by a processor to implement the steps of the method according to the first aspect or the steps of the method according to the second aspect.

[0033] In a tenth aspect, a wireless communication system is provided, including a terminal and a network-side device, the terminal being configured to implement the steps of the method according to the first aspect, and the network-side device being configured to implement the steps of the method according to the second aspect.

[0034] In an eleventh aspect, a chip is provided, the chip including a processor and a communication interface, the communication interface being coupled to the processor, and the processor being configured to run a program or instructions to implement the method according to the first aspect or the method according to the second aspect.

[0035] In a twelfth aspect, a computer program / program product is provided, the computer program / program product being stored in a storage medium, and the computer program / program product being executed by at least one processor to implement the steps of the test or monitoring method of the AI model for positioning according to the first aspect or the steps of the test or monitoring method of the AI model for positioning according to the second aspect.

[0036] In the embodiment of the present application, by obtaining the reference value and the input value, inputting the input value into the AI model for positioning locally deployed by the terminal, obtaining the output value of the AI model, taking the output value as the test / monitoring value, or taking the input value as the test / monitoring value, determining the test / monitoring result of the AI model based on the reference value and the test / monitoring value, the verification or monitoring of the AI model for positioning is realized, thereby improving the reliability of the AI model for positioning deployed in the actual environment, and the performance of AI positioning can be effectively guaranteed. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a block diagram of a wireless communication system to which the embodiment of the present application can be applied;

[0038] Figure 2 is one of flowcharts of a test or monitoring method of an artificial intelligence (AI) model for positioning provided by the embodiment of the present application;

[0039] Figure 3 is another flowchart of a test or monitoring method of an artificial intelligence (AI) model for positioning provided by the embodiment of the present application;

[0040] Figure 4 is one of structural diagrams of a test or monitoring apparatus of an artificial intelligence (AI) model for positioning provided by the embodiment of the present application;

[0041] Figure 5 is another structural diagram of a test or monitoring apparatus of an artificial intelligence (AI) model for positioning provided by the embodiment of the present application;

[0042] Figure 6 is a result diagram of a communication device provided by the embodiment of the present application;

[0043] Figure 7 is a hardware structural diagram of a terminal for implementing the embodiment of the present application;

[0044] Figure 8 is one of hardware structural diagrams of a network side device for implementing the embodiment of the present application;

[0045] Figure 9 is another hardware structural diagram of a network side device for implementing the embodiment of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0047] The terms "first", "second", and the like in the specification and claims of this application are used as identifiers for convenience and are not intended to denote a particular sequential order, unless specifically stated to denote a particular sequential order. It is to be understood that the terms so used in the description and claims are interchangeable under appropriate circumstances and that the embodiments of this application are capable of functioning in other sequences than the one described herein. The first, second, etc. objects are not limited to two or more objects, but can be one object. Furthermore, the term "or" as used in the description and claims is to be interpreted as inclusive, i.e., the inclusion of at least one of a number of things in the examples, whether

[0048] The term "indicate" in the present application can be a direct indication (or explicit indication) or an indirect indication (or implicit indication). The direct indication can be understood as that the sender explicitly informs the receiver of specific information, operations to be performed or requested results, etc. in the indication sent by the sender. The indirect indication can be understood as that the receiver determines the corresponding information according to the indication sent by the sender, or judges and determines the operations to be performed or the requested results according to the judgment result.

[0049] It is worth noting that the techniques described in the embodiments of the present application are not limited to Long Term Evolution (LTE) / LTE-Advanced (LTE-A) systems, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA) or other systems. The terms "system" and "network" in the embodiments of the present application are often used interchangeably, and the described techniques can be used in the above-mentioned systems and radio technologies, as well as in other systems and radio technologies. The following description describes a New Radio (NR) system for example purposes, and NR terminology is used in most of the following description, but these techniques can also be applied to systems other than NR systems, such as 6 thA communication system (5G, 6G).

[0050] Figure 1A block diagram of a wireless communication system to which embodiments of the present application can be applied is shown. The wireless communication system includes a terminal 11 and a network side device 12. The terminal 11 can be a terminal side device such as a mobile phone, a Tablet Personal Computer, a Laptop Computer, a notebook, a Personal Digital Assistant (PDA), a palm PC, a netbook, an Ultra-mobile Personal Computer (UMPC), a Mobile Internet Device (MID), an Augmented Reality (AR) device, a Virtual Reality (VR) device, a robot, a wearable device, a flight vehicle, a Vehicle User Equipment (VUE), a shipboard device, a Pedestrian User Equipment (PUE), a smart home (a home device with a wireless communication function such as a refrigerator, a television, a washing machine, or furniture, etc.), a game console, a Personal Computer (PC), a kiosk, or a self-service machine, etc. The wearable device includes a smart watch, a smart bracelet, a smart earphone, smart glasses, smart jewelry (a smart bracelet, a smart necklace, a smart ring, a smart necklace, a smart anklet, a smart necklace, etc.), a smart wristband, smart clothes, etc. The vehicle-mounted device can also be referred to as a vehicle-mounted terminal, a vehicle-mounted controller, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip, or a vehicle-mounted unit, etc. It should be noted that the specific type of the terminal 11 is not limited in the embodiments of the present application. The network side device 12 can include an access network device or a core network device, wherein the access network device can also be referred to as a Radio Access Network (RAN) device, a radio access network function, or a radio access network unit. The access network device can include a base station, a Wireless Local Area Network (WLAN) Access Point (AP), or a Wireless Fidelity (WiFi) node, etc.The base station can be referred to as a Node B (NB), an evolved Node B (eNB), a next generation Node B (gNB), a New Radio Node B (NR Node B), an access point, a relay station (RBS), a serving base station (SBS), a base transceiver station (BTS), a radio base station, a radio transceiver, a basic service set (BSS), an extended service set (ESS), a home Node B (HNB), a home evolved Node B, a transmission reception point (TRP), or some other suitable terminology in the art, and is not limited to a particular technical terminology, provided that the same technical effect is achieved. It should be noted that in the embodiments of the present application, only the base station in the NR system is taken as an example for introduction, and the specific type of the base station is not limited.

[0051] The core network device can include, but is not limited to, at least one of the following: a core network node, a core network function, a mobility management entity (MME), an access and mobility management function (AMF), a session management function (SMF), a user plane function (UPF), a policy control function (PCF), a policy and charging rules function (PCRF), an edge application server discovery function (EASDF), a unified data management (UDM), a unified data repository (UDR), a home subscriber server (HSS), a centralized network configuration (CNC), a network repository function (NRF), a network exposure function (NEF), a local NEF (L-NEF), a binding support function (BSF), an application function (AF), a location management function (LMF), and the like. It should be noted that only the core network device in the NR system is taken as an example for introduction in the embodiments of the present application, and the specific type of the core network device is not limited.

[0052] In some embodiments, "acquire", "obtain", "get", "receive", "transmit", "bidirectional transmission", "send and / or receive" can be replaced with each other, which can be interpreted as receiving from other subjects, obtaining from protocols, obtaining by oneself, and various meanings such as autonomous implementation.

[0053] In some embodiments, the terms "send", "transmit", "report", "issue", "transmit", "bidirectional transmission", "send and / or receive" can be replaced with each other.

[0054] In some embodiments, "predetermined" and "preset" can be interpreted as being previously stipulated in a protocol or the like, or as being previously set by the device or the like.

[0055] In some embodiments, the data, information, and the like can be obtained in compliance with the laws and regulations of the country where the location is located.

[0056] In some embodiments, the data, information, and the like can be obtained after obtaining the consent of the user.

[0057] Artificial intelligence (AI) is currently widely used in various fields. Integrating artificial intelligence into wireless communication networks significantly improves technical indicators such as throughput, latency, and user capacity, which is an important task for future wireless communication networks. AI modules have various implementation methods, such as neural networks, decision trees, support vector machines, and Bayesian classifiers. The present application takes neural networks as an example for illustration, but does not limit the specific type of AI module. Neural networks are composed of neurons, where a1, a2, … a K are inputs, w is a weight (multiplicative coefficient), b is a bias (additive coefficient), and σ(.) is an activation function. Common activation functions include Sigmoid, tanh, ReLU (Rectified Linear Unit, linear rectifier function, modified linear unit), and the like. The parameters of the neural network are optimized by a gradient optimization algorithm. Gradient optimization algorithms are a class of algorithms that minimize or maximize an objective function (sometimes also called a loss function), and the objective function is often a mathematical combination of model parameters and data. For example, given data X and its corresponding label Y, a neural network model f(.) is constructed. With the model, the predicted output f(x) can be obtained according to the input x, and the difference between the predicted value and the true value (f(x)-Y) can be calculated, which is the loss function. The purpose is to find appropriate W, b to minimize the value of the above loss function. The smaller the loss value, the closer the model is to the true situation.

[0058] The common optimization algorithm is basically based on the BP (error Back Propagation) algorithm. The basic idea of the BP algorithm is that the learning process consists of two processes of forward propagation of signals and backward propagation of errors. During forward propagation, the input sample is transmitted from the input layer to the output layer through the processing of each hidden layer. If the actual output of the output layer does not match the expected output, the backward propagation of errors is entered. Error back propagation is to transmit the output error to the input layer through the hidden layer in a certain form, and distribute the error to all units of each layer, so as to obtain the error signal of each unit, which is used as the basis for correcting the weights of each unit. The process of adjusting the weights of each layer through forward propagation of signals and backward propagation of errors is repeated. The process of continuously adjusting the weights is the learning and training process of the network. This process continues until the error of the network output is reduced to an acceptable level, or until the preset number of learning times is reached.

[0059] The common optimization algorithm includes gradient descent, stochastic gradient descent (SGD), mini-batch gradient descent, momentum method, Nesterov (the name of the inventor, specifically stochastic gradient descent with momentum), adaptive gradient descent (Adagrad), adaptive delta (Adadelta), root mean square prop (RMSprop), adaptive moment estimation (Adam), etc.

[0060] These optimization algorithms, when performing error back propagation, obtain the error / loss from the loss function, take the derivative / partial derivative of the current neuron, add the learning rate, the previous gradient / derivative / partial derivative, etc., to obtain the gradient, and pass the gradient to the previous layer.

[0061] Life cycle management refers to the process of effectively managing and controlling the entire process of AI / ML models from creation to obsolescence. This process includes the following main stages:

[0062] Requirement definition: determine business requirements and clarify the expected functions and effects of the model;

[0063] Data collection: the process of collecting data by network nodes, management entities or UEs for AI / ML model training, data analysis and inference;

[0064] Model training: a process of training an AI / ML model in a data-driven manner (by learning input / output relationship) and obtaining a trained AI / ML model;

[0065] Function / model identification: a process / method of identifying an AI / ML function / model for common understanding between network-side devices and UEs, and information about the AI / ML function / model can be shared during function identification;

[0066] Model delivery / transmission: delivering AI / ML models through an air interface in a manner that is opaque to 3GPP signaling, whether it is parameters of a model structure known to the receiving end or a new model with parameters, delivery can contain a complete model or a partial model;

[0067] Model inference: a process of using a trained AI / ML model to generate a set of outputs from a set of inputs;

[0068] Function / model selection: a process of selecting an AI / ML model for activation among multiple models for the same AI / ML-enabled function;

[0069] Function / model activation: enabling an AI / ML model for a specific AI / ML-enabled function;

[0070] Function / model deactivation: deactivating an AI / ML model for a specific AI / ML-disabled function;

[0071] Function / model switching: deactivating the currently activated AI / ML model and activating a different AI / ML model for a specific AI / ML-enabled function;

[0072] Function / model rollback: deactivating the currently activated AI / ML model and reusing or activating a previous version of the model;

[0073] Function / model supervision: a process of monitoring the performance of AI / ML function / model inference;

[0074] Model upgrade: a process of upgrading model parameters and / or model structure.

[0075] These stages constitute the model lifecycle management process, and effective management can ensure high quality, high efficiency and high reliability of AI / ML models, thereby meeting business needs and improving work efficiency.

[0076] The method of the embodiments of the present application relates to the monitoring stage after model deployment in lifecycle management.

[0077] The positioning technology based on AI model includes the following cases, as shown in Table 1:

[0078] Direct AI / ML positioning:

[0079] (First priority) Scenario 1: UE-based positioning, UE-side model, direct AI / ML positioning;

[0080] (Second priority) Scenario 2b: UE-assisted / LMF positioning based on LMF side model, direct AI / ML positioning;

[0081] (First priority) Scenario 3b: NG-RAN node-assisted positioning with LMF-side model, direct AI / ML positioning;

[0082] Direct AI / ML positioning means that the model directly infers the location of the target positioning terminal, and the AI / ML model outputs location information.

[0083] AI / ML assisted positioning:

[0084] (Second priority) Case 2a: UE-assisted / LMF-based positioning using UE-side model, AI / ML-assisted positioning

[0085] (First priority) Scenario 3a: NG-RAN node-assisted positioning with gNB-side model, AI / ML-assisted positioning.

[0086] AI / ML-assisted positioning refers to the model inferring the intermediate features of the target positioning terminal. The AI / ML model outputs the intermediate features, among which the location of the target terminal can be calculated based on the intermediate features.

[0087] Table 1 Positioning technology types of AI models

[0088]

[0089] To meet AI positioning requirements, relevant technologies need to complete consistency testing in the laboratory before deploying the AI / ML models used for positioning. However, for devices with AI capabilities that have already been deployed in the field, the AI / ML models in the devices may become unsuitable for actual positioning needs. For example, if the AI / ML models in the devices are updated or the field environment changes, it is unknown whether the AI / ML models in the devices can meet the consistency testing requirements.

[0090] Optionally, consistency testing is an evaluation method for verifying the consistency and stability of the output of the model under different conditions. Consistency testing aims to check whether the model produces consistent output results under different conditions or environments for the same input data. When performing consistency testing, the following aspects are usually considered: input variation: by changing some characteristics or attributes of the input data, test whether the output of the model remains consistent; environmental variation: the model may be sensitive to environmental changes, so test whether the output of the model is consistent under different environmental conditions; data set segmentation: divide the data set into training set and test set, and test whether the output of the model is consistent on different subsets; model version comparison: if there are multiple versions of the model, they can be compared to ensure that they produce consistent output under the same input.

[0091] The method for testing or monitoring an artificial intelligence (AI) model for positioning provided by the embodiments of the present application will be described in detail below in combination with the accompanying drawings, some embodiments and application scenarios.

[0092] Figure 2 is one of the flowcharts of the method for testing or monitoring an artificial intelligence (AI) model for positioning provided by the embodiments of the present application. As shown in Figure 2 , the method provided by the embodiments of the present application comprises:

[0093] Step 201: The terminal acquires a reference value and an input value.

[0094] It should be noted that the reference value in the embodiments of the present application, as a reference value for testing or monitoring the AI model, can be reference position information, reference intermediate quantity, reference output data set, or reference input data set.

[0095] The AI model described in the embodiments of the present application is an AI model for positioning, i.e., participating in AI positioning. The output of the AI model can be position information or an intermediate quantity. The definition of the intermediate quantity will be described below.

[0096] The AI model described in the embodiments of the present application can also be referred to as an AI unit, an AI model, an ML (machine learning) model, an ML unit, an AI structure, an AI function, an AI feature, a machine learning model, a neural network, a neural network function, a neural network function, etc., or the AI model can also refer to a processing unit capable of implementing a specific algorithm, formula, processing flow, capability, etc. related to AI, or the AI model can be a processing method, algorithm, function, module or unit for a specific data set, or the AI model can be a processing method, algorithm, function, module or unit running on AI / ML related hardware such as a GPU (Graphics Processing Unit), NPU (Neural Processing Unit), TPU (Tensor Processing Unit), ASIC (Application-Specific Integrated Circuit), etc. The embodiments of the present application do not make specific limitations. Optionally, the specific data set includes input and / or output of the AI model.

[0097] The input value can be understood as a model input for testing or monitoring the AI model, which can be measurement information or model input information processed from the measurement information. In the embodiments of the present application, the measurement information generally refers to signal / channel measurement information, and the measurement information can also be referred to as measurement quantity, measurement result, etc.

[0098] Step 202, the terminal performs a first operation, and the first operation includes one of the following:

[0099] The input value is input into the AI model for positioning deployed locally by the terminal, the output value of the AI model is obtained, the output value is taken as a test / monitoring value, and based on the reference value and the test / monitoring value, a test / monitoring result of the AI model is determined;

[0100] The input value is taken as a test / monitoring value, and based on the reference value and the test / monitoring value, a test / monitoring result of the AI model is determined.

[0101] The test / monitoring value includes a test value or a monitoring value. The test / monitoring result includes a test result or a monitoring result.

[0102] It can be understood that after the terminal obtains the reference value and the input value, there are two ways to test / monitor the performance of the AI model, one way is to verify the inference ability of the AI model, and the other way is to verify whether the AI model can adapt to the current environment, specifically to verify whether the input of the model can adapt to the current environment.

[0103] The verifying the inference capability of the AI model includes: inputting the input value into the AI model for positioning deployed locally on the terminal, obtaining an output value of the AI model, taking the output value as a test / monitoring value, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value.

[0104] The verifying the input of the AI model includes: taking the input value as a test / monitoring value, the reference value belonging to a reference input data set, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value.

[0105] It should be noted that the test or monitoring method for the AI model for positioning provided in the embodiments of the present application is not only applicable to performance test or performance monitoring after model deployment, but also can be used for performance test or performance monitoring after model generation, and test before model deployment. For example, the test result evaluation method and the test index in the present application can be used for consistency test completed in a laboratory.

[0106] The test or monitoring method for the AI model for positioning provided in the embodiments of the present application realizes the verification or monitoring of the AI model for positioning by obtaining the reference value and the input value, inputting the input value into the AI model for positioning deployed locally on the terminal, obtaining an output value of the AI model, taking the output value as a test / monitoring value, or taking the input value as a test / monitoring value, determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value, thereby improving the reliability of the AI model for positioning in actual environment, and effectively guaranteeing the performance of AI positioning.

[0107] It should be noted that each embodiment of the present application can be freely combined, the order can be changed or executed alone, and does not need to rely on or depend on a fixed execution order.

[0108] In some embodiments, the reference value and the input value can be one of the following a) to d):

[0109] a) the reference value is reference position information, and the input value is measurement information or model input information obtained by processing the measurement information;

[0110] It can be understood that if the output of the AI model is position information, the terminal obtains reference position information and measurement information or model input information obtained by processing the measurement information.

[0111] The reference position information refers to position information as a reference output. The position information can be rectangular coordinates, polar coordinates, latitude and longitude information, relative position, absolute position, etc., which are not limited by the present application.

[0112] The measurement information generally refers to signal / channel measurement information, and can be replaced by the concepts of measurement quantity, measurement result, etc.

[0113] In the embodiments of the present application, the measurement information includes at least one of the following: power, time delay, phase, channel impulse response (CIR), power delay profile (PDP), delay profile (DP).

[0114] Optionally, the power includes received power, received power of a target path in multipath, signal-to-noise ratio, etc. The time delay includes received time delay, sending time delay, round-trip time delay, etc. The phase includes phase offset, phase delay, phase difference, etc. The channel impulse response includes impulse response, frequency response, etc.

[0115] The signal can pass through multiple different paths in the transmission process to reach the receiving end, and each path corresponds to different propagation delays. The power delay profile describes the distribution of signal power at different delay times. The delay profile reflects the distribution of arrival time delays of the received signal on different propagation paths. In actual measurement or testing, it is usually a discrete value spectrum.

[0116] Optionally, processing the measurement information refers to mathematical processing such as normalization and averaging of the measurement information.

[0117] In some embodiments, the terminal inputs the input value to the AI model locally deployed on the terminal, obtains the output value of the AI model, the output value of the AI model being the position information inferred by the AI model according to the input value, and compares the output value with the reference position information as the test / monitoring value, to determine the test / monitoring result of the AI model.

[0118] In some embodiments, the terminal obtains the input value, inputs the input value to the AI model locally deployed on the terminal, obtains the output value of the AI model, and sends the output value to the network side device, and the network side device compares the output value with the reference position information as the test / monitoring value, to determine the test / monitoring result of the AI model.

[0119] When the AI model is deployed on the network side device, the network side device determines the test / monitoring result of the AI model according to the reference position information and the output value inferred by inputting the input value to the AI model.

[0120] In some embodiments, the reference position information and the input value are obtained in the following manner:

[0121] The terminal receives first information from the network side device, the first information including reference position information and an input value, the reference position information being position information of a first communication device selected by the network side device as a position known device, and the input value being measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

[0122] In the embodiments of the present application, the first communication device is a position known communication device, that is, the position of the first communication device is known to the network side device. The first communication device includes a positioning reference unit (PRU) or other communication devices supporting channel / signal measurement.

[0123] Optionally, the network side device selects a target PRU as the first communication device from a plurality of PRUs around the terminal, the target PRU reports its position information and measurement information to the network side device, and the network side device sends the position information of the target PRU as the reference position information and the measurement information of the target PRU as the input value to the terminal.

[0124] The method provided in the embodiments of the present application is suitable for the case where the output of the AI model is position information, the input value is input into the AI model for positioning locally deployed in the terminal by acquiring the reference position information and the input value, the output value of the AI model is acquired, the output value is used as the test / monitoring value, the test / monitoring result of the AI model is determined based on the reference position information and the test / monitoring value, the inference ability of the AI model for positioning is verified or monitored, and thus the reliability of the AI model for positioning deployed in the actual environment is improved, and the performance of AI positioning can be effectively guaranteed.

[0125] b) the reference value is a reference intermediate quantity, the reference intermediate quantity being intermediate information for determining the position of the terminal, the reference intermediate quantity being of the same type as the output value of the AI model, and the input value being measurement information or model input information obtained by processing the measurement information;

[0126] It can be understood that if the output of the AI model is an intermediate quantity, the reference intermediate quantity can be acquired, and the measurement information or the model input information obtained by processing the measurement information can be used as the input value to test or monitor the AI model.

[0127] The reference intermediate quantity is of the same type as the output value of the AI model. For example, if the type of the output value of the AI model is RTOA, the type of the reference intermediate quantity should also be RTOA.

[0128] The reference intermediate quantity can also be referred to as a reference intermediate feature, and is intermediate information that can determine the terminal position.

[0129] In some embodiments, the terminal inputs, as input values, measurement information or model input information obtained by processing the measurement information into an AI model deployed locally by the terminal, obtains output values of the AI model, and determines a test / monitoring result of the AI model based on the output values and the reference intermediate quantity.

[0130] Optionally, the terminal obtains input values, the terminal inputs the input values into an AI model deployed locally by the terminal, obtains output values of the AI model, and sends the output values to a network side device, and the network side device can implement test or monitoring of the AI model according to the output values and the reference intermediate quantity.

[0131] In some embodiments, the reference intermediate quantity includes at least one of the following:

[0132] a line of sight or non-line of sight (LOS / NLOS) indicator;

[0133] a reference signal time difference (RSTD);

[0134] a relative time of arrival (RTOA);

[0135] a receive-transmit (Rx-Tx) time difference;

[0136] a time of arrival (TOA);

[0137] an angle of arrival (AoA);

[0138] an angle of departure (AoD);

[0139] a reference signal received power (RSRP);

[0140] a reference signal received path power (RSRPP).

[0141] In some embodiments, the reference intermediate quantity and the input values are obtained in the following manner:

[0142] receiving a reference intermediate quantity and an input value from the network side device;

[0143] The reference intermediate quantity is obtained based on reference position information reported by a first communication device with known position and second information, and the second information is used to assist in determining the reference intermediate quantity, or the reference intermediate quantity is determined by the first communication device and reported to the network side device;

[0144] The input value is measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

[0145] It can be understood that the terminal receives the reference intermediate quantity and the input value sent by the network side device, and the present application embodiment provides two determination modes of the reference intermediate quantity:

[0146] 1) Direct determination mode, determined by the first communication device;

[0147] 2) Indirect determination mode, that is, determined by the network side device based on the reference position information of the first communication device and second information, wherein the second information is used to assist in determining the reference intermediate quantity.

[0148] Optionally, the network side device receives the reference intermediate quantity and the measurement information reported by the first communication device, and sends the reference intermediate quantity and the measurement information to the terminal, wherein the reference intermediate quantity is determined by the first communication device.

[0149] Optionally, the network side device receives the reference position information and the measurement information reported by the first communication device, determines the reference intermediate quantity based on the reference position information and the second information, and sends the reference intermediate quantity and the measurement information to the terminal.

[0150] In the present application embodiment, the first communication device is a communication device with known position, that is, the position of the first communication device is known to the network side device. Optionally, the first communication device is selected by the network side device, and the first communication device includes a positioning reference unit (PRU) or other communication device supporting channel / signal measurement.

[0151] Optionally, the network side device selects a target PRU as the first communication device from a plurality of PRUs around the terminal, the target PRU reports its position information and measurement information to the network side device, the network side device takes the position information of the target PRU as the reference position information, determines the reference intermediate quantity based on the reference position information and the second information, and sends the measurement information of the PRU to the terminal as the input value.

[0152] In some embodiments, the second information comprises at least one of: a location of a transmission and reception point (TRP) currently participating in AI positioning, historical data of measurement information, historical data of AI model output values, a historical location of the terminal, a historical location of the TRP, and a historical location of the first communication device.

[0153] For example, the second information is a location of a transmission and reception point (TRP) currently participating in AI positioning. Based on the location information of the target PRU and the location of the TRP currently participating in AI positioning, the network side device calculates intermediate quantities when the target PRU performs positioning using the TRPs currently participating in AI positioning by using geometric relationships in mathematics. These intermediate quantities are the reference intermediate quantities. The target PRU measures downlink positioning reference signals (DL-PRS) transmitted by the TRPs currently participating in AI positioning to obtain measurement information and sends the measurement information to the network side device. Optionally, the network side device sends the reference intermediate quantities and the measurement information to the terminal. The terminal takes the reference intermediate quantities as reference values and inputs the measurement information into the AI model to obtain intermediate quantities output by the AI model. The terminal takes the intermediate quantities output by the AI model as test / monitoring values and compares them with the reference intermediate quantities to determine the test / monitoring result of the AI model. Alternatively, the network side device sends the measurement information to the terminal. The terminal inputs the measurement information into the AI model to obtain intermediate quantities output by the AI model and sends the intermediate quantities output by the AI model as test / monitoring values to the network side device. The network side device compares the intermediate quantities output by the AI model with the reference intermediate quantities to determine the test / monitoring result of the AI model.

[0154] The second information can also be other auxiliary data, including historical data of measurement information, historical data of AI model output values, a historical location of the terminal, a historical location of the TRP, and / or a historical location of the first communication device.

[0155] The method provided by the embodiments of the present application is suitable for the case where the output of the AI model is intermediate quantities. By obtaining reference intermediate quantities and input values, the input values are input into the AI model for positioning deployed locally in the terminal to obtain intermediate quantities output by the AI model. Based on the reference intermediate quantities and the intermediate quantities output by the AI model, the test / monitoring result of the AI model is determined. This realizes verification or monitoring of the inference ability of the AI model for positioning, thereby improving the reliability of the AI model for positioning deployed in an actual environment and effectively ensuring the performance of AI positioning.

[0156] c) the reference value belongs to a reference output data set, and the input value belongs to a reference input data set;

[0157] It can be understood that in the embodiment, the AI model can execute the AI function according to the reference input data set, obtain the corresponding model output, take the model output as the test / monitoring value, and based on the reference output data set and the test / monitoring value, test or monitor the AI model.

[0158] The reference output data set includes reference values of the same type as the output values of the AI model.

[0159] The reference output data set or the reference input data set is autonomously generated by the terminal, autonomously generated by the network side device, generated by the first device, or predefined by a protocol.

[0160] The first device can be a server or other device other than the terminal and the network side device.

[0161] Optionally, one AI model can correspond to one or more groups of reference data sets, and one group of reference data sets can correspond to one or more AI models.

[0162] Optionally, in the case where the AI model is deployed on the terminal side, the terminal receives the reference data set from the network side device or the first device, the reference data set includes the parameter input data set and the reference output data set, the terminal takes the reference input data set as the model input, takes the model output at this time as the test / monitoring value, and compares the test / monitoring value with the reference output data set to determine the test / monitoring result of the AI model.

[0163] Optionally, in the case where the AI model is deployed on the terminal side, the terminal generates or predefines the reference input data set and the reference output data set according to the terminal, the terminal takes the reference input data set as the model input, takes the model output at this time as the test / monitoring value, and compares the test / monitoring value with the reference output data set to determine the test / monitoring result of the AI model.

[0164] Optionally, in the case where the AI model is deployed on the terminal side, the terminal receives the reference input data set from the network side device, the terminal takes the reference input data set as the model input, takes the model output at this time as the test / monitoring value, and reports the test / monitoring value to the network side device, the network side device compares the test / monitoring value with the reference output data set to determine the test / monitoring result of the AI model.

[0165] The method provided by the embodiments of the present application is applicable to the case that the AI model can perform an AI function according to a reference data set. The reference input data set is input into the AI model for positioning deployed locally in the terminal, the output value of the AI model is obtained, and the test / monitoring result of the AI model is determined based on the reference output data set and the output value of the AI model, so as to verify or monitor the inference ability of the AI model for positioning, thereby improving the reliability of the AI model for positioning deployed in the actual environment and effectively ensuring the performance of AI positioning.

[0166] d) The reference value belongs to the reference input data set, and the input value is measurement information or model input information obtained by processing the measurement information.

[0167] In order to verify whether the AI model can adapt to the current environment, the embodiments of the present application propose to compare the measurement information or model input information obtained by processing the measurement information as model input with the reference input data set, so as to test / monitor the input of the AI model.

[0168] The terminal can obtain the measurement information by actual measurement, for example, by measuring the downlink positioning reference signal transmitted by the TRP of the current reference AI positioning.

[0169] Optionally, in the case that the AI model is deployed on the terminal side, the terminal takes the measurement information or model input information obtained by processing the measurement information as test / monitoring value, and determines the test / monitoring result of the AI model based on the reference input data set and the test / monitoring value.

[0170] Optionally, in the case that the AI model is deployed on the terminal side, the terminal can also report the measurement quantity as model input to the network side device, and the network side device takes the measurement quantity as model input reported by the terminal as test / monitoring value, and determines the test / monitoring result of the AI model based on the reference input data set and the test / monitoring value. Here, the measurement quantity as model input includes measurement information or model input information obtained by processing the measurement information.

[0171] In some embodiments, the reference input data set is generated autonomously by the terminal, or is generated autonomously by the network side device, or is generated by a first device, or is predefined by a protocol.

[0172] The first device can be a server, or a device other than the terminal and the network side device.

[0173] The method provided in the embodiments of the present application is applicable to a case where the AI model can perform an AI function according to a reference data set, and by obtaining a reference input data set and an input value, a test / monitoring result of the AI model is determined, thereby verifying or monitoring whether the input of the AI model for positioning can adapt to the current environment, improving the reliability of the AI model for positioning in the actual environment, and effectively ensuring the performance of AI positioning.

[0174] In some embodiments, the test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value, including:

[0175] The difference between the reference value and the test / monitoring value of the first number of paths or samples is calculated.

[0176] The test / monitoring result of the AI model is determined based on the difference between the reference value and the test / monitoring value of the first number of paths or samples.

[0177] The first number is greater than or equal to 1.

[0178] In the embodiments of the present application, the AI model can be tested or monitored once or multiple times.

[0179] If the number of times of testing or monitoring the AI model is 1, the test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value, including:

[0180] The difference between the reference value and the test / monitoring value is calculated, and compared with a predefined accuracy requirement. If the predefined accuracy requirement is met, it is determined that the AI model is applicable, and if the predefined accuracy requirement is not met, it is determined that the AI model is not applicable.

[0181] In order to improve the accuracy of the test / monitoring, the AI model can also be tested or monitored multiple times.

[0182] Optionally, when the number of times is greater than 1, the average value of the difference between the test / monitoring value and the reference value of each execution is compared with the predefined accuracy, or the ratio of the number of times of evaluating the model as applicable to the total evaluation times needs to be greater than a first threshold value (for example, a certain percentage, for example, 90%), or the number of times of evaluating the model as applicable is greater than a predefined evaluation number (for example, 4), at this time, the test / monitoring result of the AI model is applicable, otherwise, it is not applicable.

[0183] In each execution of the test / monitoring, the difference between the reference value and the test / monitoring value of the first number of paths or samples can be calculated, and the test / monitoring result of the AI model is determined based on the difference between the reference value and the test / monitoring value of the first number of paths or samples, thereby improving the accuracy of testing or monitoring the AI model.

[0184] The method provided by the embodiments of the present application can improve the accuracy of testing / monitoring the AI model by calculating the difference between the reference values and the test / monitoring values of the first number of paths or samples, determining the test / monitoring result of the AI model based on the difference between the reference values and the test / monitoring values of the first number of paths or samples.

[0185] In some embodiments, the calculating the difference between the reference values and the test / monitoring values of the first number of paths or samples comprises at least one of the following:

[0186] calculating the Euclidean distance between the vector composed of the reference values of the first number of paths or samples and the vector composed of the test / monitoring values;

[0187] calculating the cosine similarity between the vector composed of the reference values of the first number of paths or samples and the vector composed of the test / monitoring values;

[0188] calculating the root mean square error between the vector composed of the reference values of the first number of paths or samples and the vector composed of the test / monitoring values;

[0189] calculating the difference between the reference values and the test / monitoring values of the first number of paths or samples;

[0190] calculating the average of the absolute values of the difference between the reference values and the test / monitoring values of the first number of paths or samples;

[0191] calculating the root mean square error of the difference between the reference values and the test / monitoring values of the first number of paths or samples;

[0192] calculating the straight line distance between the reference values and the test / monitoring values of the first number of paths or samples;

[0193] calculating the average of the straight line distance between the reference values and the test / monitoring values of the first number of paths or samples;

[0194] calculating the root mean square of the straight line distance between the reference values and the test / monitoring values of the first number of paths or samples.

[0195] The method provided by the embodiments of the present application can calculate the difference between the reference values and the test / monitoring values of the first number of paths or samples in the form of vectors, for example, calculating the Euclidean distance, cosine similarity, root mean square error, etc. between the vector composed of the reference values of the first number of paths or samples and the vector composed of the test / monitoring values.

[0196] Optionally, the difference between the first quantity path or the sample reference value and the test / monitoring value can be calculated in a single value manner, for example, the difference between the first quantity path or the sample reference value and the test / monitoring value, the average value of the absolute value of the difference, or the root mean square error of the difference.

[0197] Optionally, the difference between the first quantity path or the sample reference value and the test / monitoring value can be calculated in a straight line distance manner, for example, the straight line distance between the first quantity path or the sample reference value and the test / monitoring value, the average value of the straight line distance, or the root mean square of the straight line distance.

[0198] The method provided by the embodiments of the application realizes flexibility of difference calculation.

[0199] In some embodiments, the determination of the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first quantity path or sample comprises:

[0200] In the case that the difference between the reference value and the test / monitoring value of the first quantity path or sample meets the predefined accuracy requirement, the test / monitoring result of the AI model is determined to be applicable.

[0201] In the case that the difference between the reference value and the test / monitoring value of the first quantity path or sample does not meet the predefined accuracy requirement, the test / monitoring result of the AI model is determined to be inapplicable.

[0202] It can be understood that the difference between the reference value and the test / monitoring value of the first quantity path or sample meeting the predefined accuracy requirement indicates that the inference ability of the current AI model can meet the current positioning environment, or that the current AI model can adapt to the current positioning environment, that is, it can be determined that the current AI model is applicable.

[0203] Optionally, the difference between the reference value and the test / monitoring value of the first quantity path or sample meeting the predefined accuracy requirement comprises:

[0204] In the case that the number of executions is equal to or greater than 1, the average value of the difference between the test / monitoring value and the reference value of each execution is compared with the predefined accuracy, or the ratio of the number of times of single evaluation of the model as applicable to the total evaluation times is greater than a first threshold value, or the number of times of single evaluation of the model as applicable is greater than a predefined evaluation quantity, and then the test / monitoring result of the AI model is determined to be applicable.

[0205] The difference between the reference value and the test / monitoring value of the first number of paths or samples does not meet the predefined accuracy requirement, indicating that the inference ability of the current AI model cannot meet the current positioning environment, or indicating that the current AI model cannot adapt to the current positioning environment, i.e., it can be determined that the current AI model is not applicable.

[0206] In some embodiments, the predefined accuracy requirement is applicable to a specified condition. It can be understood that the predefined accuracy requirement is related to the specified condition and needs to be applicable under the specified condition. The predefined accuracy requirement can be different under different specified conditions.

[0207] Optionally, the specified condition includes at least one of the following: signal-to-interference-and-noise ratio, signal-to-noise ratio, channel strength, noise strength, channel model, reference signal bandwidth, reference signal configuration, subcarrier spacing, carrier frequency, frequency band information, duplexing mode, TRP configuration, UE configuration, AI model structure, AI model complexity, and AI quantization information.

[0208] It can be understood that the predefined accuracy requirement can be determined according to at least one of the following: signal-to-noise ratio, channel strength, noise strength, channel model, reference signal bandwidth, reference signal configuration, subcarrier spacing, carrier frequency, frequency band information, duplexing mode, TRP configuration, UE configuration, AI model structure, AI model complexity, and AI quantization information.

[0209] The method provided by the embodiments of the present application defines an evaluation criterion for judging whether the AI model is applicable. The evaluation criterion can also be applicable to the evaluation or test of the AI model before deployment. In the test, the network side device is replaced by a test equipment (TE).

[0210] The following further illustrates how to determine the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first number of paths or samples through several specific examples.

[0211] Example 1: The output of the AI model is a position

[0212] When the output of the AI model is a position, and the first information of the PRU available around the UE is known by the NW (LMF), the evaluation criterion adopted is the straight-line distance between them. The position expression is not limited to rectangular coordinates, polar coordinates, relative positions, absolute positions, etc.

[0213] For example, when the position output by the AI model is (x1, y1), and the corresponding reference position is (x0, y0);

[0214] 1) When the AI model is deployed at the UE side: the LMF sends the reference position (x0, y0) to the UE, and the UE takes the reference position (x0, y0) and the position (x1, y1) inferred by the UE's own model as the reference value and the test / monitoring value respectively, and tests the applicability of the current model deployment according to the pre-defined evaluation criteria; or, the UE sends the position (x1, y1) inferred by the UE's own model to the LMF, and the LMF side takes the reference position (x0, y0) and the position (x1, y1) sent by the UE's own model as the reference value and the test / monitoring value respectively, and tests the applicability of the current model according to the pre-defined evaluation criteria;

[0215] 2) When the AI model is deployed at the LMF side: the LMF side takes the selected reference position (x0, y0) and the position (x1, y1) inferred by the model as the reference value and the test / monitoring value respectively, and tests the applicability of the current model according to the pre-defined evaluation criteria;

[0216] The number of evaluations required by the pre-defined evaluation criteria is N. The criterion for a single evaluation is: d1 = √((x0-x1)2+(y0-y1)2).

[0217] If N = 1, compare d1 with the pre-defined accuracy d0, and if d1 is less than or less than or equal to d0, the model is applicable;

[0218] If N is greater than 1, perform the above process multiple times to obtain (d1, d2, …, dN), and compare Σdn / N with d0, if it is less than or less than or equal to d0, the model is applicable; or, d1, d2, …, dN are compared with d0 respectively, when the number of single model applications N1 (N1 ≤ N) is greater than or greater than or equal to the pre-defined N0 (N0 ≤ N), the model is applicable; or, when P1 = N1 / N is greater than or greater than or equal to the pre-defined P0, the model is applicable.

[0219] Example 2, the output of the AI model is an intermediate quantity, and the intermediate quantity is RSTD

[0220] When the output of the AI model is an intermediate quantity, the NW (LMF) infers and calculates the reference value of the intermediate quantity through the position of the TRP and the first information (including the position information and measurement information of the PRU whose position is known), for example, when the intermediate quantity is RSTD, the model output is RSTD1, and the corresponding reference value is RSTD0;

[0221] 1) When the AI model is deployed at the UE side: the UE sends the model output RSTD1 to the LMF, the LMF tests the applicability of the current model deployment according to the pre-defined evaluation criteria, according to RSTD1 and the corresponding RSTD0 as test / monitoring value and reference value respectively, or the UE receives RSTD0 sent by the LMF, the UE infers RSTD1 according to the measurement information, RSTD1 and RSTD0 as test / monitoring value and reference value respectively, the UE tests the applicability of the current model deployment according to the pre-defined evaluation criteria;

[0222] 2) When the AI model is deployed at the gNB or LMF side: the LMF side tests the applicability of the current model deployment according to the pre-defined evaluation criteria, according to the output RSTD1 and the corresponding RSTD0 as test / monitoring and reference value respectively.

[0223] The number of evaluations required by the pre-defined evaluation criteria is N. The criterion for a single evaluation is ΔRSTD1 = abs(RSTD1-RSTD0).

[0224] When N = 1, ΔRSTD1 is compared with the pre-defined accuracy ΔRSTD0, and ΔRSTD1 is less than or equal to ΔRSTD0, then the model is applicable.

[0225] When N is greater than 1, the above process is performed multiple times to obtain (ΔRSTD1, ΔRSTD2, …, ΔRSTDN), and ΣΔRSTDn / N is compared with ΔRSTD0, if less than or equal to ΔRSTD0, then the model is applicable; or ΔRSTD1, ΔRSTD2, …, ΔRSTDN are compared with ΔRSTD0 respectively, when the single model applicability times N1 (N1 ≤ N) is greater than or equal to the pre-defined N0 (N0 ≤ N), then the model is applicable; or when P1 = N1 / N is greater than or equal to the pre-defined P0, the model is applicable;

[0226] The above RSTD can be replaced by TOA, Rx-Tx time difference, RTOA, angle measurement quantity, or power measurement quantity.

[0227] In some embodiments, in the case where the reference value is a reference intermediate quantity and the reference intermediate quantity is a line of sight / non-line of sight indicator (LOS / NLOS indicator), the first probability under the specified condition is taken as the reference accuracy;

[0228] The difference between the reference value and the test / monitoring value of the first number of paths or samples meets the pre-defined accuracy requirement, which includes:

[0229] A second probability that the reference value and the test / monitoring value under the first number of paths or samples are different is calculated;

[0230] In a case where the first probability represents a maximum mismatch probability of the LOS / NLOS indicator, and the second probability is less than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement.

[0231] Alternatively,

[0232] A third probability that the reference value and the test / monitoring value are the same under the first number of paths or samples is calculated.

[0233] In a case where the first probability represents a minimum match probability of the LOS / NLOS indicator, and the third probability is greater than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement.

[0234] It should be noted that the LOS / NLOS indicator can be an indication domain indicating LOS or NLOS through different values. Alternatively, the value of the LOS / NLOS indicator is 1 when indicating LOS, and 0 when indicating NLOS. Alternatively, the value of the LOS / NLOS indicator is a hard value.

[0235] The result (1 or 0) of the AI model inference is taken as the test / monitoring value, and the reference value LOS / NLOS indicator (1 or 0) is compared to calculate the probability that the two are the same or different under the first number of paths or samples, compared with the first probability.

[0236] If the first probability represents a maximum mismatch probability of the LOS / NLOS indicator, the probability that the reference value and the test / monitoring value are different under the first number of paths or samples is calculated. If the probability that the two are different under the first number of paths or samples is less than the first probability, or less than or equal to the first probability, it indicates that the difference between the reference value and the test / monitoring value under the first number of paths or samples meets the predefined accuracy requirement, that is, the measurement / monitoring result of the current AI model is applicable.

[0237] If the first probability represents a minimum match probability of the LOS / NLOS indicator, the probability that the reference value and the test / monitoring value are the same under the first number of paths or samples is calculated. If the probability that the two are the same under the first number of paths or samples is greater than the first probability, or greater than or equal to the first probability, it indicates that the difference between the reference value and the test / monitoring value under the first number of paths or samples meets the predefined accuracy requirement, that is, the measurement / monitoring result of the current AI model is applicable.

[0238] The following further illustrates, by an example, how to determine the test / monitoring result of the AI model based on the difference between the reference value and test / monitoring value of the first quantity path or sample in the case that the reference value is a reference intermediate quantity and the reference intermediate quantity is a line-of-sight / non-line-of-sight indicator (LOS / NLOS indicator).

[0239] Example 3, the output of the AI model is an intermediate quantity, and the intermediate quantity is a LOS / NLOS indicator

[0240] When the output of the AI model is an intermediate quantity, the NW (LMF) infers and calculates the reference value of the intermediate quantity through the position of the TRP and the first information (including the position information and measurement information of the PRU whose position is known). When the intermediate quantity is a LOS / NLOS indicator hard value, the model output is a LOS / NLOS indicator hard value1, and the corresponding reference value is a LOS / NLOS indicator hard value0.

[0241] 1) When the AI model is deployed on the UE side: the UE sends the AI model output LOS / NLOS indicator hard value1 to the LMF, and the LMF takes the LOS / NLOS indicator hard value1 and the corresponding LOS / NLOS indicator hard value0 as the test / monitoring value and the reference value respectively, and tests the applicability of the current model deployment according to the pre-defined evaluation criteria, or the LMF sends the LOS / NLOS indicator hard value0 to the UE, and the UE takes the LOS / NLOS indicator hard value1 and the LOS / NLOS indicator hard value0 as the test / monitoring value and the reference value respectively, and tests the applicability of the current model deployment according to the pre-defined evaluation criteria.

[0242] 2) When the AI model is deployed on the LMF side: the LMF takes the output LOS / NLOS indicator hard value1 and the corresponding LOS / NLOS indicator hard value0 as the test / monitoring and reference values respectively, and tests the applicability of the current model deployment according to the pre-defined evaluation criteria.

[0243] The pre-defined single evaluation criteria: M is the first quantity corresponding to the path or sample, and 1 to M test values value1 m and the reference value value0 m Compare (1≤m≤M) to calculate value1m is not equal to value0 m The number M1, or the equal number M1', calculates p1=M1 / M or p1'=M1' / M (p1'=1-p1).

[0244] When N=1, compare p1 or p1' with the predefined precision p0, if p1 is less than or less than or equal to p0 (or p1' is greater than or greater than or equal to p0), the model is applicable;

[0245] When N is greater than 1, the above process is performed multiple times to obtain (p1, p2, …, pN), and Σpn / N is compared with p0, if less than or less than or equal to p0, the model is applicable; or p1, p2, …, pN are compared with p0 respectively, when the single model applicable times N1 (N1≤N) is greater than or greater than or equal to the predefined N0 (N0≤N), the model is applicable; or P1=N1 / N is greater than or greater than or equal to the predefined P0, the model is applicable;

[0246] The process for p1' is similar, which is not described here.

[0247] In particular, when performing pre-deployment testing, the reference value LOS / NLOS indicator hard value0 test equipment TE is known or directly provided by the TE to the device under test (DUT);

[0248] The method provided by the application gives a method for judging whether the AI model is applicable when the reference value is a reference intermediate quantity and the reference intermediate quantity is a line-of-sight / non-line-of-sight indicator, realizes the verification or monitoring of whether the AI model for positioning can adapt to the current environment, thereby improving the reliability of the AI model for positioning in the actual environment, and effectively guarantees the performance of AI positioning.

[0249] In some embodiments, in the case where the reference value belongs to a reference input data set and the test / monitoring value is the input value, the calculation of the difference between the reference value and the test / monitoring value of the first number of paths or samples includes one of the following:

[0250] In the case where the input value is the power or time delay or phase of the first number of paths or samples, the difference between the input value and the reference value of the same type in the reference input data set is calculated;

[0251] In the case where the input value is a first vector composed of the power or time delay or phase of the first number of paths or samples, the difference between the first vector and a second vector composed of the reference value of the same type in the reference input data set is calculated;

[0252] In the case that the input value is a first number of paths or samples of a combination of power, time delay and phase or any two of them, the difference between the input value and the reference value of the same type in the reference input data set is calculated;

[0253] In the case that the input value is a third vector of a first number of paths or samples of power, time delay and phase or any two of them, the difference between the third vector and a fourth vector of reference values of the same type in the reference input data set is calculated.

[0254] Optionally, in the case that the reference value belongs to the reference input data set and the test / monitoring value is the input value, the terminal first calculates the difference between the reference value and the test / monitoring value of a first number of paths or samples based on the reference value and the test / monitoring value, and then determines the test / monitoring result of the AI model based on the difference.

[0255] The input value includes at least one of power, time delay and phase.

[0256] It should be noted that the reference value of the same type in the reference input data set refers to the reference value of the same type in the reference input data set as the input value, for example, the input value is the power of a first number of paths or samples, and the reference value of the same type in the reference input data set is the power of the same number of paths or samples in the reference input data set.

[0257] The following further illustrates how to determine the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of a first number of paths or samples in the case that the reference value belongs to the reference input data set and the test / monitoring value is the input value.

[0258] Example 4: The reference value belongs to the reference input data set, and the test / monitoring value is the input value

[0259] When the NW can obtain the reference input data set corresponding to the deployed AI model directly (provided by the NW manufacturer or agreed by the protocol) or indirectly (from the UE manufacturer or other servers), the monitoring / test of the model input is performed.

[0260] The reference input data set can be autonomously generated by the LMF side or server, or autonomously generated by the UE side, or pre-defined in the data set or protocol, for example, defined in the 3GPP protocol.

[0261] One model can correspond to one or more groups of reference input data sets, and one group of reference input data sets can correspond to one or more models.

[0262] The model input includes, but is not limited to, CIR or PDP or DP, or a combination of one or more of the power, delay, and phase; the combination of one or more of the power, delay, and phase can be a vector or a single measurement.

[0263] When it is a vector, taking the power as an example, it is expressed as the RSRP of the first number M of paths or samples, which constitutes a vector RSRP meas meas1 meas2 measM ] and the reference value is RSRP ref ref1 ref2 refM ; taking the delay as an example, it is expressed as the delay of the first number M of paths or samples, which constitutes a vector Delay meas meas1 meas2 measM ] and the reference value is Delay ref ref1 ref2 M .

[0264] When it is a single value, taking the power as an example, the test value is the RSRP1 of a single path or sample, or the average value RSRP1 of the M paths or samples RSRP avg , and the reference value is RSRP0 or RSRP0 avg .

[0265] When the reported measurement is a combination of multiple categories, there are the following 1) to 4) situations:

[0266] 1) When the reported measurement is a combination of delay and power:

[0267] At this time, it is expressed as the combination of the first number M of paths or samples of delay and power, which constitutes a vector A meas M M ] and the reference value is A ref ref1 ref1 ref2 ref2 refM refM ​​​​​​​​​​​​​​​​​​​​RSRP1*delay1 or the average of RSRP*delay of M paths or samples (RSRP1*delay1)avg avg , the reference value is RSRP0*delay0 or (RSRP0*delay0) avg ;

[0268] 2) When the reported measurement quantity is the combination of delay and phase:

[0269] At this time, it is expressed as the combination of the first number M paths or samples of delay and phase, which constitutes a vector B meas = [exp(j*phase1)*delay1, exp(j*phase2)*delay2, …, exp(j*phase M )*delay M ], the reference value at this time is B ref = [exp(j*phase ref1 )*delay ref1 , exp(j*phase ref2 )*delay ref2 , …, exp(j*phase refM )*delay refM ], the corresponding test value is exp(j*phase1)*delay1 of a single path or sample or the average of exp(j*phase)*delay of M paths or samples (exp(j*phase1)*delay1)avg when a single value, and the reference value is exp(j*phase0)*delay0 or (exp(j*phase0)*delay0) avg ;

[0270] 3) When the reported measurement quantity is the combination of phase and power:

[0271] At this time, it is expressed as the combination of the first number M paths or samples of phase and power, which constitutes a vector C meas = [RSRP1*exp(j*phase1), RSRP2*exp(j*phase2), …, RSRP M *exp(j*phase M ),], the reference value at this time is C ref = [RSRP ref1 *exp(j*phase ref1 ), RSRP ref2 *exp(j*phase ref2 ), …, RSRP refM *exp(j*phase refM)], when a single value is used, the corresponding test value is RSRP1*exp(j*phase1) of a single path or sample, or the average value of RSRP*exp(j*phase) of M paths or samples (RSRP1*exp(j*phase1)). avg , the reference value is RSRP0*exp(j*phase0) or (RSRP0*exp(j*phase0)) avg ;

[0272] 4) When the reported measurement quantity is a combination of delay, power, and power:

[0273] This is expressed as a combination of the delay, power, and power of the first number M paths or samples, forming a vector D meas =[RSRP1*exp(j*phase1)*delay1,RSRP2*exp(j*phase2)*delay2,…,RSRP M *exp(j*phase M )*del ay M ],

[0274] The reference values ​​at this time are:

[0275] D ref =[RSRP ref1 *exp(j*phase ref1 )*delay ref1 ,RSRP ref2 *exp(j*phase ref2 )*delay ref2 ,…,RSRP refM *e

[0276] xp(j*phase refM )*delay refM ], when a single value is used, the corresponding test value is the average value of RSRP1*exp(j*phase1)*delay1 of a single path or sample or RSRP*exp(j*phase)*delay of M paths or samples (RSRP1*exp(j*phase1)*delay1) avg The reference value is RSRP0*exp(j*phase0)*delay0 or (RSRP0*exp(j*phase0)*delay0) avg .

[0277] When the AI model is deployed on the UE side: the UE reports the measurement quantity as the model input to the LMF, and the LMF takes the measurement quantity reported by the UE and the reference input data set corresponding to the model at this time as the test / monitoring value and the reference value respectively, and evaluates the applicability of the current model after deployment according to the pre-defined evaluation criteria; or, the LMF sends the UE the reference input data set corresponding to the model at this time, and the UE takes the measurement quantity at this time and the reference input data set as the test / monitoring value and the reference value respectively, and tests the applicability of the current model according to the pre-defined evaluation criteria;

[0278] When the AI model is deployed on the LMF side: the LMF takes the actually measured model input and the reference input data set as the test / monitoring value and the reference value respectively, and evaluates the applicability of the current model after deployment according to the pre-defined evaluation criteria;

[0279] The pre-defined evaluation criteria require N evaluations.

[0280] 4.1 When the difference is calculated in the form of a vector,

[0281] 1) The reported measurement quantity is power,

[0282] The pre-defined single evaluation criterion is ΔRSRP1=SGCS(RSRP meas , RSRP ref ), wherein SGCS represents the cosine similarity.

[0283] When N=1, ΔRSRP1 is compared with the pre-defined accuracy ΔRSRP0, and ΔRSRP1 is less than or less than or equal to ΔRSRP0, then the model is applicable;

[0284] When N is greater than 1, the above process is performed multiple times to obtain (ΔRSRP1, ΔRSRP2, …, ΔRSRPN), and ΣΔRSRPn / N is compared with ΔRSRP0, if less than or less than or equal to ΔRSRP0, then the model is applicable; or ΔRSRP1, ΔRSRP2, …, ΔRSRPN are compared with ΔRSRP0 respectively, when the single model applicable times N1 (N1≤N) is greater than or greater than or equal to the pre-defined N0 (N0≤N), then the model is applicable; or when P1=N1 / N is greater than or greater than or equal to the pre-defined P0, the model is applicable;

[0285] The above RSRP can also be replaced by latency or phase, which will not be described here.

[0286] 2) When the reported measurement quantity includes a combination of multiple categories

[0287] When the reported combination is latency and power:

[0288] The pre-defined single evaluation criterion is ΔA1=SGCS(A meas, A ref )

[0289] N = 1, compare ΔA1 with predefined accuracy ΔA0, ΔA1 is less than or less than or equal to ΔA0, the model is applicable;

[0290] N > 1, execute the above process multiple times to get (ΔA1, ΔA2, …, ΔAN), take ΣΔAn / N and compare with ΔA0, if less than or less than or equal to ΔA0, the model is applicable; or ΔA1, ΔA2, …, ΔAN are compared with ΔRSRP0 respectively, when the single model applicable times N1 (N1 ≤ N) is greater than or greater than or equal to the predefined N0 (N0 ≤ N), the model is applicable; or P1 = N1 / N is greater than or greater than or equal to the predefined P0, the model is applicable;

[0291] The combination of delay and phase, the combination of phase and power, and the combination of delay, power and power can be analogously extended, which will not be repeated here.

[0292] The above SGCS can also be a normalized mean squared error (NMSE), a Euclidean distance, an eigenvalue error, etc. evaluation matrix / vector similarity calculation method.

[0293] 4.2 When the difference is calculated in the form of a single value,

[0294] 1) The reported measurement quantity is power,

[0295] The predefined single evaluation criterion is ΔRSRP1 = abs (RSRP1-RSRP0) or ΔRSRPavg1 = abs (RSRP avg1-RSRP avg0).

[0296] N = 1, compare ΔRSRP1 or ΔRSRPavg1 with predefined accuracy ΔRSRP0, ΔRSRP1 or ΔRSRPavg1 is less than or less than or equal to ΔRSRP0, the model is applicable;

[0297] N > 1, execute the above process multiple times to get (ΔRSRP1, ΔRSRP2, …, ΔRSRPN) or (ΔRSRPavg1, ΔRSRPavg2, …, ΔRSRPavgN), take ΣΔRSRPn / N or ΣΔRSRPavgn / N and compare with ΔRSRP0, if less than or less than or equal to ΔRSRP0, the model is applicable; or ΔRSRP1, ΔRSRP2, …, ΔRSRPN are compared with ΔRSRP0 respectively, when the single model applicable times N1 (N1 ≤ N) is greater than or greater than or equal to the predefined N0 (N0 ≤ N), the model is applicable; or P1 = N1 / N is greater than or greater than or equal to the predefined P0, the model is applicable;

[0298] The RSRP can be replaced by delay or phase, which will not be described here.

[0299] 2) When the reported measurement quantity includes a combination of multiple categories,

[0300] The absolute value is changed to the two-norm ||.||.

[0301] The method provided by the application provides a method for determining whether an AI model is applicable in the case where the reference value is a reference input data set and the test / monitoring value is an input value, and the input value is at least one of power, time delay, and phase. The method realizes verification or monitoring of whether an AI model for positioning can adapt to the current environment, thereby improving the reliability of the deployment of the AI model for positioning in the actual environment, and effectively ensuring the performance of AI positioning.

[0302] In some embodiments, the method provided by the application also includes:

[0303] In the case where the test / monitoring result is not applicable, the terminal sends the test / monitoring result to the network side device;

[0304] The terminal receives first indication information from the network side device, and the first indication information is used to instruct the terminal to perform a second operation;

[0305] The second operation includes at least one of the following:

[0306] updating the current model;

[0307] fine-tuning the current model;

[0308] retraining the current model;

[0309] deactivating the current model;

[0310] the terminal falls back to a non-AI mode;

[0311] restarting the current AI positioning related measurement;

[0312] stopping the current AI positioning related measurement;

[0313] extending the current AI positioning related measurement.

[0314] It can be understood that when the test / monitoring result of the AI model is not applicable, if the test / monitoring is completed by the terminal, the terminal sends the test / monitoring result to the network side device, and the network side device sends first indication information to the terminal to instruct the terminal to perform a second operation.

[0315] Optionally, the second operation includes at least one of the following: a fifth operation, a sixth operation, and a seventh operation.

[0316] Optionally, the fifth operation includes one of the following: updating the current model; fine-tuning the current model; and retraining the current model.

[0317] It should be noted that after the fifth operation, post-deployment testing or monitoring should also be performed; if the test fails within the specified time or this time, the sixth operation is performed.

[0318] Optionally, the sixth operation includes one of the following: deactivating the current model; the terminal falling back to non-AI mode.

[0319] Optionally, deactivating the current model includes one of the following: selecting another activated model and switching to the selected model; selecting another inactivated model and activating the selected model; and rolling back the current model to a previous model.

[0320] Optionally, the seventh operation includes one of the following: restarting the current AI positioning related measurement; stopping the current AI positioning related measurement; and extending the current AI positioning related measurement.

[0321] Through the fifth and sixth operations, the AI ​​model can be adjusted; by performing the seventh operation, the impact of the current AI model on positioning performance can be reduced.

[0322] In some embodiments, the AI ​​model includes a first model, or an optimized model of the first model;

[0323] The first model includes at least one of the following:

[0324] a first model generated based on at least one of a protocol-standardized model structure, protocol-standardized model parameters, and a protocol-standardized dataset for training the model;

[0325] a first model determined based on a protocol standardized model structure and model parameters received by the terminal;

[0326] a received first model, the first model comprising at least a portion of a model structure and / or a portion of model parameters;

[0327] a first model generated or determined based on a first data set, where the first data set is a protocol-standardized data set or a data set sent by the network-side device to the terminal;

[0328] The terminal generates or determines a first model based on a second data set and a standardized model structure, where the second data set is a data set standardized by a protocol or a data set sent to the terminal by a peer device of the terminal.

[0329] The optimization model of the first model is a model having the same input and output mapping relationship as the first model or within a preset error range.

[0330] Optionally, the peer device of the terminal includes a network side device or a test equipment TE.

[0331] The optimization model of the first model refers to an optimization model based on the first model, which can be understood as a model that does not perfectly implement the first model but can implement a function similar to the first model, or the mapping relationship between the input and the output of the optimization model is the same as or within a certain error range of the first model, or can be understood as having a certain consistency.

[0332] Optionally, the terminal can maintain L optimization models based on the first model, and the L optimization models based on the first model can be respectively applicable to different situations, scenes, conditions or data distributions, etc. Alternatively, the performance similar to the first model can be obtained through the L optimization models based on the first model, wherein the target quantity of each optimization model is less than the first model, and the target quantity includes scale, complexity and / or storage size. L is a positive integer greater than or equal to 1.

[0333] The method provided in the embodiment of the application describes the generation or obtaining method of the AI model in each of the above embodiments. Through the test / monitoring method of the embodiment, it is verified or monitored whether the AI model for positioning generated or obtained by the above method can adapt to the current environment, thereby improving the reliability of the AI model generated or obtained by the above method in the actual environment, and effectively ensuring the performance of AI positioning.

[0334] Figure 3 is a flowchart of a second method for testing or monitoring an artificial intelligence (AI) model for positioning provided by an embodiment of the application. As shown in Figure 3 The method provided by the embodiment includes:

[0335] In step 301, the network side device obtains a reference value and an input value.

[0336] It should be noted that the reference value in the embodiment of the application, as a reference value for testing or monitoring the AI model, can be reference position information, a reference intermediate quantity, a reference output data set, or a reference input data set.

[0337] The AI model described in the embodiment of the application is an AI model for positioning, i.e., participating in AI positioning. The output of the AI model can be position information or an intermediate quantity. The definition of the intermediate quantity will be described below.

[0338] The input value can be understood as a model input for testing or monitoring the AI model, can be measurement information, or model input information processed from the measurement information. In the embodiments of the present application, the measurement information generally refers to signal / channel measurement information, and the measurement information can also be referred to as measurement quantity, measurement result, etc.

[0339] In some embodiments, the network-side device is an LMF, or other functional units capable of performing positioning calculation.

[0340] In step 302, the network-side device performs a third operation, which includes one of the following:

[0341] The reference value and the input value are sent to the terminal, and the reference value and the input value are used to determine the test / monitoring result of the AI model for positioning locally deployed by the terminal;

[0342] The input value is sent to the terminal, and the output value from the terminal is received, the output value is obtained by inputting the input value into the AI model for positioning locally deployed by the terminal, the test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value, and the test / monitoring value is the output value;

[0343] The input value is input into the AI model for positioning locally deployed by the network-side device, the output value of the AI model is obtained, and the test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value, and the test / monitoring value is the output value;

[0344] The test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value, and the test / monitoring value is the input value.

[0345] According to the position of the AI model deployment, the network-side device can perform corresponding operations.

[0346] The AI model is deployed in the terminal. In one implementation, the network-side device obtains a reference value and an input value, and sends the reference value and the input value to the terminal, and the terminal determines the test / monitoring result of the AI model for positioning locally deployed by the terminal according to the reference value and the input value. In another implementation, after the network-side device obtains the reference value and the input value, the network-side device sends the input value to the terminal, the terminal inputs the input value into the AI model, obtains the output value of the AI model, and sends the output value to the network-side device, the network-side device receives the output value from the terminal, takes the output value as a test / monitoring value, and determines the test / monitoring result of the AI model based on the reference value and the test / monitoring value.

[0347] The AI model is deployed on a network side device. In one embodiment, the network side device obtains a reference value and an input value, inputs the input value into the AI model, obtains an output value of the AI model, takes the output value as a test / monitoring value, and determines a test / monitoring result of the AI model based on the reference value and the test / monitoring value. In another embodiment, the network side device obtains a reference value and an input value, takes the input value as a test / monitoring value, and determines a test / monitoring result of the AI model based on the reference value and the test / monitoring value.

[0348] The test or monitoring method for the AI model for positioning provided in the embodiments of the present application verifies or monitors the AI model for positioning by obtaining a reference value and an input value, thereby improving the reliability of the AI model for positioning in actual deployment and effectively ensuring the performance of AI positioning.

[0349] In some embodiments, the reference value and the input value include one of the following a) to d):

[0350] a) The reference value is reference position information, and the input value is measurement information or model input information obtained by processing the measurement information;

[0351] The reference position information refers to position information as a reference output. The position information can be rectangular coordinates, polar coordinates, latitude and longitude information, relative position, absolute position, etc., which are not limited in the present application.

[0352] The measurement information generally refers to signal / channel measurement information, and the measurement information can be replaced by the concepts of measurement quantity and measurement result.

[0353] In the embodiments of the present application, the measurement information includes at least one of the following: power, time delay, phase, channel impulse response (CIR), power delay profile (PDP), and delay profile (DP).

[0354] Optionally, the power includes received power, received power of a target path in multipath, signal-to-noise ratio, etc. The time delay includes received time delay, sending time delay, round-trip time delay, etc. The phase includes phase offset, phase delay, phase difference, etc. The channel impulse response includes impulse response, frequency response, etc.

[0355] The signal can pass through multiple different paths to reach the receiving end during transmission, and each path corresponds to a different propagation delay. The power delay spectrum describes the distribution of signal power at different delay times. The delay spectrum reflects the distribution of arrival delays of the received signal on different propagation paths, and is usually given in the form of a probability density function (PDF) or power spectral density (PSD).

[0356] When the AI model is deployed on the network side device, the network side device determines the test / monitoring result of the AI model according to the reference position information and an output value obtained by inputting an input value into the AI model for inference.

[0357] When the AI model is deployed on the terminal, the terminal sends the reference position information and the input value to the terminal.

[0358] Optionally, processing the measurement information refers to mathematical processing such as normalization and averaging of the measurement information.

[0359] In some embodiments, the reference position information and the input value are obtained in the following manner:

[0360] The network side device selects position information of a first communication device with a known position as the reference position information, and the input value is measurement information corresponding to the first communication device or model input information obtained by processing the measurement information corresponding to the first communication device.

[0361] In the embodiments of the present application, the first communication device is a communication device with a known position, that is, the position of the first communication device is known to the network side device. The first communication device includes a positioning reference unit (PRU) or other communication devices supporting channel / signal measurement.

[0362] Optionally, the network side device selects a target PRU as the first communication device from multiple PRUs around the terminal, the target PRU reports its position information and measurement information to the network side device, and the network side device sends the position information of the target PRU as the reference position information and the measurement information of the target PRU as the input value to the terminal.

[0363] The method provided in the embodiments of the present application is suitable for the case where the output of the AI model is position information, and the inference ability of the AI model for positioning is verified or monitored by obtaining the reference position information and the input value, thereby improving the reliability of the AI model for positioning in actual deployment environment and effectively ensuring the performance of AI positioning.

[0364] b) the reference value is a reference intermediate quantity, the reference intermediate quantity being intermediate information for determining a terminal position, the reference intermediate quantity being of the same type as the output value of the AI model, the input value being measurement information or model input information obtained by processing the measurement information;

[0365] If the output of the AI model is an intermediate quantity, a reference intermediate quantity can be obtained, and measurement information or model input information obtained by processing the measurement information can be taken as an input value to test or monitor the AI model.

[0366] The reference intermediate quantity is of the same type as the output value of the AI model. For example, if the type of the output value of the AI model is RTOA, the type of the reference intermediate quantity should also be RTOA.

[0367] The reference intermediate quantity can also be referred to as a reference intermediate feature, which is intermediate information that can determine a terminal position.

[0368] In some embodiments, the reference intermediate quantity includes at least one of the following:

[0369] Line of Sight or Non-Line of Sight (LOS / NLOS) indicator;

[0370] Reference Signal Time Difference (RSTD);

[0371] Relative Time of Arrival (RTOA);

[0372] Reception-Transmission Rx-Tx time difference;

[0373] Time of Arrival (TOA);

[0374] Angle of Arrival (AoA);

[0375] Angle of Departure (AoD);

[0376] Reference Signal Received Power (RSRP);

[0377] Reference Signal Received Path Power (RSRPP).

[0378] In some embodiments, the reference intermediate quantity and the input value are obtained in one of the following ways:

[0379] The reference position information and the measurement information reported by the first communication device with known position are received, the reference intermediate quantity is determined based on the reference position information and second information used to assist in determining the reference intermediate quantity, and the measurement information or model input information obtained by processing the measurement information is used as the input value;

[0380] The reference intermediate quantity and the input value determined and reported by the first communication device with known position are received, and the input value is the measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

[0381] The embodiments of the present application provide two ways of determining the reference intermediate quantity:

[0382] 1) Direct determination, determined by the first communication device;

[0383] The network side device receives the reference intermediate quantity and the measurement information reported by the first communication device, where the reference intermediate quantity is determined by the first communication device.

[0384] Optionally, the network side device sends the reference intermediate quantity and the measurement information to the terminal.

[0385] 2) Indirect determination, i.e., determined by the network side device based on the reference position information of the first communication device and second information, where the second information is used to assist in determining the reference intermediate quantity.

[0386] The network side device receives the reference position information and the measurement information reported by the first communication device, determines the reference intermediate quantity based on the reference position information and the second information, and sends the reference intermediate quantity and the measurement information to the terminal.

[0387] In the embodiments of the present application, the first communication device is a communication device with known position, i.e., the position of the first communication device is known to the network side device. Optionally, the first communication device is selected by the network side device, and the first communication device includes a positioning reference unit (PRU) or other communication device supporting channel / signal measurement.

[0388] Optionally, the network side device selects a target PRU as the first communication device from multiple PRUs around the terminal, the target PRU reports its position information and measurement information to the network side device, the network side device uses the position information of the target PRU as the reference position information, determines the reference intermediate quantity based on the reference position information and the second information, and sends the measurement information of the PRU to the terminal as the input value.

[0389] In some embodiments, the second information comprises at least one of: a location of a transmission and reception point (TRP) currently participating in AI positioning, historical data of measurement information, historical data of AI model output values, a historical location of the terminal, a historical location of the TPR, and a historical location of the first communication device whose location is known.

[0390] For example, the second information is a location of a transmission and reception point (TRP) currently participating in AI positioning. Based on the location information of the target PRU and the location of the TRP currently participating in AI positioning, the network-side device calculates intermediate quantities when the target PRU performs positioning using the TRP currently participating in AI positioning by using mathematical geometric relationships. These intermediate quantities are the reference intermediate quantities. The target PRU measures the downlink positioning reference signal (DL-PRS) transmitted by the TRP currently participating in AI positioning to obtain measurement information and sends the measurement information to the network-side device. Further, the network-side device sends the reference intermediate quantities and the measurement information to the terminal. The terminal takes the reference intermediate quantities as reference values and inputs the measurement information into the AI model to obtain the intermediate quantities output by the AI model. The terminal takes the intermediate quantities output by the AI model as test / monitoring values and compares them with the reference intermediate quantities to determine the test / monitoring result of the AI model. Alternatively, the network-side device sends the measurement information to the terminal. The terminal inputs the measurement information into the AI model to obtain the intermediate quantities output by the AI model and sends the intermediate quantities output by the AI model as test / monitoring values to the network-side device. The network-side device compares the intermediate quantities output by the AI model with the reference intermediate quantities to determine the test / monitoring result of the AI model.

[0391] The second information can also be other auxiliary data, including historical data of measurement information, historical data of AI model output values, a historical location of the terminal, a historical location of the TRP, and / or a historical location of the first communication device whose location is known.

[0392] In some embodiments, when the AI model is deployed on the terminal side, the terminal inputs the input values into the AI model, obtains the intermediate quantities output by the AI model, and sends the intermediate quantities to the network-side device. The network-side device compares the intermediate quantities output by the AI model with the reference intermediate quantities to determine the test / monitoring result of the AI model.

[0393] In some embodiments, when the AI model is deployed on the network side, the network-side device inputs the measurement information or model input information obtained by processing the measurement information into the AI model deployed locally on the network side as input values, obtains the intermediate quantities output by the AI model, and determines the test / monitoring result of the AI model based on the intermediate quantities output by the AI model and the reference intermediate quantities.

[0394] It is worth noting that in the case of AI model deployment in LMF or gNB side: the LMF side inputs the measurement information or the model input information obtained after processing the measurement information into the AI model for inference, and takes the intermediate quantity output by the AI model and the corresponding reference intermediate quantity as test / monitoring and reference values respectively to determine the test / monitoring result of the AI model.

[0395] In the case of AI model deployment in gNB side: the gNB side inputs the measurement information or the model input information obtained after processing the measurement information into the AI model for inference, and reports the intermediate quantity output by the AI model to the LMF, which takes the intermediate quantity output by the AI model and the corresponding reference intermediate quantity as test / monitoring and reference values respectively to determine the test / monitoring result of the AI model.

[0396] Optionally, the gNB reports the intermediate quantity output by the AI model to the LMF, which needs to meet the reporting requirements, including the reporting range and granularity, which is selected by the gNB according to the LMF indication and notified to the LMF.

[0397] The method provided by the embodiments of the present application is suitable for the case where the output of the AI model is an intermediate quantity, and the test / monitoring result of the AI model is determined based on the reference intermediate quantity and the intermediate quantity output by the AI model, which realizes the verification or monitoring of the inference ability of the AI model for positioning, thereby improving the reliability of the AI model for positioning in actual environment, and effectively ensuring the performance of AI positioning.

[0398] c) the reference value belongs to a reference output data set, and the input value belongs to a reference input data set;

[0399] It can be understood that in the embodiments, the AI model can execute the AI function according to the reference input data set to obtain the corresponding model output, and the model output is taken as the test / monitoring value, and the test or monitoring of the AI model is realized based on the reference output data set and the test / monitoring value.

[0400] The reference output data set includes reference values of the same type as the output value of the AI model.

[0401] Optionally, one AI model can correspond to one or more groups of reference data sets, and one group of reference data sets can also correspond to one or more AI models.

[0402] The reference output data set or the reference input data set is generated autonomously by a terminal, or autonomously by a network side device, or by a first device, or predefined by a protocol.

[0403] The first device can be a server, or other device other than a terminal and a network side device.

[0404] Optionally, the network side device autonomously generates the reference data set, or receives the reference data set generated and sent by the first device, and in the case that the AI model is deployed on the terminal side, the network side device sends the reference data set to the terminal, the reference data set includes the parameter input data set and the reference output data set, the terminal takes the reference input data set as the model input, takes the model output at this time as the test / monitoring value, and compares the test / monitoring value with the reference output data set, so as to determine the test / monitoring result of the AI model.

[0405] Optionally, in the case that the AI model is deployed on the terminal side, the network side device sends the reference input data set to the terminal, the terminal takes the reference input data set as the model input, reports the model output at this time to the network side device as the test / monitoring value, the network side device compares the test / monitoring value with the reference output data set, so as to determine the test / monitoring result of the AI model. The reference input data set or the reference output data set is from the first device, or is autonomously generated by the network side device.

[0406] Optionally, in the case that the AI model is deployed on the network side, the network side device takes the reference input data set as the model input, takes the model output at this time as the test / monitoring value, and compares the test / monitoring value with the reference output data set, so as to determine the test / monitoring result of the AI model. The reference input data set or the reference output data set is predefined by a protocol, or is from the first device, or is autonomously generated by the network side device.

[0407] It should be noted that, when the AI model is deployed on the LMF side, the LMF takes the reference input data set as the model input, takes the model output at this time as the test / monitoring value, compares the test / monitoring value with the reference output data set, and determines the test / monitoring result of the AI model; in addition, when the AI model is deployed on the gNB side, the LMF sends the reference input data set to the gNB, the gNB takes the reference input data set as the model input, the gNB optionally reports the measurement result to the LMF, the LMF takes the measurement result reported by the gNB as the test / monitoring value, compares the test / monitoring value with the reference output data set, and determines the test / monitoring result of the AI model.

[0408] The method provided by the embodiments of the present application is applicable to the case that the AI model can perform an AI function according to a reference data set. The reference input data set is input into the AI model for positioning deployed locally in the terminal, the output value of the AI model is obtained, and the test / monitoring result of the AI model is determined based on the reference output data set and the output value of the AI model, so as to verify or monitor the inference ability of the AI model for positioning, thereby improving the reliability of the AI model for positioning deployed in the actual environment and effectively ensuring the performance of AI positioning.

[0409] d) The reference value belongs to the reference input data set, and the input value is measurement information or model input information obtained by processing the measurement information.

[0410] In order to verify whether the AI model can adapt to the current environment, the embodiments of the present application propose to compare the measurement information or model input information obtained by processing the measurement information as model input with the reference input data set, so as to test / monitor the input of the AI model.

[0411] The terminal can obtain the measurement information by actual measurement, for example, by measuring the downlink positioning reference signal transmitted by the TRP of the current reference AI positioning.

[0412] Optionally, in the case that the AI model is deployed on the terminal side, the network side device sends the reference input data set to the terminal, and the terminal determines the test / monitoring result of the AI model based on the reference input data set and the measurement quantity as model input.

[0413] Optionally, in the case that the AI model is deployed on the terminal side, the network side device receives the measurement quantity reported by the terminal, the network side device takes the measurement quantity reported by the terminal as a test / monitoring value, and determines the test / monitoring result of the AI model based on the reference input data set and the test / monitoring value. Here, the measurement quantity as model input includes measurement information or model input information obtained by processing the measurement information.

[0414] Optionally, in the case that the AI model is deployed on the network side, the network side device takes the actually measured model input and the reference input data set as a test / monitoring value and a reference value respectively, and determines the test / monitoring result of the AI model.

[0415] In some embodiments, the reference input data set is generated by the terminal, or is generated by the network side device, or is generated by a first device, or is predefined by a protocol.

[0416] The first device can be a server, or a device other than the terminal and the network side device.

[0417] It should be noted that when the AI model is deployed in the gNB, if the measurement quantity of the model input is measured by the gNB, the gNB can optionally report the measurement result to the LMF, and the LMF determines the test / monitoring result of the AI model according to the actual measured model input and the reference input data set as the test / monitoring value and the reference value respectively.

[0418] The method provided in the embodiments of the present application is suitable for the case that the AI model can perform an AI function according to a reference data set, and the test / monitoring result of the AI model is determined by acquiring the reference input data set and the input value, which realizes the verification or monitoring of whether the input of the AI model for positioning can adapt to the current environment, thereby improving the reliability of the AI model for positioning in the actual environment, and effectively guaranteeing the performance of AI positioning.

[0419] In some embodiments, the test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value, including:

[0420] calculating the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0421] determining the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0422] wherein the first number is greater than or equal to 1.

[0423] In the embodiments of the present application, the AI model can be tested / monitored once or multiple times.

[0424] If the number of times of testing / monitoring of the AI model is 1, the test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value, including:

[0425] calculating the difference between the reference value and the test / monitoring value, and comparing the difference with a predefined accuracy requirement, if the predefined accuracy requirement is met, it is determined that the AI model is applicable, if the predefined accuracy requirement is not met, it is determined that the AI model is not applicable.

[0426] In order to improve the accuracy of the test / monitoring, the AI model can also be tested / monitored multiple times.

[0427] Optionally, when the number of times is greater than 1, the average value of the difference between the test / monitoring value and the reference value of each execution is compared with the predefined accuracy, or the ratio of the number of times of evaluating the model as applicable to the total evaluation times needs to be greater than a first threshold value (for example, a certain percentage, for example, 90%), or the number of times of evaluating the model as applicable is greater than a predefined evaluation number (for example, 4), at this time, the test / monitoring result of the AI model is applicable, otherwise, it is not applicable.

[0428] The difference between the reference value and the test / monitoring value of the first number of paths or samples can be calculated in each execution of the test / monitoring, and the test / monitoring result of the AI model is determined based on the difference between the reference value and the test / monitoring value of the first number of paths or samples, so as to improve the accuracy of the test / monitoring of the AI model.

[0429] The method provided by the embodiments of the present application can improve the accuracy of the test / monitoring of the AI model by calculating the difference between the reference value and the test / monitoring value of the first number of paths or samples, and determining the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first number of paths or samples.

[0430] In some embodiments, the calculation of the difference between the reference value and the test / monitoring value of the first number of paths or samples includes at least one of the following:

[0431] calculating the Euclidean distance between the vector composed of the reference values of the first number of paths or samples and the vector composed of the test / monitoring values;

[0432] calculating the cosine similarity between the vector composed of the reference values of the first number of paths or samples and the vector composed of the test / monitoring values;

[0433] calculating the root mean square error between the vector composed of the reference values of the first number of paths or samples and the vector composed of the test / monitoring values;

[0434] calculating the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0435] calculating the average of the absolute values of the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0436] calculating the root mean square error of the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0437] calculating the straight line distance between the reference value and the test / monitoring value of the first number of paths or samples;

[0438] calculating the average of the straight line distance between the reference value and the test / monitoring value of the first number of paths or samples;

[0439] calculating the root mean square of the straight line distance between the reference value and the test / monitoring value of the first number of paths or samples.

[0440] The method provided in the embodiments of the present application can calculate the difference between the first quantity path or the sample reference value and the test / monitoring value in a vector manner, for example, calculating the Euclidean distance, cosine similarity, root mean square error, etc. between a vector composed of the first quantity path or the sample reference value and a vector composed of the test / monitoring value.

[0441] Optionally, the difference between the first quantity path or the sample reference value and the test / monitoring value can also be calculated in a single value manner, for example, calculating the difference between the first quantity path or the sample reference value and the test / monitoring value, the average of the absolute value of the difference, or the root mean square error of the difference.

[0442] Optionally, the difference between the first quantity path or the sample reference value and the test / monitoring value can also be calculated in a straight line distance manner, for example, calculating the straight line distance between the first quantity path or the sample reference value and the test / monitoring value, the average of the straight line distance, or the root mean square of the straight line distance.

[0443] The method provided in the embodiments of the present application realizes the flexibility of difference calculation.

[0444] In some embodiments, determining the test / monitoring result of the AI model based on the difference between the reference value of the first quantity path or the sample and the test / monitoring value comprises:

[0445] In the case that the difference between the reference value of the first quantity path or the sample and the test / monitoring value meets the predefined accuracy requirement, determining that the test / monitoring result of the AI model is applicable.

[0446] In the case that the difference between the reference value of the first quantity path or the sample and the test / monitoring value does not meet the predefined accuracy requirement, determining that the test / monitoring result of the AI model is not applicable.

[0447] It can be understood that the difference between the reference value of the first quantity path or the sample and the test / monitoring value meeting the predefined accuracy requirement indicates that the inference ability of the current AI model can meet the current positioning environment, or that the current AI model can adapt to the current positioning environment, that is, it can be determined that the current AI model is applicable.

[0448] Optionally, the difference between the reference value of the first quantity path or the sample and the test / monitoring value meeting the predefined accuracy requirement comprises:

[0449] In the case that the number of executions is equal to or greater than 1, the average of the difference between the test / monitoring value and the reference value of each execution is compared with the predefined accuracy, or the ratio of the number of times that the single evaluation model is applicable to the total evaluation times is greater than the first threshold value, or the number of times that the single evaluation model is applicable is greater than the predefined evaluation number, and the test / monitoring result of the AI model is applicable.

[0450] The difference between the reference value and the test / monitoring value of the first number of paths or samples does not meet the predefined accuracy requirement, indicating that the inference ability of the current AI model cannot meet the current positioning environment, or indicating that the current AI model cannot adapt to the current positioning environment, that is, it can be determined that the current AI model is not applicable.

[0451] In some embodiments, the predefined accuracy requirement is applicable to a specified condition, and it can be understood that the predefined accuracy requirement is related to the specified condition and needs to be applicable under the specified condition. The predefined accuracy requirement can be different under different specified conditions.

[0452] Optionally, the specified condition includes at least one of the following: signal-to-interference-and-noise ratio, signal-to-noise ratio, channel strength, noise strength, channel model, reference signal bandwidth, reference signal configuration, subcarrier spacing, carrier frequency, frequency band information, duplexing mode, TRP configuration, UE configuration, AI model structure, AI model complexity, and AI quantization information.

[0453] It can be understood that the predefined accuracy requirement can be determined according to at least one of the following: signal-to-noise ratio, channel strength, noise strength, channel model, reference signal bandwidth, reference signal configuration, subcarrier spacing, carrier frequency, frequency band information, duplexing mode, TRP configuration, UE configuration, AI model structure, AI model complexity, and AI quantization information.

[0454] The method provided by the present application defines an evaluation criterion for judging whether an AI model is applicable. The evaluation criterion can also be applicable to the evaluation or test of the AI model before deployment. In the test, the NW side device is replaced by a test equipment (TE).

[0455] In some embodiments, in the case that the reference value is a reference intermediate quantity and the reference intermediate quantity is a line-of-sight / non-line-of-sight indication, the first probability under the specified condition is taken as the reference accuracy.

[0456] The difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement, which includes:

[0457] A second probability that the reference value and the test / monitoring value under the first number of paths or samples are different is calculated.

[0458] In a case where the first probability represents a maximum mismatch probability of the LOS / NLOS indicator, and the second probability is less than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement.

[0459] Or,

[0460] A third probability that the reference value and the test / monitoring value are the same under the first number of paths or samples is calculated.

[0461] In a case where the first probability represents a minimum match probability of the LOS / NLOS indicator, and the third probability is greater than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement.

[0462] It should be noted that the LOS / NLOS indicator is an indication domain indicating LOS or NLOS through different values.

[0463] According to the result (1 or 0) of AI model inference as the test / monitoring value, the probability that the reference value and the test / monitoring value are the same or different under the first number of paths or samples is calculated, and compared with the first probability.

[0464] If the first probability represents a maximum mismatch probability of the LOS / NLOS indicator, the probability that the reference value and the test / monitoring value are different under the first number of paths or samples is calculated, and if the probability that the reference value and the test / monitoring value are different under the first number of paths or samples is less than the first probability, or less than or equal to the first probability, it indicates that the difference between the reference value and the test / monitoring value under the first number of paths or samples meets the predefined accuracy requirement, that is, the measurement / monitoring result of the current AI model is applicable.

[0465] If the first probability represents a minimum match probability of the LOS / NLOS indicator, the probability that the reference value and the test / monitoring value are the same under the first number of paths or samples is calculated, and if the probability that the reference value and the test / monitoring value are the same under the first number of paths or samples is greater than the first probability, or greater than or equal to the first probability, it indicates that the difference between the reference value and the test / monitoring value under the first number of paths or samples meets the predefined accuracy requirement, that is, the measurement / monitoring result of the current AI model is applicable.

[0466] The method provided by the application provides a method for judging whether the AI model is applicable in a case where the reference value is a reference intermediate quantity and the reference intermediate quantity is a LOS / NLOS indicator, realizes verification or monitoring of whether the AI model for positioning can adapt to the current environment, thereby improving the reliability of the AI model for positioning in the actual environment, and can effectively guarantee the performance of AI positioning.

[0467] In some embodiments, the input value comprises at least one of: power, time delay, phase.

[0468] Optionally, in the case that the reference value belongs to the reference input dataset and the test / monitoring value is the input value, the calculating the difference between the reference value and the test / monitoring value of the first number of paths or samples comprises one of:

[0469] In the case that the input value is the power or the time delay or the phase of the first number of paths or samples, the difference between the input value and the reference value of the same type in the reference input dataset is calculated;

[0470] In the case that the input value is a first vector composed of the power or the time delay or the phase of the first number of paths or samples, the difference between the first vector and a second vector composed of the reference value of the same type in the reference input dataset is calculated;

[0471] In the case that the input value is a combination of the power, the time delay and the phase of the first number of paths or samples or any two of them, the difference between the input value and the reference value of the same type in the reference input dataset is calculated;

[0472] In the case that the input value is a third vector composed of the power, the time delay and the phase of the first number of paths or samples or any two of them, the difference between the third vector and a fourth vector composed of the reference value of the same type in the reference input dataset is calculated.

[0473] Optionally, in the case that the reference value belongs to the reference input dataset and the test / monitoring value is the input value, the terminal first calculates the difference between the reference value and the test / monitoring value of the first number of paths or samples based on the reference value and the test / monitoring value, and then determines the test / monitoring result of the AI model based on the difference.

[0474] The input value comprises at least one of: power, time delay, phase.

[0475] It should be noted that the reference value of the same type in the reference input dataset refers to the reference value of the same type as the input value in the reference input dataset, for example, the input value is the power of the first number of paths or samples, and the reference value of the same type in the reference input dataset is the power of the same number of paths or samples in the reference input dataset.

[0476] The method provided in the embodiments of the application gives a method for determining whether an AI model is applicable in the case where a reference value is a reference input data set and a test / monitoring value is an input value, the input value being at least one of power, time delay, and phase, thereby achieving verification or monitoring of whether an AI model for positioning can adapt to a current environment, thereby improving the reliability of deployment of the AI model for positioning in an actual environment and effectively ensuring the performance of AI positioning.

[0477] How to determine the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first number of paths or samples can refer to instances 1-4 in the foregoing terminal-side method embodiments, which will not be described here again.

[0478] In some embodiments, the method further comprises:

[0479] receiving a test / monitoring result from the terminal, the test / monitoring result being not applicable;

[0480] sending first indication information to the terminal, the first indication information being used to instruct the terminal to perform a second operation;

[0481] The second operation comprises at least one of the following:

[0482] updating the current model;

[0483] fine-tuning the current model;

[0484] retraining the current model;

[0485] deactivating the current model;

[0486] the terminal falling back to a non-AI mode;

[0487] restarting a current AI positioning-related measurement;

[0488] stopping a current AI positioning-related measurement;

[0489] extending a current AI positioning-related measurement.

[0490] It can be understood that when the test / monitoring result of the AI model is not applicable, if the test / monitoring is completed by the terminal, the terminal sends the test / monitoring result to the network-side device, the network-side device receives the test / monitoring result from the terminal, and sends first indication information to the terminal to instruct the terminal to perform a second operation, the purpose of the second operation being to adjust the AI model by performing operations such as updating, fine-tuning, or retraining the current model, or deactivating the current model, activating other models, switching the current model, selecting other models, or falling back the current model to a past model; or restarting, stopping, or extending a current AI positioning-related measurement, thereby reducing the influence of the current AI model on the positioning performance.

[0491] In some embodiments, the method further comprises:

[0492] In the case that the test / monitoring result is not applicable, the network side device sends second indication information to the terminal, the second indication information being used to instruct the terminal to perform a fourth operation;

[0493] The fourth operation comprises: restarting the current AI positioning related measurement, stopping the current AI positioning related measurement, prolonging the current AI positioning related measurement.

[0494] It can be understood that when the test / monitoring result of the AI model is not applicable, if the test / monitoring is completed by the network side device, the network side device sends second indication information to the terminal to instruct the terminal to perform a fourth operation, and the purpose of the fourth operation is to reduce the influence of the current AI model on the positioning performance by restarting, stopping or prolonging the current AI positioning related measurement.

[0495] In some embodiments, the AI model comprises a first model, or an optimization model of the first model;

[0496] The first model comprises at least one of:

[0497] The first model is generated based on at least one of a protocol standardized model structure, a protocol standardized model parameter and a protocol standardized data set for training the model;

[0498] The first model is determined based on a protocol standardized model structure and received model parameters;

[0499] The received first model comprises at least part of the model structure and / or part of the model parameters;

[0500] The first model is generated or determined based on a first data set, the first data set being a protocol standardized data set or a data set autonomously generated by the network side device;

[0501] The first model is generated or determined based on a second data set and a standardized model structure, the second data set being a protocol standardized data set or a data set autonomously generated by the network side device or a test device.

[0502] The optimization model of the first model refers to an optimization model based on the first model, which can be understood as a model that does not perfectly implement the first model, but can implement a function similar to the first model, or the mapping relationship between the input and output of the optimization model is the same as or within a certain error range of the first model, or can be understood as having a certain consistency.

[0503] Optionally, the terminal can maintain L optimization models based on the first model, and the L optimization models based on the first model can be respectively applicable to different cases, scenarios, conditions, or data distributions, etc. Alternatively, the performance similar to the first model can be obtained through the L optimization models based on the first model, wherein the target quantity of each optimization model is less than the first model, and the target quantity includes scale, complexity, and / or storage size. L is a positive integer greater than or equal to 1.

[0504] The method provided by the embodiment of the application describes the generation or obtaining method of the AI model described in each of the above embodiments. Through the testing / monitoring method of the embodiment, it is verified or monitored whether the AI model for positioning generated or obtained by the above method can adapt to the current environment, thereby improving the reliability of the AI model generated or obtained by the above method in the actual environment, and effectively guaranteeing the performance of AI positioning.

[0505] The testing or monitoring method of the artificial intelligence AI model for positioning provided by the embodiment of the application can be executed by a testing or monitoring device for the artificial intelligence AI model for positioning. In the embodiment of the application, the testing or monitoring method of the artificial intelligence AI model for positioning is executed by the testing or monitoring device for the artificial intelligence AI model for positioning as an example to illustrate the testing or monitoring device for the artificial intelligence AI model for positioning provided by the embodiment of the application.

[0506] Figure 4 is one of the structural schematic diagrams of the testing or monitoring device for the artificial intelligence AI model for positioning provided by the embodiment of the application. As shown in Figure 4 The testing or monitoring device for the artificial intelligence AI model for positioning 400 includes:

[0507] The first obtaining unit 410 is configured to obtain a reference value and an input value.

[0508] The first execution unit 420 is configured to execute a first operation, and the first operation includes one of the following:

[0509] The input value is input into the AI model for positioning deployed locally in the terminal, and an output value of the AI model is obtained. The output value is taken as a test / monitoring value, and a test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value.

[0510] The input value is taken as a test / monitoring value, and a test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value.

[0511] In the embodiments of the present application, by obtaining the reference value and the input value, inputting the input value into the AI model for positioning locally deployed by the terminal, obtaining the output value of the AI model, taking the output value as the test / monitoring value, or taking the input value as the test / monitoring value, determining the test / monitoring result of the AI model based on the reference value and the test / monitoring value, the verification or monitoring of the AI model for positioning is realized, thereby improving the reliability of the AI model for positioning deployed in the actual environment, and the performance of AI positioning can be effectively guaranteed.

[0512] Optionally, the reference value and the input value include one of the following:

[0513] The reference value is reference position information, and the input value is measurement information or model input information obtained after processing the measurement information.

[0514] The reference value is a reference intermediate quantity, the reference intermediate quantity is intermediate information used to determine the position of the terminal, the reference intermediate quantity is of the same type as the output value of the AI model, and the input value is measurement information or model input information obtained after processing the measurement information.

[0515] The reference value belongs to a reference output data set, and the input value belongs to a reference input data set.

[0516] The reference value belongs to a reference input data set, and the input value is measurement information or model input information obtained after processing the measurement information.

[0517] Optionally, the reference position information and the input value are obtained in the following manner:

[0518] Receiving first information from the network side device, the first information including the reference position information and the input value, the reference position information being position information of a first communication device with known position selected by the network side device, and the input value being measurement information of the first communication device or model input information obtained after processing the measurement information of the first communication device.

[0519] Optionally, the reference intermediate quantity and the input value are obtained in the following manner:

[0520] Receiving the reference intermediate quantity and the input value from the network side device;

[0521] The reference intermediate quantity is obtained based on reference position information reported by a first communication device with known position and second information used to assist in determining the reference intermediate quantity, or the reference intermediate quantity is determined by the first communication device and reported to the network side device.

[0522] The input value is measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

[0523] Optionally, the second information includes at least one of the following: a current position of a transmission and reception point (TRP) participating in AI positioning, historical data of measurement information, historical data of the AI model output value, a historical position of the terminal, a historical position of the TRP, and a historical position of the first communication device.

[0524] Optionally, the reference intermediate quantity includes at least one of the following: line-of-sight / non-line-of-sight indication; reference signal time difference (RSTD); relative time of arrival (RTOA); receive-transmit (Rx-Tx) time difference; time of arrival (TOA); angle of arrival (AoA); angle of departure (AoD); reference signal received power (RSRP); and reference signal received path power (RSRPP).

[0525] Optionally, the measurement information includes at least one of the following: power, time delay, phase, channel impulse response (CIR), power delay profile (PDP), and delay profile (DP).

[0526] Optionally, the reference output data set or the reference input data set is autonomously generated by the terminal, autonomously generated by the network side device, generated by the first device, or predefined by a protocol.

[0527] Optionally, the determination of the test / monitoring result of the AI model based on the reference value and the test / monitoring value includes:

[0528] calculating a difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0529] determining the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0530] wherein the first number is greater than or equal to 1.

[0531] Optionally, the calculation of the difference between the reference value and the test / monitoring value of the first number of paths or samples includes at least one of the following:

[0532] calculating an Euclidean distance between a vector composed of the reference value and a vector composed of the test / monitoring value of the first number of paths or samples;

[0533] calculating a cosine similarity between a vector composed of the reference value and a vector composed of the test / monitoring value of the first number of paths or samples;

[0534] calculating a root mean square error between a vector composed of the reference value and a vector composed of the test / monitoring value of the first number of paths or samples.

[0535] calculating the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0536] calculating the average of the absolute value sum of the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0537] calculating the root mean square error of the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0538] calculating the straight line distance between the reference value and the test / monitoring value of the first number of paths or samples;

[0539] calculating the average of the straight line distance between the reference value and the test / monitoring value of the first number of paths or samples;

[0540] calculating the root mean square of the straight line distance between the reference value and the test / monitoring value of the first number of paths or samples.

[0541] Optionally, determining the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first number of paths or samples, comprises:

[0542] determining that the test / monitoring result of the AI model is applicable in the case that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement;

[0543] determining that the test / monitoring result of the AI model is not applicable in the case that the difference between the reference value and the test / monitoring value of the first number of paths or samples does not meet the predefined accuracy requirement.

[0544] Optionally, the predefined accuracy requirement is applicable to a specified condition, and the specified condition comprises at least one of the following: signal-to-interference noise ratio, signal-to-noise ratio, channel strength, noise strength, channel model, reference signal bandwidth, reference signal configuration, subcarrier spacing, carrier frequency, frequency band information, duplexing mode, TRP configuration, UE configuration, AI model structure, AI model complexity, AI quantization information.

[0545] Optionally, in the case that the reference value is a reference intermediate quantity and the reference intermediate quantity is a line-of-sight / non-line-of-sight indication, the first probability under the specified condition is taken as the reference accuracy;

[0546] The difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement, comprising:

[0547] calculating a second probability that the reference value and the test / monitoring value of the first number of paths or samples are different;

[0548] in case that the first probability represents a maximum mismatch probability of the line-of-sight / non-line-of-sight indication, and the second probability is less than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement;

[0549] or,

[0550] a third probability that the reference value and the test / monitoring value under the first number of paths or samples are the same is calculated;

[0551] in case that the first probability represents a minimum match probability of the line-of-sight / non-line-of-sight indication, and the third probability is greater than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement.

[0552] Optionally, in case that the reference value belongs to a reference input data set, and the test / monitoring value is the input value,

[0553] the calculation of the difference between the reference value and the test / monitoring value of the first number of paths or samples includes one of the following:

[0554] in case that the input value is power or time delay or phase of the first number of paths or samples, the difference between the input value and a reference value of the same type in the reference input data set is calculated;

[0555] in case that the input value is a first vector composed of power or time delay or phase of the first number of paths or samples, the difference between the first vector and a second vector composed of reference values of the same type in the reference input data set is calculated;

[0556] in case that the input value is a combination of power, time delay and phase of the first number of paths or samples or any two of them, the difference between the input value and a reference value of the same type in the reference input data set is calculated;

[0557] in case that the input value is a third vector composed of power, time delay and phase of the first number of paths or samples or any two of them, the difference between the third vector and a fourth vector composed of reference values of the same type in the reference input data set is calculated.

[0558] Optionally, the apparatus further comprises a first processing unit for:

[0559] in case that the test / monitoring result is not applicable, the terminal sends the test / monitoring result to the network side device;

[0560] The terminal receives first instruction information from the network-side device, where the first instruction information is used to instruct the terminal to perform a second operation;

[0561] The second operation includes at least one of the following:

[0562] Update the current model;

[0563] Fine-tune the current model;

[0564] Retrain the current model;

[0565] Deactivate the current model;

[0566] The terminal falls back to non-AI mode;

[0567] Restart the current AI positioning related measurements;

[0568] Stop the current AI positioning related measurements;

[0569] Extend the current AI positioning related measurements.

[0570] Optionally, the AI ​​model includes a first model, or an optimized model of the first model;

[0571] The first model includes at least one of the following:

[0572] a first model generated based on at least one of a protocol-standardized model structure, protocol-standardized model parameters, and a protocol-standardized dataset for training the model;

[0573] A first model determined based on a model structure standardized by a protocol and model parameters received by the terminal;

[0574] a received first model, the first model comprising at least a portion of a model structure and / or a portion of model parameters;

[0575] a first model generated or determined based on a first data set, where the first data set is a protocol-standardized data set or a data set sent by the network-side device to the terminal;

[0576] The terminal generates or determines the first model based on a second data set and a protocol standardized model structure, where the second data set is a protocol standardized data set or a data set sent to the terminal by a peer device of the terminal.

[0577] The test or monitoring apparatus for positioning AI model in the embodiments of the present application can be an electronic device, for example, an electronic device with an operating system, or a component in the electronic device, for example, an integrated circuit or a chip. The electronic device can be a terminal or other device. For example, the terminal can include, but is not limited to, the types of terminal 11 listed above, and the other device can be a server, a network attached storage (NAS), etc., which are not limited in the embodiments of the present application.

[0578] The test or monitoring apparatus for positioning AI model provided in the embodiments of the present application can realize the processes of the method embodiments and achieve the same technical effects. To avoid repetition, the details are not described here. Figure 2 The test or monitoring apparatus for positioning AI model provided in the embodiments of the present application can realize the processes of the method embodiments and achieve the same technical effects. To avoid repetition, the details are not described here.

[0579] Figure 5 FIG. 2 is a structural schematic diagram of the test or monitoring apparatus for positioning AI model provided in the embodiments of the present application. As shown in FIG. 2, the test or monitoring apparatus 200 for positioning AI model includes: Figure 5

[0580] The second obtaining unit 510 is configured to obtain a reference value and an input value.

[0581] The second execution unit 520 is configured to perform a third operation, and the third operation includes one of the following:

[0582] The reference value and the input value are sent to a terminal, and the reference value and the input value are used to determine a test / monitoring result of an AI model for positioning deployed locally in the terminal;

[0583] The input value is sent to a terminal, and an output value from the terminal is received, the output value is obtained by inputting the input value into an AI model for positioning deployed locally in the terminal, based on the reference value and a test / monitoring value, a test / monitoring result of the AI model is determined, and the test / monitoring value is the output value;

[0584] The input value is input into an AI model for positioning deployed locally in the network side device, an output value of the AI model is obtained, based on the reference value and a test / monitoring value, a test / monitoring result of the AI model is determined, and the test / monitoring value is the output value;

[0585] Based on the reference value and a test / monitoring value, a test / monitoring result of the AI model is determined, and the test / monitoring value is the input value.

[0586] ​In the embodiments of the present application, by obtaining the reference value and the input value, the verification or monitoring of the AI model for positioning is realized based on the reference value and the input value, thereby improving the reliability of the AI model for positioning in actual environment deployment, and effectively ensuring the performance of AI positioning.

[0587] Optionally, the reference value and the input value include one of the following:

[0588] The reference value is reference position information, and the input value is measurement information or model input information obtained after processing the measurement information.

[0589] The reference value is a reference intermediate quantity, the reference intermediate quantity is intermediate information used to determine the terminal position, the reference intermediate quantity is of the same type as the output value of the AI model, and the input value is measurement information or model input information obtained after processing the measurement information.

[0590] The reference value belongs to a reference output data set, and the input value belongs to a reference input data set.

[0591] The reference value belongs to a reference input data set, and the input value is measurement information or model input information obtained after processing the measurement information.

[0592] Optionally, the reference position information and the input value are obtained in the following manner:

[0593] The position information of a first communication device with known position is selected as the reference position information, and the input value is measurement information corresponding to the first communication device or model input information obtained after processing the measurement information corresponding to the first communication device.

[0594] Optionally, the reference intermediate quantity and the input value are obtained in one of the following manners:

[0595] The reference position information and the measurement information reported by a first communication device with known position are received, the reference intermediate quantity is determined based on the reference position information and second information used to assist in determining the reference intermediate quantity, and the measurement information or model input information obtained after processing the measurement information is taken as the input value.

[0596] The reference intermediate quantity and the input value determined and reported by a first communication device with known position are received, and the input value is measurement information of the first communication device or model input information obtained after processing the measurement information of the first communication device.

[0597] Optionally, the second information comprises at least one of: a current location of a transmission and reception point (TRP) participating in AI positioning, historical data of measurement information, historical data of AI model output values, historical locations of the terminal, historical locations of the TPR, and historical locations of the first communication device.

[0598] Optionally, the reference intermediate quantity comprises at least one of: a line-of-sight / non-line-of-sight indication, a reference signal time difference (RSTD), a relative time of arrival (RTOA), a receive-transmit (Rx-Tx) time difference, a time of arrival (TOA), an angle of arrival (AoA), an angle of departure (AoD), a reference signal received power (RSRP), and a reference signal received path power (RSRPP).

[0599] Optionally, the measurement information comprises at least one of: power, time delay, phase, a channel impulse response (CIR), a power delay profile (PDP), and a delay profile (DP).

[0600] Optionally, the reference output data set or the reference input data set is autonomously generated by the terminal, autonomously generated by the network side device, generated by the first device, or predefined by a protocol.

[0601] Optionally, the determination of the test / monitoring result of the AI model based on the reference values and the test / monitoring values comprises:

[0602] calculating a difference between the reference values and the test / monitoring values of the first number of paths or samples;

[0603] determining the test / monitoring result of the AI model based on the difference between the reference values and the test / monitoring values of the first number of paths or samples;

[0604] wherein the first number is greater than or equal to 1.

[0605] Optionally, the calculation of the difference between the reference values and the test / monitoring values of the first number of paths or samples comprises at least one of:

[0606] calculating an Euclidean distance between a vector composed of the reference values and a vector composed of the test / monitoring values of the first number of paths or samples;

[0607] calculating a cosine similarity between a vector composed of the reference values and a vector composed of the test / monitoring values of the first number of paths or samples;

[0608] calculating a root mean square error between a vector composed of the reference values and a vector composed of the test / monitoring values of the first number of paths or samples;

[0609] calculating a difference between the reference values and the test / monitoring values of the first number of paths or samples;

[0610] calculating an average of absolute values of differences between the reference values and the test / monitoring values of the first quantity of paths or samples;

[0611] calculating a root mean square error of differences between the reference values and the test / monitoring values of the first quantity of paths or samples;

[0612] calculating a straight line distance between the reference values and the test / monitoring values of the first quantity of paths or samples;

[0613] calculating an average of straight line distances between the reference values and the test / monitoring values of the first quantity of paths or samples;

[0614] calculating a root mean square of straight line distances between the reference values and the test / monitoring values of the first quantity of paths or samples.

[0615] Optionally, the determining the test / monitoring result of the AI model based on the differences between the reference values and the test / monitoring values of the first quantity of paths or samples comprises:

[0616] in a case where the differences between the reference values and the test / monitoring values of the first quantity of paths or samples satisfy a predefined accuracy requirement, determining the test / monitoring result of the AI model as applicable;

[0617] in a case where the differences between the reference values and the test / monitoring values of the first quantity of paths or samples do not satisfy the predefined accuracy requirement, determining the test / monitoring result of the AI model as inapplicable.

[0618] Optionally, the predefined accuracy requirement is applicable to a specified condition, and the specified condition comprises at least one of a signal-to-interference-plus-noise ratio, a signal-to-noise ratio, a channel strength, a noise strength, a channel model, a reference signal bandwidth, a reference signal configuration, a subcarrier spacing, a carrier frequency, frequency band information, a duplexing mode, a TRP configuration, a UE configuration, an AI model structure, an AI model complexity, and AI quantization information.

[0619] Optionally, in a case where the reference value is a reference intermediate quantity and the reference intermediate quantity is a line-of-sight / non-line-of-sight indication, a first probability under the specified condition is taken as a reference accuracy;

[0620] the differences between the reference values and the test / monitoring values of the first quantity of paths or samples satisfying the predefined accuracy requirement comprises:

[0621] calculating a second probability of the reference values and the test / monitoring values being different under the first quantity of paths or samples;

[0622] in case that the first probability represents a maximum mismatch probability of the line-of-sight / non-line-of-sight indication, and the second probability is less than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement;

[0623] or,

[0624] a third probability that the reference value and the test / monitoring value under the first number of paths or samples are the same is calculated;

[0625] in case that the first probability represents a minimum match probability of the line-of-sight / non-line-of-sight indication, and the third probability is greater than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement.

[0626] Optionally, in case that the reference value is a reference input data set, and the test / monitoring value is the input value,

[0627] the calculation of the difference between the reference value and the test / monitoring value of the first number of paths or samples comprises one of the following:

[0628] in case that the input value is power or time delay or phase of the first number of paths or samples, the difference between the input value and the reference value of the same type in the reference input data set is calculated;

[0629] in case that the input value is a first vector composed of power or time delay or phase of the first number of paths or samples, the difference between the first vector and a second vector composed of reference values of the same type in the reference input data set is calculated;

[0630] in case that the input value is a combination of power, time delay and phase of the first number of paths or samples or any two of them, the difference between the input value and the reference value of the same type in the reference input data set is calculated;

[0631] in case that the input value is a third vector composed of power, time delay and phase of the first number of paths or samples or any two of them, the difference between the third vector and a fourth vector composed of reference values of the same type in the reference input data set is calculated.

[0632] Optionally, the apparatus further comprises a second processing unit configured to:

[0633] receive a test / monitoring result from a terminal, the test / monitoring result being not applicable;

[0634] send first indication information to the terminal, the first indication information being used to instruct the terminal to perform a second operation;

[0635] The second operation includes at least one of:

[0636] updating the current model;

[0637] fine-tuning the current model;

[0638] retraining the current model;

[0639] deactivating the current model;

[0640] the terminal falls back to a non-AI mode;

[0641] restarting the current AI positioning-related measurement;

[0642] stopping the current AI positioning-related measurement;

[0643] extending the current AI positioning-related measurement.

[0644] Optionally, the apparatus further includes a third processing unit configured to:

[0645] in a case where the test / monitoring result is not applicable, sending second indication information to the terminal, the second indication information being used to instruct the terminal to perform a fourth operation;

[0646] The fourth operation includes one of restarting the current AI positioning-related measurement, stopping the current AI positioning-related measurement, and extending the current AI positioning-related measurement.

[0647] Optionally, the AI model includes a first model, or an optimized model of the first model.

[0648] The first model includes at least one of:

[0649] a first model generated based on at least one of a protocol-standardized model structure, protocol-standardized model parameters, and a protocol-standardized data set for training a model;

[0650] a first model determined based on a protocol-standardized model structure and received model parameters;

[0651] a received first model, the first model including at least part of a model structure and / or part of model parameters;

[0652] a first model generated or determined based on a first data set, the first data set being a protocol-standardized data set or a data set autonomously generated by a network-side device;

[0653] a first model generated or determined based on a second data set and a standardized model structure, the second data set being a protocol-standardized data set or a data set autonomously generated by a network-side device or a test device.

[0654] The test or monitoring apparatus for positioning AI model in the embodiments of the present application can be an electronic device, for example, an electronic device with an operating system, or a component in the electronic device, for example, an integrated circuit or a chip. The electronic device can be a network side device, or other devices other than the network side device. For example, the terminal can include but is not limited to the types of the network side device 12 listed above, and the other devices can be a server, a network attached storage (NAS), etc., which are not limited in the embodiments of the present application.

[0655] The test or monitoring apparatus for positioning AI model provided in the embodiments of the present application can realize the various processes of the method embodiments and achieve the same technical effects. To avoid repetition, the details are not described here. Figure 3 The method embodiments realize the various processes and achieve the same technical effects. To avoid repetition, the details are not described here.

[0656] As shown in Figure 6 , the embodiments of the present application also provide a communication device 600, which includes a processor 601 and a memory 602, and the memory 602 stores programs or instructions executable on the processor 601. For example, when the communication device 600 is a terminal, the programs or instructions are executed by the processor 601 to realize the various steps of the above-mentioned test or monitoring method for positioning AI model embodiments, and achieve the same technical effects. When the communication device 600 is a network side device, the programs or instructions are executed by the processor 601 to realize the various steps of the above-mentioned test or monitoring method for positioning AI model embodiments, and achieve the same technical effects. To avoid repetition, the details are not described here.

[0657] The embodiments of the present application also provide a terminal, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to realize the steps in the method embodiments as shown in Figure 2 . The terminal embodiments correspond to the above-mentioned terminal side method embodiments, and the various implementation processes and implementation manners of the above-mentioned method embodiments can be applied to the terminal embodiments, and achieve the same technical effects. Specifically, Figure 7 A hardware structure diagram of a terminal for realizing the embodiments of the present application.

[0658] The terminal 700 includes but is not limited to at least part of the components such as a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, and a processor 710.

[0659] Those skilled in the art can understand that the terminal 700 can also include a power supply (such as a battery) for supplying power to each component, and the power supply can be logically connected to the processor 710 through a power management system, so that the power management system can realize the functions of managing charging, discharging and power consumption management. Figure 7 The terminal structure shown in the figure does not constitute a limitation on the terminal, and the terminal can include more or fewer components than the figure, or combine certain components, or different component arrangements, which are not described here.

[0660] It should be understood that in the embodiments of the present application, the input unit 704 can include a graphics processing unit (GPU) 7041 and a microphone 7042. The graphics processor 7041 processes image data of a still picture or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 706 can include a display panel 7061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 707 includes at least one of a touch panel 7071 and other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 can include two parts of a touch detection device and a touch controller. The other input devices 7072 can include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), trackballs, mice, joysticks, which are not described here.

[0661] In the embodiments of the present application, after the radio frequency unit 701 receives the downlink data from the network side device, it can be transmitted to the processor 710 for processing. In addition, the radio frequency unit 701 can send uplink data to the network side device. Generally, the radio frequency unit 701 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc.

[0662] The memory 709 can be used to store software programs or instructions and various data. The memory 709 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory 709 can include a volatile memory or a non-volatile memory. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable Programmable ROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synchlink DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 709 in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0663] The processor 710 can include one or more processing units; optionally, the processor 710 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 710.

[0664] The processor 710 is configured to obtain a reference value and an input value; and perform a first operation, the first operation including one of the following: inputting the input value into an AI model for positioning deployed locally in the terminal, obtaining an output value of the AI model, taking the output value as a test / monitoring value, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value; and taking the input value as a test / monitoring value, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value.

[0665] In the embodiments of the present application, by obtaining the reference value and the input value, inputting the input value into the AI model for positioning locally deployed by the terminal, obtaining the output value of the AI model, taking the output value as the test / monitoring value, or taking the input value as the test / monitoring value, determining the test / monitoring result of the AI model based on the reference value and the test / monitoring value, the verification or monitoring of the AI model for positioning is realized, thereby improving the reliability of the AI model for positioning deployed in the actual environment, and the performance of AI positioning can be effectively guaranteed.

[0666] Optionally, the reference value and the input value include one of the following:

[0667] The reference value is reference position information, and the input value is measurement information or model input information obtained after processing the measurement information;

[0668] The reference value is a reference intermediate quantity, the reference intermediate quantity is intermediate information used to determine the position of the terminal, the reference intermediate quantity is of the same type as the output value of the AI model, and the input value is measurement information or model input information obtained after processing the measurement information;

[0669] The reference value belongs to a reference output data set, and the input value belongs to a reference input data set;

[0670] The reference value belongs to a reference input data set, and the input value is measurement information or model input information obtained after processing the measurement information.

[0671] Optionally, the reference position information and the input value are obtained in the following manner:

[0672] The terminal receives first information from the network side device, the first information includes the reference position information and the input value, the reference position information is the position information of a first communication device with known position selected by the network side device, and the input value is measurement information of the first communication device or model input information obtained after processing the measurement information of the first communication device.

[0673] Optionally, the reference intermediate quantity and the input value are obtained in the following manner:

[0674] The reference intermediate quantity and the input value are received from the network side device;

[0675] The reference intermediate quantity is obtained based on reference position information reported by a first communication device with known position and second information used to assist in determining the reference intermediate quantity, or the reference intermediate quantity is determined by the first communication device and reported to the network side device;

[0676] The input value is measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

[0677] Optionally, the second information includes at least one of the following: a current position of a transmission and reception point (TRP) participating in AI positioning, historical data of measurement information, historical data of the AI model output value, a historical position of the terminal, a historical position of the TRP, and a historical position of the first communication device.

[0678] Optionally, the reference intermediate quantity includes at least one of the following: line-of-sight / non-line-of-sight indication; reference signal time difference (RSTD); relative time of arrival (RTOA); receive-transmit (Rx-Tx) time difference; time of arrival (TOA); angle of arrival (AoA); angle of departure (AoD); reference signal received power (RSRP); and reference signal received path power (RSRPP).

[0679] Optionally, the measurement information includes at least one of the following: power, time delay, phase, channel impulse response (CIR), power delay profile (PDP), and delay profile (DP).

[0680] Optionally, the reference output data set or the reference input data set is autonomously generated by the terminal, autonomously generated by the network side device, generated by the first device, or predefined by a protocol.

[0681] Optionally, the determination of the test / monitoring result of the AI model based on the reference value and the test / monitoring value includes:

[0682] calculating a difference between the reference value and the test / monitoring value of a first number of paths or samples;

[0683] determining the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0684] wherein the first number is greater than or equal to 1.

[0685] Optionally, the calculation of the difference between the reference value and the test / monitoring value of the first number of paths or samples includes at least one of the following:

[0686] calculating an Euclidean distance between a vector composed of the reference value and a vector composed of the test / monitoring value of the first number of paths or samples;

[0687] calculating a cosine similarity between a vector composed of the reference value and a vector composed of the test / monitoring value of the first number of paths or samples;

[0688] calculating a root mean square error between a vector composed of the reference value and a vector composed of the test / monitoring value of the first number of paths or samples.

[0689] calculating the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0690] calculating the average of the absolute value sum of the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0691] calculating the root mean square error of the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0692] calculating the straight line distance between the reference value and the test / monitoring value of the first number of paths or samples;

[0693] calculating the average of the straight line distance between the reference value and the test / monitoring value of the first number of paths or samples;

[0694] calculating the root mean square of the straight line distance between the reference value and the test / monitoring value of the first number of paths or samples.

[0695] Optionally, determining the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first number of paths or samples, comprises:

[0696] determining that the test / monitoring result of the AI model is applicable when the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement;

[0697] determining that the test / monitoring result of the AI model is not applicable when the difference between the reference value and the test / monitoring value of the first number of paths or samples does not meet the predefined accuracy requirement.

[0698] Optionally, the predefined accuracy requirement is applicable to a specified condition, and the specified condition comprises at least one of the following: signal-to-interference noise ratio, signal-to-noise ratio, channel strength, noise strength, channel model, reference signal bandwidth, reference signal configuration, subcarrier spacing, carrier frequency, frequency band information, duplexing mode, TRP configuration, UE configuration, AI model structure, AI model complexity, AI quantization information.

[0699] Optionally, when the reference value is a reference intermediate quantity and the reference intermediate quantity is a line-of-sight / non-line-of-sight indication, the first probability under the specified condition is taken as the reference accuracy;

[0700] The difference between the reference value and the test / monitoring value of the first number of paths or samples meeting the predefined accuracy requirement comprises:

[0701] calculating a second probability that the reference value and the test / monitoring value of the first number of paths or samples are different;

[0702] in a case where the first probability represents a maximum mismatch probability of the line-of-sight / non-line-of-sight indication, and the second probability is less than or equal to the first probability, determining that a difference between the reference value and the test / monitoring value of the first number of paths or samples satisfies a predefined accuracy requirement;

[0703] or,

[0704] calculating a third probability that the reference value and the test / monitoring value are the same under a second number of paths or samples;

[0705] in a case where the first probability represents a minimum match probability of the line-of-sight / non-line-of-sight indication, and the third probability is greater than or equal to the first probability, determining that a difference between the reference value and the test / monitoring value of the first number of paths or samples satisfies a predefined accuracy requirement.

[0706] Optionally, in a case where the reference value belongs to a reference input data set, and the test / monitoring value is the input value,

[0707] the calculating the difference between the reference value and the test / monitoring value of the first number of paths or samples comprises one of:

[0708] in a case where the input value is power or time delay or phase of the first number of paths or samples, calculating a difference between the input value and a reference value of the same type in the reference input data set;

[0709] in a case where the input value is a first vector composed of power or time delay or phase of the first number of paths or samples, calculating a difference between the first vector and a second vector composed of reference values of the same type in the reference input data set;

[0710] in a case where the input value is a combination of power, time delay and phase of the first number of paths or samples or any two of them, calculating a difference between the input value and a reference value of the same type in the reference input data set;

[0711] in a case where the input value is a third vector composed of power, time delay and phase of the first number of paths or samples or any two of them, calculating a difference between the third vector and a fourth vector composed of reference values of the same type in the reference input data set.

[0712] Optionally, the processor 710 is further configured to:

[0713] in a case where the test / monitoring result is not applicable, sending the test / monitoring result to the network side device;

[0714] receiving first indication information from the network side device, the first indication information being used to instruct the terminal to perform a second operation;

[0715] The second operation comprises at least one of:

[0716] updating the current model;

[0717] fine-tuning the current model;

[0718] retraining the current model;

[0719] deactivating the current model;

[0720] the terminal falls back to a non-AI mode;

[0721] restarting the current AI positioning-related measurement;

[0722] stopping the current AI positioning-related measurement;

[0723] extending the current AI positioning-related measurement.

[0724] Optionally, the AI model comprises a first model, or an optimized model of the first model.

[0725] The first model comprises at least one of:

[0726] a first model generated based on at least one of a protocol-standardized model structure, protocol-standardized model parameters and a protocol-standardized data set for training a model;

[0727] a first model determined based on a protocol-standardized model structure and model parameters received by the terminal;

[0728] a received first model, the first model comprising at least part of a model structure and / or part of model parameters;

[0729] a first model generated or determined based on a first data set, the first data set being a protocol-standardized data set or a data set sent by the network-side device to the terminal;

[0730] a first model generated or determined by the terminal based on a second data set and a protocol-standardized model structure, the second data set being a protocol-standardized data set or a data set sent by a peer device of the terminal to the terminal.

[0731] It can be understood that the implementation process of each implementation manner mentioned in the embodiment can refer to the related description of the foregoing terminal-side method embodiment and achieve the same or corresponding technical effects. To avoid repetition, it will not be described here again.

[0732] The embodiments of the present application also provide a network-side device comprising a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is configured to run programs or instructions to implement the method as described above.Figure 3 The steps of the method embodiment are shown. The network side device embodiment corresponds to the network side device method embodiment described above, and each implementation process and implementation manner of the method embodiment can be applied to the network side device embodiment and achieve the same technical effects.

[0733] Specifically, the embodiments of the present application also provide a network side device. As shown in Figure 8 The network side device 800 includes a processor 801, a network interface 802 and a memory 803. The network interface 802 is, for example, a common public radio interface (CPRI).

[0734] Specifically, the network side device 800 of the embodiments of the present application further includes instructions or programs stored on the memory 803 and executable on the processor 801, and the processor 801 invokes the instructions or programs in the memory 803 to perform the methods executed by the modules shown in Figure 5 The methods executed by the modules shown in the above embodiments and achieve the same technical effects. To avoid repetition, they will not be described here.

[0735] Specifically, the embodiments of the present application also provide a network side device. As shown in Figure 9 The network side device 900 includes an antenna 91, a radio frequency device 92, a baseband device 93, a processor 94 and a memory 95. The antenna 91 is connected to the radio frequency device 92. In the uplink direction, the radio frequency device 92 receives information through the antenna 91 and sends the received information to the baseband device 93 for processing. In the downlink direction, the baseband device 93 processes the information to be sent and sends it to the radio frequency device 92, which processes the received information and sends it out through the antenna 91.

[0736] The method executed by the network side device in the above embodiments can be implemented in the baseband device 93, which includes a baseband processor.

[0737] The baseband device 93 may, for example, include at least one baseband board on which a plurality of chips are disposed, as shown in Figure 9 One of the chips is, for example, a baseband processor connected to the memory 95 through a bus interface to invoke the programs in the memory 95 and perform the network device operations shown in the above method embodiments.

[0738] The network side device can also include a network interface 96, which is, for example, a common public radio interface (CPRI).

[0739] Specifically, the network side device 900 of the embodiment of the application further includes instructions or programs stored on the storage 95 and executable on the processor 94, and the processor 94 invokes the instructions or programs in the storage 95 to perform the method executed by each module shown in the figure and achieve the same technical effects. To avoid repetition, details are not described herein. Figure 5 The method executed by each module shown in the figure and achieve the same technical effects. To avoid repetition, details are not described herein.

[0740] The embodiment of the application further provides a readable storage medium, and the readable storage medium stores programs or instructions, and the programs or instructions are executed by a processor to implement each process of the above-mentioned test or monitoring method for positioning artificial intelligence AI model, and the same technical effects can be achieved. To avoid repetition, details are not described herein.

[0741] The processor is the processor in the terminal in the above-mentioned embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc. In some examples, the readable storage medium can be a non-transient readable storage medium.

[0742] The embodiment of the application further provides a chip, and the chip includes a processor and a communication interface. The communication interface is coupled with the processor, and the processor is used to run programs or instructions to implement each process of the above-mentioned test or monitoring method for positioning artificial intelligence AI model, and the same technical effects can be achieved. To avoid repetition, details are not described herein.

[0743] It should be understood that the chip mentioned in the embodiment of the application can also be referred to as a system chip, a system chip, a chip system or a system on chip, etc.

[0744] The embodiment of the application further provides a computer program / program product, and the computer program / program product is stored in a storage medium. The computer program / program product is executed by at least one processor to implement each process of the above-mentioned test or monitoring method for positioning artificial intelligence AI model, and the same technical effects can be achieved. To avoid repetition, details are not described herein.

[0745] The embodiment of the application further provides a test or monitoring system for positioning artificial intelligence AI model, and the test or monitoring system includes a terminal and a network side device. The terminal can be used to execute the steps of the above-mentioned test or monitoring method for positioning artificial intelligence AI model, and the network side device can be used to execute the steps of the above-mentioned test or monitoring method for positioning artificial intelligence AI model.

[0746] It should be noted that, in the present document, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a", "comprising", or the like does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element. Furthermore, it is to be understood that the methods and apparatuses of the present application can be carried out by specific hardware, by software, or by a combination of hardware and software. It is therefore, contemplated to this patent to cover any and all modifications, variations, or equivalents that fall within the scope of the present application. Accordingly, where a concept can have been illustrated in only one of the exemplary embodiments, various aspects of the concept can be modified and / or combined to produce a variety of other embodiments that are not specifically illustrated. Thus, for purposes of describing the present application, certain aspects of the application can be presented in terms of sequences of actions, but it should be appreciated that these sequences are examples and are not limiting. The sequences of actions could be changed, and other sequences could be implemented. Moreover, it should be appreciated that sometimes it is easier to describe one aspect of the application in terms of another aspect of the application. Therefore, the description herein of one aspect of the application in terms of another aspect of the application is used merely to more particularly exemplify the application. It should be appreciated that the use of the word "about" in describing the application is intended to mean that the amount or value in question is not exact but is intended to include the amount or value in question plus or minus 10% of the amount or value in question.

[0747] From the above description of the embodiments of the present application, it is apparent that the above-described method of the embodiments can be realized by means of a computer software product and general hardware platform, of course, it can also be realized by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disc, optical disc, etc.), and includes a plurality of instructions for making the terminal or network side device execute the method described in each embodiment of the present application.

[0748] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are merely illustrative, but not restrictive. Those skilled in the art can make many forms of embodiments under the inspiration of the present application without departing from the scope of the present application and the protection scope of the claims, and these embodiments all belong to the protection scope of the present application.

Claims

1. A method for testing or monitoring an artificial intelligence (AI) model for positioning, characterized in that: include: The terminal obtains the reference value and the input value; The terminal performs a first operation, where the first operation includes one of the following: Inputting the input value into an AI model for positioning deployed locally on the terminal, obtaining an output value of the AI ​​model, using the output value as a test / monitoring value, and determining a test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value; The input value is used as a test / monitoring value, and a test / monitoring result of the AI ​​model is determined based on the reference value and the test / monitoring value.

2. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 1, characterized in that: The reference value and input value include one of the following: The reference value is reference position information, and the input value is measurement information or model input information obtained by processing the measurement information; The reference value is a reference intermediate quantity, which is intermediate information used to determine the terminal position. The reference intermediate quantity is of the same type as the output value of the AI ​​model. The input value is measurement information or model input information obtained by processing the measurement information. The reference values ​​belong to a reference output data set, and the input values ​​belong to a reference input data set; The reference value belongs to a reference input data set, and the input value is measurement information or model input information obtained by processing the measurement information.

3. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 2, wherein: The method for obtaining the reference position information and the input value includes: The terminal receives first information from a network side device, where the first information includes the reference location information and an input value. The reference location information is location information of a first communication device with a known location selected by the network side device, and the input value is measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

4. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 2, wherein: The method for obtaining the reference intermediate quantity and the input value includes: Receive reference intermediate quantities and input values ​​from network-side devices; The reference intermediate quantity is obtained based on reference location information reported by a first communication device whose location is known and second information, where the second information is used to assist in determining the reference intermediate quantity, or the reference intermediate quantity is determined by the first communication device and reported to the network-side device; The input value is measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

5. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 4, characterized in that: The second information includes at least one of the following: the TRP position of the transmitting and receiving point currently participating in AI positioning, historical data of measurement information, historical data of the AI ​​model output value, the historical position of the terminal, the historical position of the TRP, and the historical position of the first communication device.

6. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 2, 4 or 5, characterized in that: The reference intermediate quantity includes at least one of the following: line-of-sight / non-line-of-sight indication; reference signal time difference RSTD; relative time of arrival RTOA; receive-transmit Rx-Tx time difference; arrival time TOA; arrival angle AoA; departure angle AoD; reference signal received power RSRP; reference signal received path power RSRPP.

7. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to any one of claims 2 to 6, characterized in that: The measurement information includes at least one of the following: power, delay, phase, channel impulse response CIR, power delay profile PDP, and delay profile DP.

8. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 2, wherein: The reference output data set or the reference input data set is autonomously generated by the terminal, or autonomously generated by a network-side device, or generated by the first device, or predefined by a protocol.

9. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to any one of claims 1 to 8, characterized in that: Determining the test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value includes: calculating a difference between the reference value and the test / monitoring value for a first quantity diameter or sample; determining a test / monitoring result of the AI ​​model based on a difference between the reference value and the test / monitoring value of the first quantity diameter or sample; The first number is greater than or equal to 1.

10. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 9, characterized in that: Calculating the difference between the reference value and the test / monitoring value of the first quantity diameter or sample comprises at least one of the following: Calculating the Euclidean distance between a vector of reference values ​​and a vector of test / monitoring values ​​for a first quantity path or sample; Calculating the cosine similarity between the vector consisting of the reference values ​​and the vector consisting of the test / monitoring values ​​of the first quantity path or sample; Calculating the root mean square error between a vector of reference values ​​and a vector of test / monitoring values ​​for a first quantity diameter or sample; calculating a difference between a reference value and a test / monitoring value for a first quantity diameter or sample; calculating the average of the sum of the absolute values ​​of the differences between the reference value and the test / monitoring value for the first quantity diameter or sample; calculating a root mean square error of the difference between the reference value and the test / monitoring value for the first quantity diameter or sample; calculating a first quantity diameter or a straight-line distance between a reference value and a test / monitoring value for the sample; calculating an average of the first number diameters or straight-line distances between the reference value and the test / monitoring value of the sample; Calculate the root mean square of the straight-line distance between the reference value and the test / monitor value for a first quantity diameter or sample.

11. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 9 or 10, characterized in that: The determining of the test / monitoring result of the AI ​​model based on the difference between the reference value and the test / monitoring value of the first quantity diameter or sample includes: If the difference between the reference value and the test / monitoring value of the first quantity diameter or sample meets a predefined accuracy requirement, determining the test / monitoring result of the AI ​​model as applicable; In a case where the difference between the reference value and the test / monitoring value of the first quantity diameter or sample does not meet a predefined accuracy requirement, the test / monitoring result of the AI ​​model is determined to be not applicable.

12. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 11, characterized in that: The predefined accuracy requirement applies to specified conditions, and the specified conditions include at least one of the following: signal-to-interference-plus-noise ratio, signal-to-noise ratio, channel strength, noise strength, channel model, reference signal bandwidth, reference signal configuration, subcarrier spacing, carrier frequency, frequency band information, duplex system, TRP configuration, UE configuration, AI model structure, AI model complexity, and AI quantization information.

13. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 12, characterized in that: In a case where the reference value is a reference intermediate quantity and the reference intermediate quantity is a line-of-sight / non-line-of-sight indication, taking the first probability under the specified condition as a reference accuracy; The difference between the reference value and the test / monitoring value of the first quantity diameter or sample satisfies a predefined accuracy requirement including: calculating a second probability that the reference value and the test / monitoring value for the first quantity size or sample are different; determining that a difference between a reference value and a test / monitoring value of the first quantity diameter or sample satisfies a predefined accuracy requirement when the first probability represents a maximum mismatch probability of a line-of-sight / non-line-of-sight indication and the second probability is less than or equal to the first probability; or, calculating a third probability that the reference value and the test / monitoring value are the same for the first quantity size or sample; When the first probability represents a minimum matching probability of line-of-sight / non-line-of-sight indication and the third probability is greater than or greater than the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first quantity diameter or sample meets the predefined accuracy requirement.

14. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 9, characterized in that: In the case where the reference value belongs to a reference input data set and the test / monitoring value is the input value, Calculating the difference between the reference value and the test / monitoring value of the first quantity diameter or sample comprises one of the following: In a case where the input value is a power or a time delay or a phase of a first quantity path or sample, calculating a difference between the input value and a reference value of the same type in the reference input data set; In a case where the input value is a first vector consisting of power or delay or phase of a first quantity path or sample, calculating the difference between the first vector and a second vector consisting of reference values ​​of the same type in the reference input data set; When the input value is a power, a delay, and a phase of a first quantity path or sample, or a combination of any two thereof, calculating a difference between the input value and a reference value of the same type in the reference input data set; When the input value is a third vector consisting of power, delay and phase of a first quantity path or sample, or any two of them, a difference between the third vector and a fourth vector consisting of reference values ​​of the same type in the reference input data set is calculated.

15. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to any one of claims 1 to 14, characterized in that: The method further comprises: If the test / monitoring result is not applicable, the terminal sends the test / monitoring result to the network side device; The terminal receives first instruction information from the network-side device, where the first instruction information is used to instruct the terminal to perform a second operation; The second operation includes at least one of the following: Update the current model; Fine-tune the current model; Retrain the current model; Deactivate the current model; The terminal falls back to non-AI mode; Restart the current AI positioning related measurements; Stop the current AI positioning related measurements; Extend the current AI positioning related measurements.

16. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to any one of claims 1 to 15, characterized in that: The AI ​​model includes a first model, or an optimized model of the first model; The first model includes at least one of the following: a first model generated based on at least one of a protocol-standardized model structure, protocol-standardized model parameters, and a protocol-standardized dataset for training the model; A first model determined based on a model structure standardized by a protocol and model parameters received by the terminal; a received first model, the first model comprising at least a portion of a model structure and / or a portion of model parameters; a first model generated or determined based on a first data set, where the first data set is a protocol-standardized data set or a data set sent to the terminal by a network-side device; The terminal generates or determines the first model based on a second data set and a protocol standardized model structure, where the second data set is a protocol standardized data set or a data set sent to the terminal by a peer device of the terminal.

17. A method for testing or monitoring an artificial intelligence (AI) model for positioning, characterized in that: include: The network side device obtains the reference value and input value; The network-side device performs a third operation, where the third operation includes one of the following: Sending the reference value and the input value to the terminal, where the reference value and the input value are used to determine a test / monitoring result of an AI model for positioning deployed locally on the terminal; Sending the input value to a terminal and receiving an output value from the terminal, wherein the output value is obtained by inputting the input value into an AI model for positioning deployed locally on the terminal, and determining a test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value, wherein the test / monitoring value is the output value; Inputting the input value into an AI model for positioning deployed locally on the network-side device, obtaining an output value of the AI ​​model, and determining a test / monitoring result of the AI ​​model based on the reference value and a test / monitoring value, where the test / monitoring value is the output value; Based on the reference value and the test / monitoring value, a test / monitoring result of the AI ​​model is determined, wherein the test / monitoring value is the input value.

18. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 17, characterized in that: The reference value and input value include one of the following: The reference value is reference position information, and the input value is measurement information or model input information obtained by processing the measurement information; The reference value is a reference intermediate quantity, which is intermediate information used to determine the terminal position. The reference intermediate quantity is of the same type as the output value of the AI ​​model. The input value is measurement information or model input information obtained by processing the measurement information. The reference values ​​belong to a reference output data set, and the input values ​​belong to a reference input data set; The reference value belongs to a reference input data set, and the input value is measurement information or model input information obtained by processing the measurement information.

19. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 18, characterized in that: The method for obtaining the reference position information and the input value includes: The network side device selects location information of a first communication device with a known location as the reference location information, and the input value is measurement information corresponding to the first communication device or model input information obtained by processing the measurement information corresponding to the first communication device.

20. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 18, wherein: The method for obtaining the reference intermediate quantity and the input value includes one of the following: receiving reference location information and measurement information reported by a first communication device with a known location, determining the reference intermediate quantity based on the reference location information and the second information, wherein the second information is used to assist in determining the reference intermediate quantity, and using the measurement information or model input information obtained by processing the measurement information as the input value; A reference intermediate quantity and an input value determined and reported by a first communication device with a known location are received, where the input value is measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

21. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 20, characterized in that: The second information includes at least one of the following: the TRP position of the transmitting and receiving point currently participating in AI positioning, historical data of measurement information, historical data of the AI ​​model output value, the historical position of the terminal, the historical position of the TPR, and the historical position of the first communication device.

22. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 18, 20 or 21, characterized in that: The reference intermediate quantity includes at least one of the following: line-of-sight / non-line-of-sight indication; reference signal time difference RSTD; relative arrival time RTOA; receive-transmit Rx-Tx time difference; arrival time TOA; arrival angle AoA; departure angle AoD; reference signal received power RSRP; reference signal received path power RSRPP.

23. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to any one of claims 17 to 22, characterized in that: Determining the test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value includes: calculating a difference between the reference value and the test / monitoring value for a first quantity diameter or sample; determining a test / monitoring result of the AI ​​model based on a difference between the reference value and the test / monitoring value of the first quantity diameter or sample; The first number is greater than or equal to 1.

24. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 23, wherein: Calculating the difference between the reference value and the test / monitoring value of the first quantity diameter or sample comprises at least one of the following: Calculating the Euclidean distance between a vector of reference values ​​and a vector of test / monitoring values ​​for a first quantity path or sample; Calculating the cosine similarity between the vector consisting of the reference values ​​and the vector consisting of the test / monitoring values ​​of the first quantity path or sample; Calculating the root mean square error between a vector of reference values ​​and a vector of test / monitoring values ​​for a first quantity diameter or sample; calculating a difference between a reference value and a test / monitoring value for a first quantity diameter or sample; calculating the average of the sum of the absolute values ​​of the differences between the reference value and the test / monitoring value for the first quantity diameter or sample; calculating a root mean square error of the difference between the reference value and the test / monitoring value for the first quantity diameter or sample; calculating a first quantity diameter or a straight-line distance between a reference value and a test / monitoring value for the sample; calculating an average of the first number diameters or straight-line distances between the reference value and the test / monitoring value of the sample; Calculate the root mean square of the straight-line distance between the reference value and the test / monitor value for a first quantity diameter or sample.

25. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 23 or 24, characterized in that: The determining of the test / monitoring result of the AI ​​model based on the difference between the reference value and the test / monitoring value of the first quantity diameter or sample includes: If the difference between the reference value and the test / monitoring value of the first quantity diameter or sample meets a predefined accuracy requirement, determining the test / monitoring result of the AI ​​model as applicable; In a case where the difference between the reference value and the test / monitoring value of the first quantity diameter or sample does not meet a predefined accuracy requirement, the test / monitoring result of the AI ​​model is determined to be not applicable.

26. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 25, characterized in that: The predefined accuracy requirement applies to specified conditions, and the specified conditions include at least one of the following: signal-to-interference-plus-noise ratio, signal-to-noise ratio, channel strength, noise strength, channel model, reference signal bandwidth, reference signal configuration, subcarrier spacing, carrier frequency, frequency band information, duplex system, TRP configuration, UE configuration, AI model structure, AI model complexity, and AI quantization information.

27. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 26, wherein: In a case where the reference value is a reference intermediate quantity and the reference intermediate quantity is a line-of-sight / non-line-of-sight indication, taking the first probability under the specified condition as a reference accuracy; The difference between the reference value and the test / monitoring value of the first quantity diameter or sample satisfies a predefined accuracy requirement including: calculating a second probability that the reference value and the test / monitoring value for the first quantity size or sample are different; determining that a difference between a reference value and a test / monitoring value of the first quantity diameter or sample satisfies a predefined accuracy requirement when the first probability represents a maximum mismatch probability of a line-of-sight / non-line-of-sight indication and the second probability is less than or equal to the first probability; or, calculating a third probability that the reference value and the test / monitoring value are the same for the first quantity size or sample; When the first probability represents a minimum matching probability of line-of-sight / non-line-of-sight indication and the third probability is greater than or greater than the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first quantity diameter or sample meets the predefined accuracy requirement.

28. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 23, wherein: In the case where the reference value is a reference input data set and the test / monitoring value is the input value, Calculating the difference between the reference value and the test / monitoring value of the first quantity diameter or sample comprises one of the following: In a case where the input value is a power or a time delay or a phase of a first quantity path or sample, calculating a difference between the input value and a reference value of the same type in the reference input data set; In a case where the input value is a first vector consisting of power or delay or phase of a first quantity path or sample, calculating a difference between the first vector and a second vector consisting of reference values ​​of the same type as the reference input data set; When the input value is a power, a delay, and a phase of a first quantity path or sample, or a combination of any two thereof, calculating a difference between the input value and a reference value of the same type in the reference input data set; When the input value is a third vector consisting of power, delay and phase of a first quantity path or sample, or any two of them, a difference between the third vector and a fourth vector consisting of reference values ​​of the same type in the reference input data set is calculated.

29. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to any one of claims 17 to 22, characterized in that: The method further comprises: receiving a test / monitoring result from a terminal, wherein the test / monitoring result is not applicable; Sending first instruction information to the terminal, where the first instruction information is used to instruct the terminal to perform a second operation; The second operation includes at least one of the following: Update the current model; Fine-tune the current model; Retrain the current model; Deactivate the current model; The terminal falls back to non-AI mode; Restart the current AI positioning related measurements; Stop the current AI positioning related measurements; Extend the current AI positioning related measurements.

30. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to any one of claims 17 to 29, characterized in that: The method further comprises: If the test / monitoring result is not applicable, the network-side device sends second indication information to the terminal, where the second indication information is used to instruct the terminal to perform a fourth operation; The fourth operation includes one of the following: restarting the current AI positioning related measurement, stopping the current AI positioning related measurement, and extending the current AI positioning related measurement.

31. A testing or monitoring device for an artificial intelligence (AI) model for positioning, characterized in that: include: A first acquiring unit, configured to acquire a reference value and an input value; The first execution unit is configured to execute a first operation, where the first operation includes one of the following: Inputting the input value into an AI model for positioning deployed locally on the terminal, obtaining an output value of the AI ​​model, using the output value as a test / monitoring value, and determining a test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value; The input value is used as a test / monitoring value, and a test / monitoring result of the AI ​​model is determined based on the reference value and the test / monitoring value.

32. The device for testing or monitoring an artificial intelligence (AI) model according to claim 31, wherein: The reference value and input value include one of the following: The reference value is reference position information, and the input value is measurement information or model input information obtained by processing the measurement information; The reference value is a reference intermediate quantity, which is intermediate information used to determine the terminal position. The reference intermediate quantity is of the same type as the output value of the AI ​​model. The input value is measurement information or model input information obtained by processing the measurement information. The reference values ​​belong to a reference output data set, and the input values ​​belong to a reference input data set; The reference value belongs to a reference input data set, and the input value is measurement information or model input information obtained by processing the measurement information.

33. The testing or monitoring device for an artificial intelligence (AI) model for positioning according to claim 32, characterized in that: The method for obtaining the reference position information and the input value includes: Receive first information from the network side device, the first information including the reference location information and an input value, the reference location information being the location information of a first communication device with a known location selected by the network side device, and the input value being the measurement information of the first communication device or the model input information obtained by processing the measurement information of the first communication device.

34. The testing or monitoring device for an artificial intelligence (AI) model for positioning according to claim 32, characterized in that: The method for obtaining the reference intermediate quantity and the input value includes: receiving a reference intermediate quantity and an input value from the network-side device; The reference intermediate quantity is obtained based on reference location information reported by a first communication device whose location is known and second information, where the second information is used to assist in determining the reference intermediate quantity, or the reference intermediate quantity is determined by the first communication device and reported to the network-side device; The input value is measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

35. The testing or monitoring device for an artificial intelligence (AI) model for positioning according to claim 32 or 34, characterized in that: The reference intermediate quantity includes at least one of the following: line-of-sight / non-line-of-sight indication; reference signal time difference RSTD; relative time of arrival RTOA; receive-transmit Rx-Tx time difference; arrival time TOA; arrival angle AoA; departure angle AoD; reference signal received power RSRP; reference signal received path power RSRPP.

36. The testing or monitoring device for an artificial intelligence (AI) model for positioning according to any one of claims 31 to 35, characterized in that: Determining the test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value includes: calculating a difference between the reference value and the test / monitoring value for a first quantity diameter or sample; determining a test / monitoring result of the AI ​​model based on a difference between the reference value and the test / monitoring value of the first quantity diameter or sample; The first number is greater than or equal to 1.

37. The testing or monitoring device for an artificial intelligence (AI) model for positioning according to claim 36, characterized in that: Calculating the difference between the reference value and the test / monitoring value of the first quantity diameter or sample comprises at least one of the following: Calculating the Euclidean distance between a vector of reference values ​​and a vector of test / monitoring values ​​for a first quantity path or sample; Calculating the cosine similarity between the vector consisting of the reference values ​​and the vector consisting of the test / monitoring values ​​of the first quantity path or sample; Calculating the root mean square error between a vector of reference values ​​and a vector of test / monitoring values ​​for a first quantity diameter or sample; calculating a difference between a reference value and a test / monitoring value for a first quantity diameter or sample; calculating the average of the sum of the absolute values ​​of the differences between the reference value and the test / monitoring value for the first quantity diameter or sample; calculating a root mean square error of the difference between the reference value and the test / monitoring value for the first quantity diameter or sample; calculating a first quantity diameter or a straight-line distance between a reference value and a test / monitoring value for the sample; calculating an average of the first number diameters or straight-line distances between the reference value and the test / monitoring value of the sample; Calculate the root mean square of the straight-line distance between the reference value and the test / monitor value for a first quantity diameter or sample.

38. A testing or monitoring device for an artificial intelligence (AI) model for positioning, characterized in that: include: A second acquiring unit, configured to acquire a reference value and an input value; The second execution unit is configured to execute a third operation, where the third operation includes one of the following: Sending the reference value and the input value to the terminal, where the reference value and the input value are used to determine a test / monitoring result of an AI model for positioning deployed locally on the terminal; Sending the input value to a terminal and receiving an output value from the terminal, wherein the output value is obtained by inputting the input value into an AI model for positioning deployed locally on the terminal, and determining a test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value, wherein the test / monitoring value is the output value; Inputting the input value into an AI model for positioning deployed locally on the network-side device, obtaining an output value of the AI ​​model, and determining a test / monitoring result of the AI ​​model based on the reference value and a test / monitoring value, where the test / monitoring value is the output value; Based on the reference value and the test / monitoring value, a test / monitoring result of the AI ​​model is determined, wherein the test / monitoring value is the input value.

39. The testing or monitoring device for an artificial intelligence (AI) model for positioning according to claim 38, characterized in that: Determining the test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value includes: calculating a difference between the reference value and the test / monitoring value for a first quantity diameter or sample; determining a test / monitoring result of the AI ​​model based on a difference between the reference value and the test / monitoring value of the first quantity diameter or sample; The first number is greater than or equal to 1.

40. The testing or monitoring device for an artificial intelligence (AI) model for positioning according to claim 38 or 39, characterized in that: The apparatus further includes a second processing unit, configured to: receiving a test / monitoring result from a terminal, wherein the test / monitoring result is not applicable; Sending first instruction information to the terminal, where the first instruction information is used to instruct the terminal to perform a second operation; The second operation includes at least one of the following: Update the current model; Fine-tune the current model; Retrain the current model; Deactivate the current model; The terminal falls back to non-AI mode; Restart the current AI positioning related measurements; Stop the current AI positioning related measurements; Extend the current AI positioning related measurements.

41. The testing or monitoring device for an artificial intelligence (AI) model for positioning according to claim 38 or 39, characterized in that: The apparatus further includes a third processing unit, configured to: If the test / monitoring result is not applicable, sending second indication information to the terminal, where the second indication information is used to instruct the terminal to perform a fourth operation; The fourth operation includes one of the following: restarting the current AI positioning related measurement, stopping the current AI positioning related measurement, and extending the current AI positioning related measurement.

42. A terminal, characterized in that: It includes a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the testing or monitoring method of the artificial intelligence AI model for positioning are implemented as described in any one of claims 1 to 16.

43. A network side device, characterized in that: It includes a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the testing or monitoring method of the artificial intelligence AI model for positioning as described in any one of claims 17 to 30 are implemented.

44. A readable storage medium, characterized in that The readable storage medium stores a program or instruction, which, when executed by the processor, implements the testing or monitoring method for the artificial intelligence AI model for positioning as described in any one of claims 1 to 16, or implements the steps of the testing or monitoring method for the artificial intelligence AI model for positioning as described in any one of claims 17 to 30.

Images