Method for testing or monitoring artificial intelligence (AI) model for positioning, and terminal

Abstract

The present application relates to the technical field of communications, and discloses a method for testing or monitoring an artificial intelligence (AI) model for positioning, and a terminal. The method for testing or monitoring an AI model for positioning in embodiments of the present application comprises: a terminal acquires a reference value and an input value; and the terminal performs a first operation, wherein the first operation comprises one of the following: inputting the input value into the AI model for positioning locally deployed on the terminal, acquiring an output value of the AI model, using the output value as a test / monitoring value, and determining a test / monitoring result of the AI model on the basis of the reference value and the test / monitoring value; and using the input value as a test / monitoring value, and determining a test / monitoring result of the AI model on the basis of the reference value and the test / monitoring value.

Description

Test or monitoring method of artificial intelligence (AI) model for positioning and terminal

[0001] Cross-reference to Related Applications

[0002] The present application claims priority to the Chinese patent application No. 202410417067.9, filed on April 8, 2024, with the State Intellectual Property Office, and entitled "Test or monitoring method of artificial intelligence (AI) model for positioning and terminal", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present application belongs to the field of communication technology, and particularly relates to a test or monitoring method of an artificial intelligence (AI) model for positioning and a terminal. BACKGROUND

[0004] With the gradual maturity of artificial intelligence (AI) technology, there are more and more researches on AI / Machine Learning (ML) based air interface technology. For example, an AI / ML model can be used for positioning of a terminal. In order to meet the AI positioning requirements, before deployment of an AI / ML model for positioning, a related technology needs to complete consistency test in a laboratory. However, for a device that has been deployed in a field and has an AI function, the AI / ML model in the device can become unsuitable for actual positioning requirements. For example, the AI / ML model in the device is updated, the field environment changes, etc. At this time, it is not known whether the AI / ML model in the device can meet the consistency test requirements. SUMMARY

[0005] Embodiments of the present application provide a test or monitoring method of an artificial intelligence (AI) model for positioning and a terminal, which can test or monitor a deployed AI model for positioning.

[0006] In a first aspect, a test or monitoring method of an artificial intelligence (AI) model for positioning is provided, which is executed by a terminal, and the method comprises:

[0007] The terminal acquires a reference value and an input value;

[0008] The terminal performs a first operation, and the first operation comprises one of the following:

[0009] The input value is input into an AI model for positioning deployed locally by the terminal, an output value of the AI model is acquired, the output value is taken as a test / monitoring value, and a test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value;

[0010] determine a test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the input value.

[0011] In a second aspect, a method for testing or monitoring an artificial intelligence (AI) model for positioning is provided, performed by a network-side device, and includes:

[0012] The network-side device obtains a reference value and an input value.

[0013] The network-side device performs a third operation, the third operation including one of:

[0014] The network-side device sends the reference value and the input value to a terminal, the reference value and the input value being used to determine a test / monitoring result of an AI model for positioning locally deployed by the terminal.

[0015] The network-side device sends the input value to a terminal and receives an output value from the terminal, the output value being obtained by inputting the input value to an AI model for positioning locally deployed by the terminal, determines a test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the output value.

[0016] The network-side device inputs the input value to an AI model for positioning locally deployed by the network-side device, obtains an output value of the AI model, and determines a test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the output value.

[0017] The network-side device determines a test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the input value.

[0018] In a third aspect, a device for testing or monitoring an artificial intelligence (AI) model for positioning is provided, and includes:

[0019] A first obtaining unit is configured to obtain a reference value and an input value.

[0020] A first execution unit is configured to perform a first operation, the first operation including one of:

[0021] The input value is input to an AI model for positioning locally deployed by the terminal, an output value of the AI model is obtained, the output value is taken as a test / monitoring value, and a test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value.

[0022] The input value is taken as a test / monitoring value, and a test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value.

[0023] In a fourth aspect, a test or monitoring apparatus for an artificial intelligence (AI) model for positioning is provided, comprising:

[0024] a second obtaining unit configured to obtain a reference value and an input value;

[0025] a second performing unit configured to perform a third operation, the third operation comprising one of:

[0026] sending the reference value and the input value to a terminal, the reference value and the input value being used to determine a test / monitoring result of an AI model for positioning deployed locally by the terminal;

[0027] sending the input value to a terminal and receiving an output value from the terminal, the output value being obtained by inputting the input value into an AI model for positioning deployed locally by the terminal, determining a test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the output value;

[0028] inputting the input value into an AI model for positioning deployed locally by the network-side device, obtaining an output value of the AI model, and determining a test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the output value;

[0029] determining a test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the input value.

[0030] In a fifth aspect, a terminal is provided, comprising a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions being executed by the processor to implement the steps of the method according to the first aspect.

[0031] In a sixth aspect, a terminal is provided, comprising a processor and a communication interface, wherein the processor is configured to obtain a reference value and an input value, and perform a first operation, the first operation comprising one of: inputting the input value into an AI model for positioning deployed locally by the terminal, obtaining an output value of the AI model, and taking the output value as a test / monitoring value, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value; and taking the input value as a test / monitoring value, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value.

[0032] In a seventh aspect, a network-side device is provided, comprising a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions being executed by the processor to implement the steps of the method according to the second aspect.

[0033] In an eighth aspect, a network-side device is provided, comprising a processor and a communication interface, wherein the processor is configured to obtain a reference value and an input value; and perform a third operation, the third operation comprising one of the following: sending the reference value and the input value to a terminal, the reference value and the input value being used to determine a test / monitoring result of an AI model for positioning locally deployed by the terminal; sending the input value to the terminal and receiving an output value from the terminal, the output value being obtained by inputting the input value into the AI model for positioning locally deployed by the terminal, determining the test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the output value; inputting the input value into an AI model for positioning locally deployed by the network-side device, obtaining an output value of the AI model, determining the test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the output value; and determining the test / monitoring result of the AI model based on the reference value and a test / monitoring value, the test / monitoring value being the input value.

[0034] In a ninth aspect, a readable storage medium is provided, the readable storage medium storing a program or instructions, the program or instructions being executed by a processor to implement the steps of the method according to the first aspect, or to implement the steps of the method according to the second aspect.

[0035] In a tenth aspect, a wireless communication system is provided, comprising a terminal and a network-side device, the terminal being configured to implement the steps of the method according to the first aspect, and the network-side device being configured to implement the steps of the method according to the second aspect.

[0036] In an eleventh aspect, a chip is provided, the chip comprising a processor and a communication interface, the communication interface being coupled to the processor, and the processor being configured to run a program or instructions to implement the method according to the first aspect, or to implement the method according to the second aspect.

[0037] In a twelfth aspect, a computer program / program product is provided, the computer program / program product being stored in a storage medium, and the computer program / program product being executed by at least one processor to implement the steps of the method for testing or monitoring an artificial intelligence (AI) model for positioning according to the first aspect, or to implement the steps of the method for testing or monitoring an artificial intelligence (AI) model for positioning according to the second aspect.

[0038] In the embodiment of the present application, by obtaining the reference value and the input value, inputting the input value into the AI model for positioning locally deployed by the terminal, obtaining the output value of the AI model, taking the output value as the test / monitoring value, or taking the input value as the test / monitoring value, determining the test / monitoring result of the AI model based on the reference value and the test / monitoring value, the verification or monitoring of the AI model for positioning is realized, thereby improving the reliability of the AI model for positioning deployed in the actual environment, and the performance of AI positioning can be effectively guaranteed. BRIEF DESCRIPTION OF DRAWINGS

[0039] FIG. 1 is a block diagram of a wireless communication system to which embodiments of the present application can be applied;

[0040] FIG. 2 is a flowchart of a method for testing or monitoring an artificial intelligence (AI) model for positioning according to an embodiment of the present application;

[0041] FIG. 3 is a flowchart of a method for testing or monitoring an artificial intelligence (AI) model for positioning according to another embodiment of the present application;

[0042] FIG. 4 is a block diagram of a device for testing or monitoring an artificial intelligence (AI) model for positioning according to an embodiment of the present application;

[0043] FIG. 5 is a block diagram of a device for testing or monitoring an artificial intelligence (AI) model for positioning according to another embodiment of the present application;

[0044] FIG. 6 is a block diagram of a communication device according to an embodiment of the present application;

[0045] FIG. 7 is a block diagram of a terminal according to an embodiment of the present application;

[0046] FIG. 8 is a block diagram of a network-side device according to an embodiment of the present application;

[0047] FIG. 9 is a block diagram of a network-side device according to another embodiment of the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be described clearly below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0049] The terms "first", "second", and the like in the specification and claims of this application are used as identifiers for convenience and are not intended to convey an importance or a chronological sequence among or between the identified objects. It is to be understood that the terms so used are interchangeable under appropriate circumstances and that the embodiments of this application are capable of functioning in other sequences than those described herein. The terms "first", "second", and the like are used to distinguish between similar objects, and are not used to describe a particular order or sequence. It is to be understood that such terms so used are interchangeable under appropriate circumstances and that the embodiments of this application are capable of functioning in other sequences than those described herein, and that the terms "first", "second", and the like are used to distinguish between similar objects, and are not used to describe a particular order or sequence, for example, a first object can be one or more than one, and a second object can be one or more than one. Furthermore, "or" is used herein to mean, and is used interchangeably with, the inclusive or. As used herein, "or" means at least one of the items, for example, A or B means any of the following: A alone, B alone, or A and B together. In addition, characters " / " are generally used to represent an "or" relationship between the associated objects.

[0050] The term "indicate" in this application can be a direct indication (or explicit indication) or an indirect indication (or implicit indication). The direct indication can be understood as that the sender explicitly informs the receiver of specific information, operations to be performed or requested results, etc. in the sent indication. The indirect indication can be understood as that the receiver determines the corresponding information according to the indication sent by the sender, or judges and determines the operations to be performed or the requested results according to the judgment result.

[0051] It is worth noting that the techniques described in the embodiments of this application are not limited to Long Term Evolution (LTE) / LTE-Advanced (LTE-A) systems, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA) or other systems. The terms "system" and "network" in the embodiments of this application are often used interchangeably, and the described techniques can be used in the above mentioned systems and radio technologies, as well as in other systems and radio technologies. The following description describes a New Radio (NR) system for example purposes, and NR terminology is used in most of the following description, but these techniques can also be applied to systems other than NR systems, such as 6 thA communication system (5G, 6G) communication system.

[0052] FIG. 1 shows a block diagram of a wireless communication system to which embodiments of the present application can be applied. The wireless communication system includes a terminal 11 and a network-side device 12. The terminal 11 can be a terminal-side device such as a mobile phone, a Tablet Personal Computer, a Laptop Computer, a notebook computer, a Personal Digital Assistant (PDA), a palmtop computer, a netbook, an Ultra-mobile Personal Computer (UMPC), a Mobile Internet Device (MID), an Augmented Reality (AR) device, a Virtual Reality (VR) device, a robot, a wearable device, a flight vehicle, a Vehicle User Equipment (VUE), a shipboard device, a Pedestrian User Equipment (PUE), a smart home (a home device with a wireless communication function such as a refrigerator, a television, a washing machine, or furniture), a game console, a Personal Computer (PC), a kiosk, or a self-service machine. The wearable device includes a smart watch, a smart bracelet, a smart earphone, smart glasses, smart jewelry (a smart bracelet, a smart necklace, a smart ring, a smart necklace, a smart anklet, a smart necklace, etc.), a smart wristband, smart clothing, etc. The vehicle-mounted device can also be referred to as a vehicle-mounted terminal, a vehicle-mounted controller, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip, or a vehicle-mounted unit, etc. It should be noted that the specific type of the terminal 11 is not limited in the embodiments of the present application. The network-side device 12 can include an access network device or a core network device. The access network device can also be referred to as a Radio Access Network (RAN) device, a radio access network function, or a radio access network unit. The access network device can include a base station, a Wireless Local Area Network (WLAN) Access Point (AP), or a Wireless Fidelity (WiFi) node, etc.The base station can be referred to as a Node B (NB), an evolved Node B (eNB), a next generation Node B (gNB), a New Radio Node B (NR Node B), an access point, a relay station (RBS), a serving base station (SBS), a base transceiver station (BTS), a radio base station, a radio transceiver, a basic service set (BSS), an extended service set (ESS), a home Node B (HNB), a home evolved Node B, a transmission reception point (TRP), or some other suitable terminology in the art, and is not limited to a particular technical terminology, provided that the same technical effect is achieved. It should be noted that in the embodiments of the present application, only the base station in the NR system is taken as an example for introduction, and the specific type of the base station is not limited.

[0053] The core network device can include, but is not limited to, at least one of the following: a core network node, a core network function, a mobility management entity (MME), an access and mobility management function (AMF), a session management function (SMF), a user plane function (UPF), a policy control function (PCF), a policy and charging rules function (PCRF), an edge application server discovery function (EASDF), a unified data management (UDM), a unified data repository (UDR), a home subscriber server (HSS), a centralized network configuration (CNC), a network repository function (NRF), a network exposure function (NEF), a local NEF (L-NEF), a binding support function (BSF), an application function (AF), a location management function (LMF), and the like. It should be noted that only the core network device in the NR system is taken as an example for introduction in the embodiments of the present application, and the specific type of the core network device is not limited.

[0054] In some embodiments, "acquire", "obtain", "get", "receive", "transmit", "bidirectional transmission", "send and / or receive" can be replaced with each other, which can be interpreted as receiving from other subjects, obtaining from protocols, obtaining by oneself, and various meanings such as autonomous implementation.

[0055] In some embodiments, the terms "send", "transmit", "report", "issue", "transmit", "bidirectional transmission", "send and / or receive" can be replaced with each other.

[0056] In some embodiments, "predetermined" and "preset" can be interpreted as being previously stipulated in a protocol or the like, or as being previously set by the device or the like.

[0057] In some embodiments, the data, information, and the like can be acquired in compliance with the laws and regulations of the country where the location is located.

[0058] In some embodiments, the data, information, and the like can be acquired after obtaining the consent of the user.

[0059] Artificial intelligence (AI) is currently widely used in various fields. Integrating artificial intelligence into wireless communication networks significantly improves technical indicators such as throughput, latency, and user capacity, which is an important task for future wireless communication networks. AI modules have various implementation methods, such as neural networks, decision trees, support vector machines, and Bayesian classifiers. The present application takes neural networks as an example for illustration, but does not limit the specific type of AI module. Neural networks are composed of neurons, where a1, a2, … a K are inputs, w is a weight (multiplicative coefficient), b is a bias (additive coefficient), and σ(.) is an activation function. Common activation functions include Sigmoid, tanh, ReLU (Rectified Linear Unit, linear rectifier function, modified linear unit), and the like. The parameters of the neural network are optimized by a gradient optimization algorithm. Gradient optimization algorithms are a class of algorithms that minimize or maximize an objective function (sometimes also called a loss function), and the objective function is often a mathematical combination of model parameters and data. For example, given data X and its corresponding label Y, a neural network model f(.) is constructed. With the model, the predicted output f(x) can be obtained according to the input x, and the difference between the predicted value and the true value (f(x)-Y) can be calculated, which is the loss function. The purpose is to find appropriate W, b to minimize the value of the above loss function. The smaller the loss value, the closer the model is to the true situation.

[0060] The common optimization algorithm is basically based on error back propagation (BP) algorithm. The basic idea of BP algorithm is that the learning process consists of two processes of forward propagation of signals and backward propagation of errors. During forward propagation, the input sample is transmitted from the input layer to the output layer through the processing of each hidden layer. If the actual output of the output layer does not match the expected output, the backward propagation of error is entered. Error back propagation is to transmit the output error to the input layer through the hidden layer in a certain form, and distribute the error to all units of each layer, so as to obtain the error signal of each unit, which is used as the basis for correcting the weight of each unit. The weight adjustment process of each layer in the forward propagation of signals and the backward propagation of errors is repeated. The process of continuously adjusting the weight is the learning and training process of the network. This process continues until the error of the network output is reduced to an acceptable level, or the learning time reaches the preset learning time.

[0061] The common optimization algorithm includes gradient descent, stochastic gradient descent (SGD), mini-batch gradient descent, momentum method, Nesterov (the name of the inventor, specifically stochastic gradient descent with momentum), adaptive gradient descent (Adagrad), adaptive delta (Adadelta), root mean square prop (RMSprop), adaptive moment estimation (Adam), etc.

[0062] These optimization algorithms, when error back propagation, are based on the error / loss obtained from the loss function, and the derivative / partial derivative of the current neuron is added to the learning rate, the previous gradient / derivative / partial derivative, etc. to obtain the gradient, and the gradient is transmitted to the previous layer.

[0063] Life cycle management refers to the process of effectively managing and controlling the whole process of AI / ML model from creation to abandonment. This process includes the following main stages:

[0064] Requirement definition: determine the business requirements, and clarify the expected functions and effects of the model;

[0065] Data collection: the process of collecting data by network nodes, management entities or UEs, for AI / ML model training, data analysis and inference;

[0066] Model training: a process of training an AI / ML model in a data-driven manner (by learning input / output relationship) and obtaining a trained AI / ML model;

[0067] Function / model identification: a process / method of identifying an AI / ML function / model for common understanding between network-side devices and UEs, and information about the AI / ML function / model can be shared during function identification;

[0068] Model transfer / sending: transferring AI / ML models through an air interface in a manner that is opaque to third generation partnership project (3 rd Generation Partnership Project, 3GPP) signaling, whether parameters of a model structure known to a receiving end or a new model with parameters, delivery can contain a complete model or a partial model;

[0069] Model inference: a process of using a trained AI / ML model to generate a set of outputs according to a set of inputs;

[0070] Function / model selection: a process of selecting an AI / ML model for activation among multiple models for the same AI / ML-enabled function;

[0071] Function / model activation: enabling an AI / ML model for a specific AI / ML-enabled function;

[0072] Function / model deactivation: deactivating an AI / ML model for a specific AI / ML-deactivated function;

[0073] Function / model switching: deactivating a currently activated AI / ML model and activating a different AI / ML model for a specific AI / ML-enabled function;

[0074] Function / model rollback: deactivating a currently activated AI / ML model and reusing or activating a previous version of the model;

[0075] Function / model supervision: a process of monitoring AI / ML function / model inference performance;

[0076] Model upgrade: a process of upgrading model parameters and / or model structure.

[0077] The above stages constitute the model lifecycle management process, and effective management can ensure high quality, high efficiency and high reliability of AI / ML models, thereby meeting business needs and improving work efficiency.

[0078] The method of the embodiments of the present application relates to the monitoring stage after model deployment in lifecycle management.

[0079] The positioning techniques based on AI models include the following cases as shown in Table 1:

[0080] Direct AI / ML positioning:

[0081] Case 1 (first priority): UE-based positioning, UE-side model, direct AI / ML positioning;

[0082] Case 2b (second priority): UE-assisted / LMF positioning based on LMF-side model, direct AI / ML positioning;

[0083] Case 3b (first priority): NG-RAN node-assisted positioning with LMF-side model, direct AI / ML positioning;

[0084] Direct AI / ML positioning refers to that the model directly infers the position of the target positioning terminal, and the AI / ML model outputs the position information.

[0085] AI / ML assisted positioning:

[0086] Case 2a (second priority): UE-assisted / LMF-based positioning using UE-side model, AI / ML assisted positioning

[0087] Case 3a (first priority): NG-RAN node-assisted positioning with gNB-side model, AI / ML assisted positioning.

[0088] AI / ML assisted positioning refers to that the model infers the intermediate features of the target positioning terminal, and the AI / ML model outputs the intermediate features, wherein the position of the target terminal can be calculated according to the intermediate features.

[0089] Table 1: Types of positioning techniques based on AI models

[0090] To meet the AI positioning requirements, before the AI / ML model for positioning is deployed, the related technology needs to complete the consistency test in the laboratory, but for the device with AI function that has been deployed in the field, the AI / ML model in the device may become unsuitable for actual positioning requirements, for example, the AI / ML model in the device is updated, the field environment changes, etc., at this time it is not known whether the AI / ML model in the device can meet the consistency test requirements.

[0091] Optionally, consistency testing is an evaluation method for verifying the consistency and stability of the output of the model under different conditions. Consistency testing aims to check whether the model produces consistent output results under different conditions or environments for the same input data. When performing consistency testing, the following aspects are usually considered: input variation: by changing some characteristics or attributes of the input data, test whether the output of the model remains consistent; environmental variation: the model may be sensitive to environmental changes, so test whether the output of the model is consistent under different environmental conditions; data set segmentation: divide the data set into training set and test set, and test whether the output of the model is consistent on different subsets; model version comparison: if there are multiple versions of the model, they can be compared to ensure that they produce consistent output under the same input.

[0092] The method for testing or monitoring the artificial intelligence (AI) model for positioning provided by the embodiments of the present application will be described in detail below in combination with the accompanying drawings, some embodiments and application scenarios.

[0093] FIG. 2 is a flowchart of a method for testing or monitoring an artificial intelligence (AI) model for positioning provided by an embodiment of the present application. As shown in FIG. 2, the method provided by the embodiment includes:

[0094] Step 201: A terminal acquires a reference value and an input value.

[0095] It should be noted that the reference value in the embodiments of the present application, as a reference value for testing or monitoring the AI model, can be reference position information, reference intermediate quantity, reference output data set, or reference input data set.

[0096] The AI model described in the embodiments of the present application is an AI model for positioning, i.e., participating in AI positioning. The output of the AI model can be position information or an intermediate quantity. The definition of the intermediate quantity will be described below.

[0097] The AI model described in the embodiments of the present application can also be referred to as an AI unit, an AI model, a machine learning (ML) model, an ML unit, an AI structure, an AI function, an AI feature, a machine learning model, a neural network, a neural network function, a neural network function, etc., or the AI model can also refer to a processing unit capable of implementing a specific algorithm, formula, processing flow, capability, etc. related to AI, or the AI model can be a processing method, algorithm, function, module or unit for a specific data set, or the AI model can be a processing method, algorithm, function, module or unit running on AI / ML related hardware such as a graphics processing unit (GPU), a neural processing unit (NPU), a tensor processing unit (TPU), an application-specific integrated circuit (ASIC), etc. The embodiments of the present application do not make specific limitations. Optionally, the specific data set includes input and / or output of the AI model.

[0098] The input value can be understood as a model input for testing or monitoring the AI model, which can be measurement information or model input information processed from the measurement information. In the embodiments of the present application, the measurement information generally refers to signal / channel measurement information, and the measurement information can also be referred to as measurement quantity, measurement result, etc.

[0099] The terminal performs a first operation, and the first operation includes one of the following:

[0100] The input value is input into the AI model for positioning deployed locally by the terminal, an output value of the AI model is obtained, the output value is taken as a test / monitoring value, and a test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value;

[0101] The input value is taken as a test / monitoring value, and a test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value.

[0102] The test / monitoring value includes a test value or a monitoring value. The test / monitoring result includes a test result or a monitoring result.

[0103] It can be understood that after the terminal obtains the reference value and the input value, there are two ways to test / monitor the performance of the AI model, one way is to verify the inference ability of the AI model, and the other way is to verify whether the AI model can adapt to the current environment, specifically to verify whether the input of the model can adapt to the current environment.

[0104] The verifying the inference capability of the AI model includes: inputting the input value into the AI model for positioning deployed locally on the terminal, obtaining an output value of the AI model, taking the output value as a test / monitoring value, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value.

[0105] The verifying the input of the AI model includes: taking the input value as a test / monitoring value, the reference value belonging to a reference input data set, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value.

[0106] It should be noted that the test or monitoring method for the AI model for positioning provided in the embodiments of the present application is not only applicable to performance test or performance monitoring after model deployment, but also can be used for performance test or performance monitoring after model generation, and test before model deployment. For example, the test result evaluation method and the test index in the present application can be used for consistency test completed in a laboratory.

[0107] The test or monitoring method for the AI model for positioning provided in the embodiments of the present application realizes the verification or monitoring of the AI model for positioning by obtaining the reference value and the input value, inputting the input value into the AI model for positioning deployed locally on the terminal, obtaining an output value of the AI model, taking the output value as a test / monitoring value, or taking the input value as a test / monitoring value, determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value, thereby improving the reliability of the AI model for positioning in actual environment, and effectively guaranteeing the performance of AI positioning.

[0108] It should be noted that each embodiment of the present application can be freely combined, the order can be changed or executed alone, and does not need to rely on or depend on a fixed execution order.

[0109] In some embodiments, the reference value and the input value can be one of the following a) to d):

[0110] a) the reference value is reference position information, and the input value is measurement information or model input information obtained by processing the measurement information;

[0111] It can be understood that if the output of the AI model is position information, the terminal obtains reference position information and measurement information or model input information obtained by processing the measurement information.

[0112] The reference position information refers to position information as a reference output. The position information can be rectangular coordinates, polar coordinates, latitude and longitude information, relative position, absolute position, etc., which are not limited in the present application.

[0113] The measurement information generally refers to signal / channel measurement information, and can be replaced by the concepts of measurement quantity, measurement result, etc.

[0114] In the embodiments of the present application, the measurement information includes at least one of the following: power, time delay, phase, channel impulse response (CIR), power delay profile (PDP), delay profile (DP).

[0115] Optionally, the power includes received power, received power of a target path in multipath, signal-to-noise ratio, etc. The time delay includes received time delay, sending time delay, round-trip time delay, etc. The phase includes phase offset, phase delay, phase difference, etc. The channel impulse response includes impulse response, frequency response, etc.

[0116] The signal can pass through multiple different paths during transmission to reach the receiving end, and each path corresponds to different propagation delays. The power delay profile describes the distribution of signal power at different delay times. The delay profile reflects the distribution of arrival time delays of the received signal on different propagation paths, and is usually a discrete value spectrum in actual measurement or testing.

[0117] Optionally, processing the measurement information refers to mathematical processing such as normalization and averaging of the measurement information.

[0118] In some embodiments, the terminal inputs the input value to the AI model locally deployed on the terminal, obtains the output value of the AI model, the output value of the AI model being the position information inferred by the AI model according to the input value, and compares the output value with the reference position information as the test / monitoring value, to determine the test / monitoring result of the AI model.

[0119] In some embodiments, the terminal obtains the input value, inputs the input value to the AI model locally deployed on the terminal, obtains the output value of the AI model, and sends the output value to the network side device, and the network side device compares the output value with the reference position information as the test / monitoring value, to determine the test / monitoring result of the AI model.

[0120] When the AI model is deployed on the network side device, the network side device determines the test / monitoring result of the AI model according to the reference position information and the output value inferred by inputting the input value to the AI model.

[0121] In some embodiments, the reference position information and the input value are obtained in the following manner:

[0122] The terminal receives first information from the network side device, the first information including reference position information and an input value, the reference position information being position information of a first communication device selected by the network side device as a position known device, and the input value being measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

[0123] In the embodiments of the present application, the first communication device is a position known communication device, that is, the position of the first communication device is known to the network side device. The first communication device includes a positioning reference unit (PRU) or other communication devices supporting channel / signal measurement.

[0124] Optionally, the network side device selects a target PRU as the first communication device from a plurality of PRUs around the terminal, the target PRU reports its position information and measurement information to the network side device, and the network side device sends the position information of the target PRU as the reference position information and the measurement information of the target PRU as the input value to the terminal.

[0125] The method provided in the embodiments of the present application is suitable for the case where the output of the AI model is position information, the input value is input into the AI model for positioning locally deployed in the terminal by obtaining the reference position information and the input value, the output value of the AI model is obtained, the output value is used as the test / monitoring value, the test / monitoring result of the AI model is determined based on the reference position information and the test / monitoring value, the inference ability of the AI model for positioning is verified or monitored, and thus the reliability of the AI model for positioning deployed in the actual environment is improved, and the performance of AI positioning can be effectively guaranteed.

[0126] b) the reference value is a reference intermediate quantity, the reference intermediate quantity being intermediate information used for determining the position of the terminal, the reference intermediate quantity being of the same type as the output value of the AI model, and the input value being measurement information or model input information obtained by processing the measurement information;

[0127] It can be understood that if the output of the AI model is an intermediate quantity, the reference intermediate quantity can be obtained, and the measurement information or the model input information obtained by processing the measurement information can be used as the input value to test or monitor the AI model.

[0128] The reference intermediate quantity is of the same type as the output value of the AI model. For example, if the type of the output value of the AI model is RTOA, the type of the reference intermediate quantity should also be RTOA.

[0129] The reference intermediate quantity can also be referred to as a reference intermediate feature, and is intermediate information that can determine the terminal position.

[0130] In some embodiments, the terminal inputs, as input values, measurement information or model input information obtained by processing the measurement information into an AI model locally deployed by the terminal, obtains an output value of the AI model, and determines a test / monitoring result of the AI model based on the output value and the reference intermediate quantity.

[0131] Optionally, the terminal obtains an input value, the terminal inputs the input value into an AI model locally deployed by the terminal, obtains an output value of the AI model, and sends the output value to a network side device, and the network side device can implement test or monitoring of the AI model according to the output value and the reference intermediate quantity.

[0132] In some embodiments, the reference intermediate quantity includes at least one of:

[0133] a line of sight or non-line of sight (LOS / NLOS) indicator;

[0134] a reference signal time difference (RSTD);

[0135] a relative time of arrival (RTOA);

[0136] a receive-transmit (Rx-Tx) time difference;

[0137] a time of arrival (TOA);

[0138] an angle of arrival (AoA);

[0139] an angle of departure (AoD);

[0140] a reference signal received power (RSRP);

[0141] a reference signal received path power (RSRPP).

[0142] In some embodiments, the reference intermediate quantity and the input value are obtained in the following manner:

[0143] receive a reference intermediate quantity and an input value from the network side device;

[0144] The reference intermediate quantity is obtained based on reference position information reported by a first communication device with known position and second information, and the second information is used to assist in determining the reference intermediate quantity, or the reference intermediate quantity is determined by the first communication device and reported to the network side device;

[0145] The input value is measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

[0146] It can be understood that the terminal receives the reference intermediate quantity and the input value sent by the network side device, and the present application embodiment provides two determination modes of the reference intermediate quantity:

[0147] 1) Direct determination mode, determined by the first communication device;

[0148] 2) Indirect determination mode, that is, determined by the network side device based on the reference position information of the first communication device and second information, wherein the second information is used to assist in determining the reference intermediate quantity.

[0149] Optionally, the network side device receives the reference intermediate quantity and the measurement information reported by the first communication device, and sends the reference intermediate quantity and the measurement information to the terminal, wherein the reference intermediate quantity is determined by the first communication device.

[0150] Optionally, the network side device receives the reference position information and the measurement information reported by the first communication device, determines the reference intermediate quantity based on the reference position information and the second information, and sends the reference intermediate quantity and the measurement information to the terminal.

[0151] In the present application embodiment, the first communication device is a communication device with known position, that is, the position of the first communication device is known to the network side device. Optionally, the first communication device is selected by the network side device, and the first communication device includes a positioning reference unit (PRU) or other communication device supporting channel / signal measurement.

[0152] Optionally, the network side device selects a target PRU as the first communication device from a plurality of PRUs around the terminal, the target PRU reports its position information and measurement information to the network side device, the network side device takes the position information of the target PRU as the reference position information, determines the reference intermediate quantity based on the reference position information and the second information, and takes the measurement information of the PRU as the input value and sends it to the terminal.

[0153] In some embodiments, the second information comprises at least one of: a location of a transmission and reception point (TRP) currently participating in AI positioning, historical data of measurement information, historical data of AI model output values, a historical location of the terminal, a historical location of the TRP, and a historical location of the first communication device.

[0154] For example, the second information is a location of a transmission and reception point (TRP) currently participating in AI positioning. Based on the location information of the target PRU and the location of the TRP currently participating in AI positioning, the network side device calculates intermediate quantities when the target PRU performs positioning using the TRPs currently participating in AI positioning by using geometric relationships in mathematics. These intermediate quantities are the reference intermediate quantities. The target PRU measures downlink positioning reference signals (DL-PRS) transmitted by the TRPs currently participating in AI positioning to obtain measurement information and sends the measurement information to the network side device. Alternatively, the network side device sends the reference intermediate quantities and the measurement information to the terminal. The terminal takes the reference intermediate quantities as reference values and inputs the measurement information into the AI model to obtain intermediate quantities output by the AI model. The terminal takes the intermediate quantities output by the AI model as test / monitoring values and compares them with the reference intermediate quantities to determine the test / monitoring result of the AI model. Alternatively, the network side device sends the measurement information to the terminal. The terminal inputs the measurement information into the AI model to obtain intermediate quantities output by the AI model and sends the intermediate quantities output by the AI model as test / monitoring values to the network side device. The network side device compares the intermediate quantities output by the AI model with the reference intermediate quantities to determine the test / monitoring result of the AI model.

[0155] The second information can also be other auxiliary data, including historical data of measurement information, historical data of AI model output values, a historical location of the terminal, a historical location of the TRP, and / or a historical location of the first communication device.

[0156] The method provided by the embodiments of the present application is suitable for the case where the output of the AI model is intermediate quantities. By obtaining reference intermediate quantities and input values, inputting the input values into the AI model for positioning deployed locally in the terminal, obtaining intermediate quantities output by the AI model, and determining the test / monitoring result of the AI model based on the reference intermediate quantities and the intermediate quantities output by the AI model, the inference ability of the AI model for positioning is verified or monitored, thereby improving the reliability of the AI model for positioning deployed in an actual environment and effectively ensuring the performance of AI positioning.

[0157] c) the reference value belongs to a reference output data set, and the input value belongs to a reference input data set;

[0158] It can be understood that in the embodiment, the AI model can execute the AI function according to the reference input data set, obtain the corresponding model output, take the model output as the test / monitoring value, and based on the reference output data set and the test / monitoring value, test or monitor the AI model.

[0159] The reference output data set includes reference values of the same type as the output values of the AI model.

[0160] The reference output data set or the reference input data set is autonomously generated by the terminal, autonomously generated by the network side device, generated by the first device, or predefined by a protocol.

[0161] The first device can be a server or other device other than the terminal and the network side device.

[0162] Optionally, one AI model can correspond to one or more groups of reference data sets, and one group of reference data sets can correspond to one or more AI models.

[0163] Optionally, in the case where the AI model is deployed on the terminal side, the terminal receives the reference data set from the network side device or the first device, the reference data set includes the parameter input data set and the reference output data set, the terminal takes the reference input data set as the model input, takes the model output at this time as the test / monitoring value, and compares it with the reference output data set to determine the test / monitoring result of the AI model.

[0164] Optionally, in the case where the AI model is deployed on the terminal side, the terminal generates or predefines the reference input data set and the reference output data set according to the terminal, the terminal takes the reference input data set as the model input, takes the model output at this time as the test / monitoring value, and compares it with the reference output data set to determine the test / monitoring result of the AI model.

[0165] Optionally, in the case where the AI model is deployed on the terminal side, the terminal receives the reference input data set from the network side device, the terminal takes the reference input data set as the model input, takes the model output at this time as the test / monitoring value, and reports it to the network side device, the network side device compares the test / monitoring value with the reference output data set to determine the test / monitoring result of the AI model.

[0166] The method provided by the embodiments of the present application is applicable to the case that the AI model can perform an AI function according to a reference data set. The reference input data set is input into the AI model for positioning deployed locally in the terminal, the output value of the AI model is obtained, and the test / monitoring result of the AI model is determined based on the reference output data set and the output value of the AI model, so as to verify or monitor the inference ability of the AI model for positioning, thereby improving the reliability of the AI model for positioning deployed in the actual environment and effectively ensuring the performance of AI positioning.

[0167] d) The reference value belongs to the reference input data set, and the input value is measurement information or model input information obtained by processing the measurement information.

[0168] In order to verify whether the AI model can adapt to the current environment, the embodiments of the present application propose to compare the measurement information or model input information obtained by processing the measurement information as model input with the reference input data set, so as to test / monitor the input of the AI model.

[0169] The terminal can obtain the measurement information by actual measurement, for example, by measuring the downlink positioning reference signal transmitted by the TRP of the current reference AI positioning.

[0170] Optionally, in the case that the AI model is deployed on the terminal side, the terminal takes the measurement information or model input information obtained by processing the measurement information as test / monitoring value, and determines the test / monitoring result of the AI model based on the reference input data set and the test / monitoring value.

[0171] Optionally, in the case that the AI model is deployed on the terminal side, the terminal can also report the measurement quantity as model input to the network side device, and the network side device takes the measurement quantity as model input reported by the terminal as test / monitoring value, and determines the test / monitoring result of the AI model based on the reference input data set and the test / monitoring value. Here, the measurement quantity as model input includes the measurement information or model input information obtained by processing the measurement information.

[0172] In some embodiments, the reference input data set is generated by the terminal autonomously, or is generated by the network side device autonomously, or is generated by a first device, or is predefined by a protocol.

[0173] The first device can be a server, or a device other than the terminal and the network side device.

[0174] The method provided in the embodiments of the present application is applicable to a case where the AI model can perform an AI function according to a reference data set, and by obtaining a reference input data set and an input value, a test / monitoring result of the AI model is determined, thereby verifying or monitoring whether the input of the AI model for positioning can adapt to the current environment, improving the reliability of the AI model for positioning in the actual environment, and effectively ensuring the performance of AI positioning.

[0175] In some embodiments, the test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value, including:

[0176] The difference between the reference value and the test / monitoring value of the first number of paths or samples is calculated.

[0177] The test / monitoring result of the AI model is determined based on the difference between the reference value and the test / monitoring value of the first number of paths or samples.

[0178] The first number is greater than or equal to 1.

[0179] In the embodiments of the present application, the AI model can be tested / monitored once or multiple times.

[0180] If the number of times of testing / monitoring the AI model is 1, the test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value, including:

[0181] The difference between the reference value and the test / monitoring value is calculated, and compared with a predefined accuracy requirement. If the predefined accuracy requirement is met, it is determined that the AI model is applicable, and if the predefined accuracy requirement is not met, it is determined that the AI model is not applicable.

[0182] In order to improve the accuracy of testing / monitoring, the AI model can also be tested / monitored multiple times.

[0183] Optionally, when the number of times of execution is greater than 1, the average value of the difference between the test / monitoring value and the reference value of each execution is compared with the predefined accuracy, or the ratio of the number of times of evaluating the model as applicable to the total evaluation times needs to be greater than a first threshold value (for example, a certain percentage, for example, 90%), or the number of times of evaluating the model as applicable is greater than a predefined evaluation number (for example, 4), at this time, the test / monitoring result of the AI model is applicable, otherwise, it is not applicable.

[0184] In each execution of testing / monitoring, the difference between the reference value and the test / monitoring value of the first number of paths or samples can be calculated, and the test / monitoring result of the AI model is determined based on the difference between the reference value and the test / monitoring value of the first number of paths or samples, thereby improving the accuracy of testing / monitoring the AI model.

[0185] The method provided by the embodiments of the present application can improve the accuracy of testing / monitoring the AI model by calculating the difference between the reference values and the test / monitoring values of the first quantity of paths or samples, and determining the test / monitoring result of the AI model based on the difference between the reference values and the test / monitoring values of the first quantity of paths or samples.

[0186] In some embodiments, the calculating the difference between the reference values and the test / monitoring values of the first quantity of paths or samples comprises at least one of the following:

[0187] calculating the Euclidean distance between the vector composed of the reference values of the first quantity of paths or samples and the vector composed of the test / monitoring values;

[0188] calculating the cosine similarity between the vector composed of the reference values of the first quantity of paths or samples and the vector composed of the test / monitoring values;

[0189] calculating the root mean square error between the vector composed of the reference values of the first quantity of paths or samples and the vector composed of the test / monitoring values;

[0190] calculating the difference between the reference values and the test / monitoring values of the first quantity of paths or samples;

[0191] calculating the average of the absolute values of the difference between the reference values and the test / monitoring values of the first quantity of paths or samples;

[0192] calculating the root mean square error of the difference between the reference values and the test / monitoring values of the first quantity of paths or samples;

[0193] calculating the straight line distance between the reference values and the test / monitoring values of the first quantity of paths or samples;

[0194] calculating the average of the straight line distance between the reference values and the test / monitoring values of the first quantity of paths or samples;

[0195] calculating the root mean square of the straight line distance between the reference values and the test / monitoring values of the first quantity of paths or samples.

[0196] The method provided by the embodiments of the present application can calculate the difference between the reference values and the test / monitoring values of the first quantity of paths or samples in the form of vectors, for example, calculating the Euclidean distance, cosine similarity, root mean square error, etc. between the vector composed of the reference values of the first quantity of paths or samples and the vector composed of the test / monitoring values.

[0197] Alternatively, the difference between the reference values and the test / monitoring values of the first quantity of paths or samples can also be calculated in the form of a single value, for example, calculating the difference between the reference values and the test / monitoring values of the first quantity of paths or samples, the average of the absolute values of the difference, or the root mean square error of the difference.

[0198] Optionally, the difference between the first quantity path or the sample reference value and the test / monitoring value is calculated, which can also be calculated in the form of linear distance, for example, the linear distance between the first quantity path or the sample reference value and the test / monitoring value, the average of the linear distance, or the root mean square of the linear distance.

[0199] The method provided by the embodiments of the application realizes flexibility of difference calculation.

[0200] In some embodiments, the determination of the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first quantity path or sample comprises:

[0201] In the case where the difference between the reference value and the test / monitoring value of the first quantity path or sample meets the predefined accuracy requirement, the test / monitoring result of the AI model is determined to be applicable.

[0202] In the case where the difference between the reference value and the test / monitoring value of the first quantity path or sample does not meet the predefined accuracy requirement, the test / monitoring result of the AI model is determined to be inapplicable.

[0203] It can be understood that the difference between the reference value and the test / monitoring value of the first quantity path or sample meeting the predefined accuracy requirement indicates that the inference ability of the current AI model can meet the current positioning environment, or that the current AI model can adapt to the current positioning environment, that is, it can be determined that the current AI model is applicable.

[0204] Optionally, the difference between the reference value and the test / monitoring value of the first quantity path or sample meeting the predefined accuracy requirement comprises:

[0205] In the case where the number of executions is equal to or greater than 1, the average of the difference between the test / monitoring value and the reference value of each execution is compared with the predefined accuracy, or the ratio of the number of times of single evaluation of the model as applicable to the total evaluation times is greater than a first threshold value, or the number of times of single evaluation of the model as applicable is greater than a predefined evaluation quantity, and then the test / monitoring result of the AI model is determined to be applicable.

[0206] The difference between the reference value and the test / monitoring value of the first quantity path or sample not meeting the predefined accuracy requirement indicates that the inference ability of the current AI model cannot meet the current positioning environment, or that the current AI model cannot adapt to the current positioning environment, that is, it can be determined that the current AI model is inapplicable.

[0207] In some embodiments, the predefined accuracy requirement is applicable to a specified condition, and it can be understood that the predefined accuracy requirement is related to the specified condition and needs to be applicable under the specified condition, and the predefined accuracy requirement can be different under different specified conditions.

[0208] Optionally, the specified condition includes at least one of the following: signal-to-interference noise ratio, signal-to-noise ratio, channel strength, noise strength, channel model, reference signal bandwidth, reference signal configuration, subcarrier spacing, carrier frequency, frequency band information, duplex mode, TRP configuration, UE configuration, AI model structure, AI model complexity, and AI quantization information.

[0209] It can be understood that the predefined accuracy requirement can be determined according to at least one of the following: signal-to-noise ratio, channel strength, noise strength, channel model, reference signal bandwidth, reference signal configuration, subcarrier spacing, carrier frequency, frequency band information, duplex mode, TRP configuration, UE configuration, AI model structure, AI model complexity, and AI quantization information.

[0210] The method provided by the embodiments of the present application defines an evaluation criterion for determining whether the AI model is applicable. The evaluation criterion can also be applicable to the evaluation or testing of the AI model before deployment. In the test, the network side device is replaced by a test equipment (TE).

[0211] The following further illustrates how to determine the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first number of paths or samples.

[0212] Example 1: The output of the AI model is a position

[0213] When the output of the AI model is a position, and the first information of the PRU available around the UE is known by the NW (LMF), the evaluation criterion adopted is the straight-line distance between the two, and the position expression is not limited to rectangular coordinates, polar coordinates, relative position, absolute position, etc.

[0214] For example, when the position output by the AI model is (x1, y1), and the corresponding reference position is (x0, y0);

[0215] 1) When the AI model is deployed at the UE side: the LMF sends the reference position (x0, y0) to the UE, and the UE takes the reference position (x0, y0) and the position (x1, y1) inferred by the UE's own model as the reference value and the test / monitoring value respectively, and tests the applicability of the current model deployment according to the pre-defined evaluation criteria; or, the UE sends the position (x1, y1) inferred by the UE's own model to the LMF, and the LMF side takes the reference position (x0, y0) and the position (x1, y1) sent by the UE's own model as the reference value and the test / monitoring value respectively, and tests the applicability of the current model according to the pre-defined evaluation criteria;

[0216] 2) When the AI model is deployed at the LMF side: the LMF side takes the selected reference position (x0, y0) and the position (x1, y1) inferred by the model as the reference value and the test / monitoring value respectively, and tests the applicability of the current model according to the pre-defined evaluation criteria;

[0217] The number of evaluations required by the pre-defined evaluation criteria is N. The criterion for a single evaluation is: d1 = √((x0-x1)2+(y0-y1)2).

[0218] If N = 1, compare d1 with the pre-defined accuracy d0, and if d1 is less than or less than or equal to d0, the model is applicable;

[0219] If N is greater than 1, perform the above process multiple times to obtain (d1, d2, …, dN), and compare Σdn / N with d0, if it is less than or less than or equal to d0, the model is applicable; or, d1, d2, …, dN are compared with d0 respectively, when the number of single model applications N1 (N1 ≤ N) is greater than or greater than or equal to the pre-defined N0 (N0 ≤ N), the model is applicable; or, when P1 = N1 / N is greater than or greater than or equal to the pre-defined P0, the model is applicable.

[0220] Example 2, the output of the AI model is an intermediate quantity, and the intermediate quantity is RSTD

[0221] When the output of the AI model is an intermediate quantity, the NW (LMF) infers and calculates the reference value of the intermediate quantity through the position of the TRP and the first information (including the position information and measurement information of the PRU whose position is known), for example, when the intermediate quantity is RSTD, the model output is RSTD1, and the corresponding reference value is RSTD0;

[0222] 1) When the AI model is deployed at the UE side: the UE sends the model output RSTD1 to the LMF, the LMF tests the applicability of the current model deployment according to the pre-defined evaluation criteria, according to RSTD1 and the corresponding RSTD0 as test / monitoring value and reference value respectively, or the UE receives RSTD0 sent by the LMF, the UE infers RSTD1 according to the measurement information, RSTD1 and RSTD0 as test / monitoring value and reference value respectively, the UE tests the applicability of the current model deployment according to the pre-defined evaluation criteria;

[0223] 2) When the AI model is deployed at the gNB or LMF side: the LMF side tests the applicability of the current model deployment according to the pre-defined evaluation criteria, according to the output RSTD1 and the corresponding RSTD0 as test / monitoring and reference value respectively.

[0224] The number of evaluations required by the pre-defined evaluation criteria is N. The criterion for a single evaluation is ΔRSTD1 = abs(RSTD1-RSTD0).

[0225] When N = 1, ΔRSTD1 is compared with the pre-defined accuracy ΔRSTD0, and ΔRSTD1 is less than or equal to ΔRSTD0, then the model is applicable.

[0226] When N is greater than 1, the above process is performed multiple times to obtain (ΔRSTD1, ΔRSTD2, …, ΔRSTDN), and ΣΔRSTDn / N is compared with ΔRSTD0, if less than or equal to ΔRSTD0, then the model is applicable; or ΔRSTD1, ΔRSTD2, …, ΔRSTDN are compared with ΔRSTD0 respectively, when the single model applicability times N1 (N1 ≤ N) is greater than or equal to the pre-defined N0 (N0 ≤ N), the model is applicable; or when P1 = N1 / N is greater than or equal to the pre-defined P0, the model is applicable;

[0227] The above RSTD can be replaced by TOA, Rx-Tx time difference, RTOA, angle measurement quantity, or power measurement quantity.

[0228] In some embodiments, in the case where the reference value is a reference intermediate quantity and the reference intermediate quantity is a line of sight / non-line of sight indicator (LOS / NLOS indicator), the first probability under the specified condition is taken as the reference accuracy;

[0229] The difference between the reference value and the test / monitoring value of the first number of paths or samples meets the pre-defined accuracy requirement, which includes:

[0230] Calculate the second probability that the reference value and the test / monitoring value under the first number of paths or samples are different;

[0231] In the case that the first probability represents the maximum mismatch probability of the LOS / NLOS indicator, and the second probability is less than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement.

[0232] Alternatively,

[0233] A third probability that the reference value and the test / monitoring value under the first number of paths or samples are the same is calculated.

[0234] In the case that the first probability represents the minimum matching probability of the LOS / NLOS indicator, and the third probability is greater than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement.

[0235] It should be noted that the LOS / NLOS indicator can be an indication domain indicating LOS or NLOS through different values. Alternatively, the value of the LOS / NLOS indicator is 1 when indicating LOS, and 0 when indicating NLOS. Alternatively, the value of the LOS / NLOS indicator is a hard value.

[0236] The result (1 or 0) of the AI model inference is taken as the test / monitoring value, and the reference value LOS / NLOS indicator (1 or 0) is compared to calculate the probability that the two are the same or different under the first number of paths or samples, compared with the first probability.

[0237] If the first probability represents the maximum mismatch probability of the LOS / NLOS indicator, the probability that the reference value and the test / monitoring value under the first number of paths or samples are different is calculated. If the probability that the two are different under the first number of paths or samples is less than the first probability, or less than or equal to the first probability, it indicates that the difference between the reference value and the test / monitoring value under the first number of paths or samples meets the predefined accuracy requirement, that is, the measurement / monitoring result of the current AI model is applicable.

[0238] If the first probability represents the minimum matching probability of the LOS / NLOS indicator, the probability that the reference value and the test / monitoring value under the first number of paths or samples are the same is calculated. If the probability that the two are the same under the first number of paths or samples is greater than the first probability, or greater than or equal to the first probability, it indicates that the difference between the reference value and the test / monitoring value under the first number of paths or samples meets the predefined accuracy requirement, that is, the measurement / monitoring result of the current AI model is applicable.

[0239] The following example further illustrates how to determine the test / monitoring result of the AI ​​model based on the difference between the reference value and the test / monitoring value of the first quantity diameter or sample when the reference value is a reference intermediate quantity and the reference intermediate quantity is a line-of-sight / non-line-of-sight indicator (LOS / NLOS indicator).

[0240] Example 3: The output of the AI ​​model is an intermediate quantity, which is the LOS / NLOS indicator.

[0241] When the output of the AI ​​model is an intermediate quantity, the NW (LMF) infers and calculates the reference value of the intermediate quantity through the position of the TRP and the first information (including the position information and measurement information of the PRU whose position is known). When the intermediate quantity is the LOS / NLOS indicator hard value, the model output is LOS / NLOS indicator hard value 1, and the corresponding reference value is LOS / NLOS indicator hard value 0;

[0242] 1) When the AI ​​model is deployed on the UE side: the UE sends the LOS / NLOS indicator hard value1 output by the AI ​​model to the LMF, and the LMF uses the LOS / NLOS indicator hard value1 and the corresponding LOS / NLOS indicator hard value0 as the test / monitoring value and reference value, respectively, and tests the applicability of the current model after deployment according to the predefined evaluation criteria, or the LMF sends the LOS / NLOS indicator hard value0 to the UE, and the UE uses the LOS / NLOS indicator hard value1 and the LOS / NLOS indicator hard value0 as the test / monitoring value and reference value, respectively, and the UE tests the applicability of the current model after deployment according to the predefined evaluation criteria;

[0243] 2) When the AI ​​model is deployed on the LMF side: LMF uses the output LOS / NLOS indicator hard value1 and the corresponding LOS / NLOS indicator hard value0 as test / monitoring and reference values, respectively, and tests the applicability of the current model after deployment according to predefined evaluation criteria.

[0244] Predefined single evaluation criteria: M is the first number of paths or samples, and 1 to M test values ​​value1 m With reference value value0 mComparing (1≤m≤M) the value1 m is not equal to value0 m The number M1, or the equal number M1', calculates p1=M1 / M or p1'=M1' / M (p1'=1-p1).

[0245] When N=1, compare p1 or p1' with the predefined precision p0, if p1 is less than or less than or equal to p0 (or p1' is greater than or greater than or equal to p0), the model is applicable;

[0246] When N is greater than 1, execute the above process multiple times to get (p1, p2, …, pN), compare Σpn / N with p0, if less than or less than or equal to p0, the model is applicable; or compare p1, p2, …, pN with p0 respectively, when the single model applicable times N1 (N1≤N) is greater than or greater than or equal to the predefined N0 (N0≤N), the model is applicable; or P1=N1 / N is greater than or greater than or equal to the predefined P0, the model is applicable;

[0247] The process for p1' is similar, which is not described here.

[0248] In particular, when performing pre-deployment testing, the reference value LOS / NLOS indicator hard value0 test equipment TE is known or directly provided by the TE to the device under test (DUT);

[0249] The method provided by the application gives a method for judging whether the AI model is applicable when the reference value is a reference intermediate quantity and the reference intermediate quantity is a line of sight / non-line of sight indicator, realizes the verification or monitoring of whether the AI model for positioning can adapt to the current environment, thereby improving the reliability of the AI model for positioning in the actual environment, and effectively guarantees the performance of AI positioning.

[0250] In some embodiments, in the case where the reference value belongs to the reference input data set and the test / monitoring value is the input value, the calculation of the difference between the reference value and the test / monitoring value of the first number of paths or samples includes one of the following:

[0251] In the case where the input value is the power or time delay or phase of the first number of paths or samples, the difference between the input value and the reference value of the same type in the reference input data set is calculated;

[0252] In the case where the input value is a first vector composed of the power or time delay or phase of the first number of paths or samples, the difference between the first vector and a second vector composed of the reference value of the same type in the reference input data set is calculated;

[0253] When the input value is a power, a delay, and a phase of a first quantity path or sample, or a combination of any two thereof, calculating a difference between the input value and a reference value of the same type in the reference input data set;

[0254] When the input value is a third vector consisting of power, delay and phase of a first quantity path or sample, or any two of them, a difference between the third vector and a fourth vector consisting of reference values ​​of the same type in the reference input data set is calculated.

[0255] Optionally, when the reference value belongs to a reference input data set and the test / monitoring value is an input value, the terminal first calculates the difference between the reference value and the test / monitoring value of the first quantity path or sample based on the reference value and the test / monitoring value, and then determines the test / monitoring result of the AI ​​model based on the difference.

[0256] The input value includes at least one of power, delay, and phase.

[0257] It should be noted that the reference value of the same type in the reference input data set refers to the reference value of the same type as the input value in the reference input data set. For example, if the input value is the power of a first quantity diameter or sample, then the reference of the same type in the reference input data set is the power of the same quantity diameter or sample in the reference input data set.

[0258] The following examples further illustrate how to determine the test / monitoring result of the AI ​​model based on the difference between the reference value and the test / monitoring value of the first quantity diameter or sample when the reference value belongs to the reference input data set and the test / monitoring value is the input value.

[0259] Example 4: Reference values ​​belong to the reference input data set, and test / monitoring values ​​are input values.

[0260] When the NW is able to obtain the reference input data set corresponding to the deployed AI model directly (provided by the NW manufacturer or agreed upon in the protocol) or indirectly (from the UE manufacturer or other server), the model input is monitored / tested.

[0261] The reference input data set may be generated autonomously by the LMF side or a server, or may be generated autonomously by the UE side, or may be a predefined data set or a protocol predefined data set, such as defined in the 3GPP protocol.

[0262] A model can correspond to one or more reference input data sets, and a set of reference input data sets can also correspond to one or more models.

[0263] The model input includes but is not limited to: CIR or PDP or DP, or a combination of one or more of the power, delay, and phase; the combination of one or more of the power, delay, and phase can be a vector or a single measurement.

[0264] When it is a vector, taking power as an example, it is expressed as the RSRP of the first number M of paths or samples, which constitutes a vector RSRP meas meas1 meas2 measM ] and the reference value is RSRP ref ref1 ref2 refM When it is a vector, taking delay as an example, it is expressed as the delay of the first number M of paths or samples, which constitutes a vector Delay meas meas1 meas2 measM ] and the reference value is Delay ref ref1 ref2 M .

[0265] When it is a single value, taking power as an example, the test value is the RSRP1 of a single path or sample, or the average value RSRP1 of the M paths or samples RSRP avg avg .

[0266] When the reported measurement is a combination of multiple categories, there are the following 1) ~ 4) situations:

[0267] 1) When the reported measurement is a combination of delay and power:

[0268] At this time, it is expressed as the combination of the first number M of paths or samples of delay and power, which constitutes a vector A meas M M ] and the reference value is A ref ref1 ref1 ref2 ref2 refM refM ​​​​​​​​​​​​​​​​​​​​​RSRP1*delay1 or the average of RSRP*delay of M paths or samples (RSRP1*delay1)avg avg , the reference value is RSRP0*delay0 or (RSRP0*delay0) avg ;

[0269] 2) When the reported measurement quantity is the combination of delay and phase:

[0270] At this time, it is expressed as the combination of the first number M paths or samples of delay and phase, which constitutes a vector B meas = [exp(j*phase1)*delay1, exp(j*phase2)*delay2, …, exp(j*phase M )*delay M ], the reference value at this time is B ref = [exp(j*phase ref1 )*delay ref1 , exp(j*phase ref2 )*delay ref2 , …, exp(j*phase refM )*delay refM ], the corresponding test value is exp(j*phase1)*delay1 of a single path or sample or the average of exp(j*phase)*delay of M paths or samples (exp(j*phase1)*delay1)avg when a single value, and the reference value is exp(j*phase0)*delay0 or (exp(j*phase0)*delay0) avg ;

[0271] 3) When the reported measurement quantity is the combination of phase and power:

[0272] At this time, it is expressed as the combination of the first number M paths or samples of phase and power, which constitutes a vector C meas = [RSRP1*exp(j*phase1), RSRP2*exp(j*phase2), …, RSRP M *exp(j*phase M ),], the reference value at this time is C ref = [RSRP ref1 *exp(j*phase ref1 ), RSRP ref2 *exp(j*phase ref2 ), …, RSRP refM *exp(j*phase refM)], when a single value is used, the corresponding test value is RSRP1*exp(j*phase1) of a single path or sample, or the average value of RSRP*exp(j*phase) of M paths or samples (RSRP1*exp(j*phase1)). avg , the reference value is RSRP0*exp(j*phase0) or (RSRP0*exp(j*phase0)) avg ;

[0273] 4) When the reported measurement quantity is a combination of delay, power, and power:

[0274] This is expressed as a combination of the delay, power, and power of the first number M paths or samples, forming a vector D meas =[RSRP1*exp(j*phase1)*delay1,RSRP2*exp(j*phase2)*delay2,…,RSRP M *exp(j*phase M )*delay M ],

[0275] The reference values ​​at this time are:

[0276] D ref =[RSRP ref1 *exp(j*phase ref1 )*delay ref1 ,RSRP ref2 *exp(j*phase ref2 )*delay ref2 ,…,RSRP refM *exp(j*phase refM )*delay refM ], when a single value is used, the corresponding test value is the average value of RSRP1*exp(j*phase1)*delay1 of a single path or sample or RSRP*exp(j*phase)*delay of M paths or samples (RSRP1*exp(j*phase1)*delay1) avg The reference value is RSRP0*exp(j*phase0)*delay0 or (RSRP0*exp(j*phase0)*delay0) avg .

[0277] When the AI model is deployed at the UE side: the UE reports the measurement quantity as the model input to the LMF, and the LMF takes the measurement quantity reported by the UE and the reference input data set corresponding to the model at this time as the test / monitoring value and the reference value respectively, and evaluates the applicability of the current model after deployment according to the pre-defined evaluation criteria; or, the LMF sends the UE the reference input data set corresponding to the model at this time, and the UE takes the measurement quantity at this time and the reference input data set as the test / monitoring value and the reference value respectively, and tests the applicability of the current model according to the pre-defined evaluation criteria;

[0278] When the AI model is deployed at the LMF side: the LMF takes the actually measured model input and the reference input data set as the test / monitoring value and the reference value respectively, and evaluates the applicability of the current model after deployment according to the pre-defined evaluation criteria;

[0279] The pre-defined evaluation criteria require N evaluations.

[0280] 4.1 When the difference is calculated in the form of a vector,

[0281] 1) The reported measurement quantity is power,

[0282] The pre-defined single evaluation criterion is ΔRSRP1=SGCS(RSRP meas , RSRP ref ), wherein SGCS represents the cosine similarity.

[0283] When N=1, ΔRSRP1 is compared with the pre-defined accuracy ΔRSRP0, and if ΔRSRP1 is less than or less than or equal to ΔRSRP0, the model is applicable;

[0284] When N is greater than 1, the above process is performed multiple times to obtain (ΔRSRP1, ΔRSRP2, …, ΔRSRPN), and ΣΔRSRPn / N is compared with ΔRSRP0, if it is less than or less than or equal to ΔRSRP0, the model is applicable; or ΔRSRP1, ΔRSRP2, …, ΔRSRPN are compared with ΔRSRP0 respectively, when the single model applicable times N1 (N1≤N) is greater than or greater than or equal to the pre-defined N0 (N0≤N), the model is applicable; or when P1=N1 / N is greater than or greater than or equal to the pre-defined P0, the model is applicable;

[0285] The above RSRP can also be replaced by latency or phase, which will not be described here.

[0286] 2) When the reported measurement quantity includes a combination of multiple categories

[0287] When the reported combination is latency and power:

[0288] The pre-defined single evaluation criterion is ΔA1=SGCS(A meas, A ref )

[0289] N = 1, compare ΔA1 with predefined accuracy ΔA0, ΔA1 is less than or less than or equal to ΔA0, the model is applicable;

[0290] N > 1, execute the above process multiple times to get (ΔA1, ΔA2, …, ΔAN), take ΣΔAn / N and compare with ΔA0, if less than or less than or equal to ΔA0, the model is applicable; or ΔA1, ΔA2, …, ΔAN are compared with ΔRSRP0 respectively, when the single model applicable times N1 (N1 ≤ N) is greater than or greater than or equal to the predefined N0 (N0 ≤ N), the model is applicable; or P1 = N1 / N is greater than or greater than or equal to the predefined P0, the model is applicable;

[0291] The combination of delay and phase, the combination of phase and power, and the combination of delay, power and power can be analogously extended, which will not be repeated here.

[0292] The above SGCS can also be a normalized mean squared error (NMSE), a Euclidean distance, an eigenvalue error, etc. evaluation matrix / vector similarity calculation method.

[0293] 4.2 When the difference is calculated in the form of a single value,

[0294] 1) The reported measurement quantity is power,

[0295] The predefined single evaluation criterion is ΔRSRP1 = abs (RSRP1-RSRP0) or ΔRSRPavg1 = abs (RSRP avg1-RSRP avg0).

[0296] N = 1, compare ΔRSRP1 or ΔRSRPavg1 with predefined accuracy ΔRSRP0, ΔRSRP1 or ΔRSRPavg1 is less than or less than or equal to ΔRSRP0, the model is applicable;

[0297] N > 1, execute the above process multiple times to get (ΔRSRP1, ΔRSRP2, …, ΔRSRPN) or (ΔRSRPavg1, ΔRSRPavg2, …, ΔRSRPavgN), take ΣΔRSRPn / N or ΣΔRSRPavgn / N and compare with ΔRSRP0, if less than or less than or equal to ΔRSRP0, the model is applicable; or ΔRSRP1, ΔRSRP2, …, ΔRSRPN are compared with ΔRSRP0 respectively, when the single model applicable times N1 (N1 ≤ N) is greater than or greater than or equal to the predefined N0 (N0 ≤ N), the model is applicable; or P1 = N1 / N is greater than or greater than or equal to the predefined P0, the model is applicable;

[0298] The RSRP can be replaced by delay or phase, which will not be described here.

[0299] 2) When the reported measurement quantity includes a combination of multiple categories,

[0300] The absolute value abs is changed to the two-norm ||.||.

[0301] The method provided by the application embodiment gives a method for judging whether an AI model is applicable in the case where the reference value is a reference input data set and the test / monitoring value is an input value, and the input value is at least one of power, time delay, and phase. The method realizes verification or monitoring of whether an AI model for positioning can adapt to the current environment, thereby improving the reliability of the AI model for positioning in actual deployment, and effectively guarantees the performance of AI positioning.

[0302] In some embodiments, the method provided by the application embodiment further includes:

[0303] In the case where the test / monitoring result is not applicable, the terminal sends the test / monitoring result to the network side device;

[0304] The terminal receives first indication information from the network side device, and the first indication information is used to instruct the terminal to perform a second operation;

[0305] The second operation includes at least one of the following:

[0306] updating the current model;

[0307] fine-tuning the current model;

[0308] retraining the current model;

[0309] deactivating the current model;

[0310] the terminal falls back to a non-AI mode;

[0311] restarting the current AI positioning related measurement;

[0312] stopping the current AI positioning related measurement;

[0313] extending the current AI positioning related measurement.

[0314] It can be understood that when the test / monitoring result of the AI model is not applicable, if the test / monitoring is completed by the terminal, the terminal sends the test / monitoring result to the network side device, and the network side device sends first indication information to the terminal to instruct the terminal to perform a second operation.

[0315] Optionally, the second operation includes at least one of the following: a fifth operation, a sixth operation, and a seventh operation.

[0316] Optionally, the fifth operation includes one of the following: updating the current model; fine-tuning the current model; and retraining the current model.

[0317] It should be noted that after the fifth operation, post-deployment testing or monitoring should also be performed; if the test fails within the specified time or this time, the sixth operation is performed.

[0318] Optionally, the sixth operation includes one of the following: deactivating the current model; the terminal falling back to a non-AI mode.

[0319] Optionally, deactivating the current model includes one of the following: selecting another activated model and switching to the selected model; selecting another inactivated model and activating the selected model; and rolling back the current model to a previous model.

[0320] Optionally, the seventh operation includes one of the following: restarting the current AI positioning related measurement; stopping the current AI positioning related measurement; and extending the current AI positioning related measurement.

[0321] Through the fifth and sixth operations, the AI ​​model can be adjusted; by performing the seventh operation, the impact of the current AI model on positioning performance can be reduced.

[0322] In some embodiments, the AI ​​model includes a first model, or an optimized model of the first model;

[0323] The first model includes at least one of the following:

[0324] a first model generated based on at least one of a protocol-standardized model structure, protocol-standardized model parameters, and a protocol-standardized dataset for training the model;

[0325] a first model determined based on a protocol standardized model structure and model parameters received by the terminal;

[0326] a received first model, the first model comprising at least a portion of a model structure and / or a portion of model parameters;

[0327] a first model generated or determined based on a first data set, where the first data set is a protocol-standardized data set or a data set sent by the network-side device to the terminal;

[0328] The terminal generates or determines a first model based on a second data set and a standardized model structure, where the second data set is a data set standardized by a protocol or a data set sent to the terminal by a peer device of the terminal.

[0329] The optimization model of the first model is a model having the same input and output mapping relationship as the first model or within a preset error range.

[0330] Optionally, the peer device of the terminal includes a network side device or a test equipment TE.

[0331] The optimization model of the first model refers to an optimization model based on the first model, which can be understood as a model that does not perfectly implement the first model but can implement a function similar to the first model, or the mapping relationship between the input and the output of the optimization model is the same as or within a certain error range of the first model, or can be understood as having a certain consistency.

[0332] Optionally, the terminal can maintain L optimization models based on the first model, and the L optimization models based on the first model can be respectively applicable to different situations, scenes, conditions or data distributions, etc. Alternatively, the performance similar to the first model can be obtained through the L optimization models based on the first model, wherein the target quantity of each optimization model is less than the first model, and the target quantity includes scale, complexity, and / or storage size. L is a positive integer greater than or equal to 1.

[0333] The method provided in the embodiment of the application describes the generation or obtaining method of the AI model described in each of the above embodiments. Through the test / monitoring method of the embodiment, it is verified or monitored whether the AI model for positioning generated or obtained by the above method can adapt to the current environment, thereby improving the reliability of the AI model generated or obtained by the above method in the actual environment, and effectively ensuring the performance of AI positioning.

[0334] FIG. 3 is a flowchart of a second method for testing or monitoring an artificial intelligence (AI) model for positioning according to an embodiment of the application. As shown in FIG. 3, the method provided in the embodiment includes the following steps.

[0335] In step 301, the network side device obtains a reference value and an input value.

[0336] It should be noted that the reference value in the embodiment of the application, as a reference value for testing or monitoring the AI model, can be reference position information, a reference intermediate quantity, a reference output data set, or a reference input data set.

[0337] The AI model described in the embodiment of the application is an AI model for positioning, i.e., participating in AI positioning. The output of the AI model can be position information or an intermediate quantity. The definition of the intermediate quantity will be described below.

[0338] The input value can be understood as the model input used to test or monitor the AI ​​model, which can be measurement information or model input information after processing the measurement information. In the embodiments of the present application, the measurement information generally refers to signal / channel measurement information, and the measurement information can also be referred to as measurement quantity, measurement result, etc.

[0339] In some embodiments, the network side device is a LMF, or other functional unit capable of performing positioning calculations.

[0340] Step 302: The network-side device performs a third operation, where the third operation includes one of the following:

[0341] Sending the reference value and the input value to the terminal, where the reference value and the input value are used to determine a test / monitoring result of an AI model for positioning deployed locally on the terminal;

[0342] Sending the input value to a terminal and receiving an output value from the terminal, wherein the output value is obtained by inputting the input value into an AI model for positioning deployed locally on the terminal, and determining a test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value, wherein the test / monitoring value is the output value;

[0343] Inputting the input value into an AI model for positioning deployed locally on the network-side device, obtaining an output value of the AI ​​model, and determining a test / monitoring result of the AI ​​model based on the reference value and a test / monitoring value, where the test / monitoring value is the output value;

[0344] Based on the reference value and the test / monitoring value, a test / monitoring result of the AI ​​model is determined, wherein the test / monitoring value is the input value.

[0345] Depending on where the AI ​​model is deployed, network-side devices can perform corresponding operations.

[0346] The AI ​​model is deployed on the terminal. In one embodiment, the network-side device obtains a reference value and an input value, and sends the reference value and input value to the terminal, and the terminal determines the test / monitoring result of the AI ​​model for positioning deployed locally on the terminal based on the reference value and input value. In another embodiment, after the network-side device obtains the reference value and input value, it sends the input value to the terminal, and the terminal inputs the input value into the AI ​​model, obtains the output value of the AI ​​model and sends it to the network-side device, and the network-side device receives the output value from the terminal, uses the output value as the test / monitoring value, and determines the test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value.

[0347] The AI model is deployed on a network side device. In one embodiment, the network side device obtains a reference value and an input value, inputs the input value into the AI model, obtains an output value of the AI model, takes the output value as a test / monitoring value, and determines a test / monitoring result of the AI model based on the reference value and the test / monitoring value. In another embodiment, the network side device obtains a reference value and an input value, takes the input value as a test / monitoring value, and determines a test / monitoring result of the AI model based on the reference value and the test / monitoring value.

[0348] The test or monitoring method for the AI model for positioning provided by the embodiments of the present application can verify or monitor the AI model for positioning by obtaining a reference value and an input value, thereby improving the reliability of the AI model for positioning in actual deployment and effectively ensuring the performance of AI positioning.

[0349] In some embodiments, the reference value and the input value include one of the following a) to d):

[0350] a) The reference value is reference position information, and the input value is measurement information or model input information obtained by processing the measurement information;

[0351] The reference position information refers to position information as a reference output. The position information can be rectangular coordinates, polar coordinates, latitude and longitude information, relative position, absolute position, etc., which are not limited in the present application.

[0352] The measurement information generally refers to signal / channel measurement information, and the measurement information can be replaced by the concepts of measurement quantity and measurement result.

[0353] In the embodiments of the present application, the measurement information includes at least one of the following: power, time delay, phase, channel impulse response (CIR), power delay profile (PDP), and delay profile (DP).

[0354] Optionally, the power includes received power, received power of a target path in multipath, signal-to-noise ratio, etc. The time delay includes received time delay, sending time delay, round-trip time delay, etc. The phase includes phase offset, phase delay, phase difference, etc. The channel impulse response includes impulse response, frequency response, etc.

[0355] The signal can pass through multiple different paths to reach the receiving end during transmission, and each path corresponds to a different propagation delay. The power delay spectrum describes the distribution of signal power at different delay times. The delay spectrum reflects the distribution of arrival delays of the received signal on different propagation paths, and is usually given in the form of a probability density function (PDF) or power spectral density (PSD).

[0356] When the AI model is deployed on the network side device, the network side device determines the test / monitoring result of the AI model according to the reference position information and an output value obtained by inputting an input value into the AI model for inference.

[0357] When the AI model is deployed on the terminal, the terminal sends the reference position information and the input value to the terminal.

[0358] Optionally, processing the measurement information refers to mathematical processing such as normalization and averaging of the measurement information.

[0359] In some embodiments, the manner of obtaining the reference position information and the input value includes:

[0360] The network side device selects position information of a first communication device with a known position as the reference position information, and the input value is measurement information corresponding to the first communication device or model input information obtained by processing the measurement information corresponding to the first communication device.

[0361] In the embodiments of the present application, the first communication device is a communication device with a known position, that is, the position of the first communication device is known to the network side device. The first communication device includes a positioning reference unit (PRU) or other communication devices supporting channel / signal measurement.

[0362] Optionally, the network side device selects a target PRU as the first communication device from multiple PRUs around the terminal, the target PRU reports its position information and measurement information to the network side device, and the network side device sends the position information of the target PRU as the reference position information and the measurement information of the target PRU as the input value to the terminal.

[0363] The method provided in the embodiments of the present application is suitable for the case where the output of the AI model is position information, and the inference ability of the AI model for positioning is verified or monitored by obtaining the reference position information and the input value, thereby improving the reliability of the AI model for positioning in actual deployment environment and effectively ensuring the performance of AI positioning.

[0364] b) the reference value is a reference intermediate quantity, the reference intermediate quantity being intermediate information for determining a terminal position, the reference intermediate quantity being of the same type as the output value of the AI model, the input value being measurement information or model input information obtained by processing the measurement information;

[0365] If the output of the AI model is an intermediate quantity, a reference intermediate quantity can be obtained, and measurement information or model input information obtained by processing the measurement information can be taken as an input value to test or monitor the AI model.

[0366] The reference intermediate quantity is of the same type as the output value of the AI model. For example, if the type of the output value of the AI model is RTOA, the type of the reference intermediate quantity should also be RTOA.

[0367] The reference intermediate quantity can also be referred to as a reference intermediate feature, which is intermediate information that can determine a terminal position.

[0368] In some embodiments, the reference intermediate quantity includes at least one of the following:

[0369] Line of Sight or Non-Line of Sight (LOS / NLOS) indicator;

[0370] Reference Signal Time Difference (RSTD);

[0371] Relative Time of Arrival (RTOA);

[0372] Reception-Transmission Rx-Tx time difference;

[0373] Time of Arrival (TOA);

[0374] Angle of Arrival (AoA);

[0375] Angle of Departure (AoD);

[0376] Reference Signal Received Power (RSRP);

[0377] Reference Signal Received Path Power (RSRPP).

[0378] In some embodiments, the reference intermediate quantity and the input value are obtained in one of the following ways:

[0379] The reference position information and the measurement information reported by the first communication device with known position are received, the reference intermediate quantity is determined based on the reference position information and second information used to assist in determining the reference intermediate quantity, and the measurement information or model input information obtained by processing the measurement information is used as the input value;

[0380] The reference intermediate quantity and the input value determined and reported by the first communication device with known position are received, and the input value is the measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

[0381] The embodiments of the present application provide two ways of determining the reference intermediate quantity:

[0382] 1) Direct determination, determined by the first communication device;

[0383] The network side device receives the reference intermediate quantity and the measurement information reported by the first communication device, where the reference intermediate quantity is determined by the first communication device.

[0384] Optionally, the network side device sends the reference intermediate quantity and the measurement information to the terminal.

[0385] 2) Indirect determination, that is, determined by the network side device based on the reference position information of the first communication device and second information, where the second information is used to assist in determining the reference intermediate quantity.

[0386] The network side device receives the reference position information and the measurement information reported by the first communication device, determines the reference intermediate quantity based on the reference position information and the second information, and sends the reference intermediate quantity and the measurement information to the terminal.

[0387] In the embodiments of the present application, the first communication device is a communication device with known position, that is, the position of the first communication device is known to the network side device. Optionally, the first communication device is selected by the network side device, and the first communication device includes a positioning reference unit (PRU) or other communication device supporting channel / signal measurement.

[0388] Optionally, the network side device selects a target PRU as the first communication device from a plurality of PRUs around the terminal, the target PRU reports its position information and measurement information to the network side device, the network side device takes the position information of the target PRU as the reference position information, determines the reference intermediate quantity based on the reference position information and the second information, and sends the measurement information of the PRU to the terminal as the input value.

[0389] In some embodiments, the second information comprises at least one of: a location of a transmission and reception point (TRP) currently participating in AI positioning, historical data of measurement information, historical data of AI model output values, a historical location of the terminal, a historical location of the TPR, and a historical location of the first communication device whose location is known.

[0390] For example, the second information is a location of a transmission and reception point (TRP) currently participating in AI positioning. Based on the location information of the target PRU and the location of the TRP currently participating in AI positioning, the network-side device calculates intermediate quantities when the target PRU performs positioning using the TRP currently participating in AI positioning by using mathematical geometric relationships. These intermediate quantities are the reference intermediate quantities. The target PRU measures the downlink positioning reference signal (DL-PRS) transmitted by the TRP currently participating in AI positioning to obtain measurement information and sends the measurement information to the network-side device. Further, the network-side device sends the reference intermediate quantities and the measurement information to the terminal. The terminal takes the reference intermediate quantities as reference values and inputs the measurement information into the AI model to obtain the intermediate quantities output by the AI model. The terminal takes the intermediate quantities output by the AI model as test / monitoring values and compares them with the reference intermediate quantities to determine the test / monitoring result of the AI model. Alternatively, the network-side device sends the measurement information to the terminal. The terminal inputs the measurement information into the AI model to obtain the intermediate quantities output by the AI model and sends the intermediate quantities output by the AI model as test / monitoring values to the network-side device. The network-side device compares the intermediate quantities output by the AI model with the reference intermediate quantities to determine the test / monitoring result of the AI model.

[0391] The second information can also be other auxiliary data, including historical data of measurement information, historical data of AI model output values, a historical location of the terminal, a historical location of the TRP, and / or a historical location of the first communication device whose location is known.

[0392] In some embodiments, when the AI model is deployed on the terminal side, the terminal inputs the input values into the AI model, obtains the intermediate quantities output by the AI model, and sends the intermediate quantities to the network-side device. The network-side device compares the intermediate quantities output by the AI model with the reference intermediate quantities to determine the test / monitoring result of the AI model.

[0393] In some embodiments, when the AI model is deployed on the network side, the network-side device inputs the measurement information or model input information obtained by processing the measurement information into the AI model deployed locally on the network side as input values, obtains the intermediate quantities output by the AI model, and determines the test / monitoring result of the AI model based on the intermediate quantities output by the AI model and the reference intermediate quantities.

[0394] It is worth noting that in the case of AI model deployment at the LMF or gNB side: the LMF side inputs the measurement information or the model input information obtained after processing the measurement information into the AI model for inference, and takes the intermediate quantity output by the AI model and the corresponding reference intermediate quantity as test / monitoring and reference values respectively to determine the test / monitoring result of the AI model.

[0395] In the case of AI model deployment at the gNB side: the gNB side inputs the measurement information or the model input information obtained after processing the measurement information into the AI model for inference, and reports the intermediate quantity output by the AI model to the LMF, which takes the intermediate quantity output by the AI model and the corresponding reference intermediate quantity as test / monitoring and reference values respectively to determine the test / monitoring result of the AI model.

[0396] Optionally, the gNB reports the intermediate quantity output by the AI model to the LMF, which needs to meet the reporting requirements, including the reporting range and granularity, which is selected by the gNB according to the LMF indication and notified to the LMF.

[0397] The method provided by the embodiments of the present application is suitable for the case where the output of the AI model is an intermediate quantity, and the test / monitoring result of the AI model is determined based on the reference intermediate quantity and the intermediate quantity output by the AI model, which realizes the verification or monitoring of the inference ability of the AI model for positioning, thereby improving the reliability of the AI model for positioning in actual deployment environment and effectively ensuring the performance of AI positioning.

[0398] c) the reference value belongs to a reference output data set, and the input value belongs to a reference input data set;

[0399] It can be understood that in the embodiments, the AI model can execute the AI function according to the reference input data set to obtain the corresponding model output, and the model output is taken as the test / monitoring value, and the test / monitoring of the AI model is realized based on the reference output data set and the test / monitoring value.

[0400] The reference output data set includes reference values of the same type as the output value of the AI model.

[0401] Optionally, one AI model can correspond to one or more groups of reference data sets, and one group of reference data sets can also correspond to one or more AI models.

[0402] The reference output data set or the reference input data set is generated autonomously by a terminal, or autonomously by a network side device, or by a first device, or predefined by a protocol.

[0403] The first device can be a server, or other device except the terminal and the network side device.

[0404] Optionally, the network side device autonomously generates the reference data set, or receives the reference data set generated and sent by the first device, and in the case that the AI model is deployed on the terminal side, the network side device sends the reference data set to the terminal, the reference data set includes the parameter input data set and the reference output data set, the terminal takes the reference input data set as the model input, takes the model output at this time as the test / monitoring value, and compares the test / monitoring value with the reference output data set, so as to determine the test / monitoring result of the AI model.

[0405] Optionally, in the case that the AI model is deployed on the terminal side, the network side device sends the reference input data set to the terminal, the terminal takes the reference input data set as the model input, takes the model output at this time as the test / monitoring value, and reports the test / monitoring value to the network side device, the network side device compares the test / monitoring value with the reference output data set, so as to determine the test / monitoring result of the AI model. The reference input data set or the reference output data set is from the first device, or is autonomously generated by the network side device.

[0406] Optionally, in the case that the AI model is deployed on the network side, the network side device takes the reference input data set as the model input, takes the model output at this time as the test / monitoring value, and compares the test / monitoring value with the reference output data set, so as to determine the test / monitoring result of the AI model. The reference input data set or the reference output data set is predefined by the protocol, or is from the first device, or is autonomously generated by the network side device.

[0407] It should be noted that, when the AI model is deployed on the LMF side, the LMF takes the reference input data set as the model input, takes the model output at this time as the test / monitoring value, compares the test / monitoring value with the reference output data set, and determines the test / monitoring result of the AI model; in addition, when the AI model is deployed on the gNB side, the LMF sends the reference input data set to the gNB, the gNB takes the reference input data set as the model input, the gNB optionally reports the measurement result to the LMF, the LMF takes the measurement result reported by the gNB as the test / monitoring value, compares the test / monitoring value with the reference output data set, and determines the test / monitoring result of the AI model.

[0408] The method provided by the embodiments of the present application is applicable to the case that the AI model can perform an AI function according to a reference data set. The reference input data set is input into the AI model for positioning deployed locally in the terminal, the output value of the AI model is obtained, and the test / monitoring result of the AI model is determined based on the reference output data set and the output value of the AI model, so as to verify or monitor the inference ability of the AI model for positioning, thereby improving the reliability of the AI model for positioning deployed in the actual environment and effectively ensuring the performance of AI positioning.

[0409] d) The reference value belongs to the reference input data set, and the input value is measurement information or model input information obtained by processing the measurement information.

[0410] In order to verify whether the AI model can adapt to the current environment, the embodiments of the present application propose to compare the measurement information or model input information obtained by processing the measurement information as model input with the reference input data set, so as to test / monitor the input of the AI model.

[0411] The terminal can obtain the measurement information by actual measurement, for example, by measuring the downlink positioning reference signal transmitted by the TRP of the current reference AI positioning.

[0412] Optionally, in the case that the AI model is deployed on the terminal side, the network side device sends the reference input data set to the terminal, and the terminal determines the test / monitoring result of the AI model based on the reference input data set and the measurement quantity as model input.

[0413] Optionally, in the case that the AI model is deployed on the terminal side, the network side device receives the measurement quantity reported by the terminal, the network side device takes the measurement quantity reported by the terminal as a test / monitoring value, and determines the test / monitoring result of the AI model based on the reference input data set and the test / monitoring value. Here, the measurement quantity as model input includes measurement information or model input information obtained by processing the measurement information.

[0414] Optionally, in the case that the AI model is deployed on the network side, the network side device takes the actually measured model input and the reference input data set as a test / monitoring value and a reference value respectively, and determines the test / monitoring result of the AI model.

[0415] In some embodiments, the reference input data set is generated by the terminal, or is generated by the network side device, or is generated by a first device, or is predefined by a protocol.

[0416] The first device can be a server, or a device other than the terminal and the network side device.

[0417] It should be noted that when the AI model is deployed in the gNB, if the measurement quantity of the model input is measured by the gNB, the gNB can optionally report the measurement result to the LMF, and the LMF determines the test / monitoring result of the AI model according to the actual measured model input and the reference input data set as the test / monitoring value and the reference value respectively.

[0418] The method provided in the embodiments of the present application is suitable for the case that the AI model can perform AI functions according to the reference data set, and the test / monitoring result of the AI model is determined by acquiring the reference input data set and the input value, which realizes the verification or monitoring of whether the input of the AI model for positioning can adapt to the current environment, thereby improving the reliability of the AI model for positioning in the actual environment, and effectively ensuring the performance of AI positioning.

[0419] In some embodiments, the test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value, including:

[0420] The difference between the reference value and the test / monitoring value of the first number of paths or samples is calculated.

[0421] The test / monitoring result of the AI model is determined based on the difference between the reference value and the test / monitoring value of the first number of paths or samples.

[0422] The first number is greater than or equal to 1.

[0423] In the embodiments of the present application, the AI model can be tested or monitored once or multiple times.

[0424] If the number of times of testing or monitoring the AI model is 1, the test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value, including:

[0425] The difference between the reference value and the test / monitoring value is calculated, and compared with the predefined accuracy requirement. If the predefined accuracy requirement is met, it is determined that the AI model is applicable, and if the predefined accuracy requirement is not met, it is determined that the AI model is not applicable.

[0426] In order to improve the accuracy of the test / monitoring, the AI model can also be tested or monitored multiple times.

[0427] Optionally, when the number of times is greater than 1, the average value of the difference between the test / monitoring value and the reference value of each execution is compared with the predefined accuracy, or the ratio of the number of times of evaluating the model as applicable to the total evaluation times needs to be greater than a first threshold value (for example, a certain percentage, for example, 90%), or the number of times of evaluating the model as applicable is greater than a predefined evaluation number (for example, 4), at this time, the test / monitoring result of the AI model is applicable, otherwise, it is not applicable.

[0428] The difference between the reference value and the test / monitoring value of the first number of paths or samples can be calculated in each execution of the test / monitoring, and the test / monitoring result of the AI model is determined based on the difference between the reference value and the test / monitoring value of the first number of paths or samples, so as to improve the accuracy of the test / monitoring of the AI model.

[0429] The method provided by the embodiments of the present application can improve the accuracy of the test / monitoring of the AI model by calculating the difference between the reference value and the test / monitoring value of the first number of paths or samples, and determining the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first number of paths or samples.

[0430] In some embodiments, the calculation of the difference between the reference value and the test / monitoring value of the first number of paths or samples includes at least one of the following:

[0431] Euclidean distance between a vector composed of the reference values of the first number of paths or samples and a vector composed of the test / monitoring values;

[0432] Cosine similarity between a vector composed of the reference values of the first number of paths or samples and a vector composed of the test / monitoring values;

[0433] Root mean square error between a vector composed of the reference values of the first number of paths or samples and a vector composed of the test / monitoring values;

[0434] Difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0435] Average value of the absolute value sum of the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0436] Root mean square error of the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0437] Straight line distance between the reference value and the test / monitoring value of the first number of paths or samples;

[0438] Average value of the straight line distance between the reference value and the test / monitoring value of the first number of paths or samples;

[0439] Root mean square of the straight line distance between the reference value and the test / monitoring value of the first number of paths or samples.

[0440] The method provided by the embodiments of the present application can calculate the difference between the first quantity path or the sample reference value and the test / monitoring value in a vector manner, for example, calculate the Euclidean distance, cosine similarity, root mean square error, etc. between the vector composed of the first quantity path or the sample reference value and the vector composed of the test / monitoring value.

[0441] Alternatively, the difference between the first quantity path or the sample reference value and the test / monitoring value can also be calculated in a single value manner, for example, calculate the difference between the first quantity path or the sample reference value and the test / monitoring value, the average of the absolute value of the difference, or the root mean square error of the difference.

[0442] Alternatively, the difference between the first quantity path or the sample reference value and the test / monitoring value can also be calculated in a straight line distance manner, for example, calculate the straight line distance between the first quantity path or the sample reference value and the test / monitoring value, the average of the straight line distance, or the root mean square of the straight line distance.

[0443] The method provided by the embodiments of the present application realizes the flexibility of difference calculation.

[0444] In some embodiments, the determination of the test / monitoring result of the AI model based on the difference between the reference value of the first quantity path or the sample and the test / monitoring value comprises:

[0445] In the case that the difference between the reference value of the first quantity path or the sample and the test / monitoring value meets the predefined accuracy requirement, the test / monitoring result of the AI model is determined to be applicable.

[0446] In the case that the difference between the reference value of the first quantity path or the sample and the test / monitoring value does not meet the predefined accuracy requirement, the test / monitoring result of the AI model is determined to be not applicable.

[0447] It can be understood that the difference between the reference value of the first quantity path or the sample and the test / monitoring value meeting the predefined accuracy requirement indicates that the inference ability of the current AI model can meet the current positioning environment, or that the current AI model can adapt to the current positioning environment, that is, it can be determined that the current AI model is applicable.

[0448] Alternatively, the difference between the reference value of the first quantity path or the sample and the test / monitoring value meeting the predefined accuracy requirement comprises:

[0449] In the case that the number of executions is equal to or greater than 1, the average of the difference between the test / monitoring value and the reference value of each execution is compared with the predefined accuracy, or the ratio of the number of times that the single evaluation model is applicable to the total evaluation times is greater than the first threshold value, or the number of times that the single evaluation model is applicable is greater than the predefined evaluation number, and the test / monitoring result of the AI model is applicable.

[0450] The difference between the reference value and the test / monitoring value of the first number of paths or samples does not meet the predefined accuracy requirement, indicating that the inference ability of the current AI model cannot meet the current positioning environment, or indicating that the current AI model cannot adapt to the current positioning environment, that is, it can be determined that the current AI model is not applicable.

[0451] In some embodiments, the predefined accuracy requirement is applicable to a specified condition, and it can be understood that the predefined accuracy requirement is related to the specified condition and needs to be applicable under the specified condition. The predefined accuracy requirement can be different under different specified conditions.

[0452] Optionally, the specified condition includes at least one of the following: signal-to-interference noise ratio, signal-to-noise ratio, channel strength, noise strength, channel model, reference signal bandwidth, reference signal configuration, subcarrier spacing, carrier frequency, frequency band information, duplexing mode, TRP configuration, UE configuration, AI model structure, AI model complexity, and AI quantization information.

[0453] It can be understood that the predefined accuracy requirement can be determined according to at least one of the following: signal-to-noise ratio, channel strength, noise strength, channel model, reference signal bandwidth, reference signal configuration, subcarrier spacing, carrier frequency, frequency band information, duplexing mode, TRP configuration, UE configuration, AI model structure, AI model complexity, and AI quantization information.

[0454] The method provided by the present application defines an evaluation criterion for judging whether an AI model is applicable. The evaluation criterion can also be applicable to the evaluation or test of the AI model before deployment. In the test, the NW side device is replaced by a test equipment (TE).

[0455] In some embodiments, in the case that the reference value is a reference intermediate quantity and the reference intermediate quantity is a line-of-sight / non-line-of-sight indication, the first probability under the specified condition is taken as the reference accuracy.

[0456] The difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement, which includes:

[0457] A second probability of the difference between the reference value and the test / monitoring value under the first number of paths or samples is calculated.

[0458] In a case where the first probability represents a maximum mismatch probability of the LOS / NLOS indicator, and the second probability is less than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement.

[0459] Alternatively,

[0460] A third probability that the reference value and the test / monitoring value are the same under the first number of paths or samples is calculated.

[0461] In a case where the first probability represents a minimum match probability of the LOS / NLOS indicator, and the third probability is greater than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement.

[0462] It should be noted that the LOS / NLOS indicator is an indication domain indicating LOS or NLOS through different values.

[0463] According to the result (1 or 0) of AI model inference as the test / monitoring value, the probability that the reference value and the test / monitoring value are the same or different under the first number of paths or samples is calculated, and compared with the first probability.

[0464] If the first probability represents a maximum mismatch probability of the LOS / NLOS indicator, the probability that the reference value and the test / monitoring value are different under the first number of paths or samples is calculated, and if the probability that the reference value and the test / monitoring value are different under the first number of paths or samples is less than the first probability, or less than or equal to the first probability, it indicates that the difference between the reference value and the test / monitoring value under the first number of paths or samples meets the predefined accuracy requirement, that is, the measurement / monitoring result of the current AI model is applicable.

[0465] If the first probability represents a minimum match probability of the LOS / NLOS indicator, the probability that the reference value and the test / monitoring value are the same under the first number of paths or samples is calculated, and if the probability that the reference value and the test / monitoring value are the same under the first number of paths or samples is greater than the first probability, or greater than or equal to the first probability, it indicates that the difference between the reference value and the test / monitoring value under the first number of paths or samples meets the predefined accuracy requirement, that is, the measurement / monitoring result of the current AI model is applicable.

[0466] The method provided by the application provides a method for judging whether the AI model is applicable in a case where the reference value is a reference intermediate quantity and the reference intermediate quantity is a LOS / NLOS indicator, realizes verification or monitoring of whether the AI model for positioning can adapt to the current environment, thereby improving the reliability of the AI model for positioning in the actual environment, and can effectively guarantee the performance of AI positioning.

[0467] In some embodiments, the input value comprises at least one of: power, time delay, phase.

[0468] Optionally, in the case that the reference value belongs to the reference input dataset and the test / monitoring value is the input value, the calculating the difference between the reference value and the test / monitoring value of the first number of paths or samples comprises one of:

[0469] In the case that the input value is the power or the time delay or the phase of the first number of paths or samples, the difference between the input value and the reference value of the same type in the reference input dataset is calculated;

[0470] In the case that the input value is a first vector composed of the power or the time delay or the phase of the first number of paths or samples, the difference between the first vector and a second vector composed of the reference value of the same type in the reference input dataset is calculated;

[0471] In the case that the input value is a combination of the power, the time delay and the phase of the first number of paths or samples or any two of them, the difference between the input value and the reference value of the same type in the reference input dataset is calculated;

[0472] In the case that the input value is a third vector composed of the power, the time delay and the phase of the first number of paths or samples or any two of them, the difference between the third vector and a fourth vector composed of the reference value of the same type in the reference input dataset is calculated.

[0473] Optionally, in the case that the reference value belongs to the reference input dataset and the test / monitoring value is the input value, the terminal first calculates the difference between the reference value and the test / monitoring value of the first number of paths or samples based on the reference value and the test / monitoring value, and then determines the test / monitoring result of the AI model based on the difference.

[0474] The input value comprises at least one of: power, time delay, phase.

[0475] It should be noted that the reference value of the same type in the reference input dataset refers to the reference value of the same type as the input value in the reference input dataset, for example, the input value is the power of the first number of paths or samples, and the reference value of the same type in the reference input dataset is the power of the same number of paths or samples in the reference input dataset.

[0476] The method provided by the application provides a method for determining whether an AI model is applicable when a reference value is a reference input data set and a test / monitoring value is an input value, and the input value is at least one of power, time delay, and phase. The method verifies or monitors whether an AI model for positioning can adapt to a current environment, thereby improving the reliability of deploying the AI model for positioning in an actual environment and effectively ensuring the performance of AI positioning.

[0477] How to determine the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first number of paths or samples can refer to examples 1-4 in the foregoing terminal-side method embodiments, which will not be described here.

[0478] In some embodiments, the method further comprises:

[0479] receiving a test / monitoring result from the terminal, the test / monitoring result being inapplicable;

[0480] sending first indication information to the terminal, the first indication information being used to instruct the terminal to perform a second operation;

[0481] The second operation comprises at least one of:

[0482] updating the current model;

[0483] fine-tuning the current model;

[0484] retraining the current model;

[0485] deactivating the current model;

[0486] the terminal falls back to a non-AI mode;

[0487] restarting a current AI positioning-related measurement;

[0488] stopping a current AI positioning-related measurement;

[0489] extending a current AI positioning-related measurement.

[0490] It can be understood that when the test / monitoring result of the AI model is inapplicable, if the test / monitoring is completed by the terminal, the terminal sends the test / monitoring result to the network-side device, the network-side device receives the test / monitoring result from the terminal, and sends first indication information to the terminal to instruct the terminal to perform a second operation. The purpose of the second operation is to adjust the AI model by performing operations such as updating, fine-tuning, or retraining the current model, or deactivating the current model, activating other models, switching the current model, selecting other models, or falling back the current model to a past model; or restarting, stopping, or extending a current AI positioning-related measurement, thereby reducing the influence of the current AI model on the positioning performance.

[0491] In some embodiments, the method further comprises:

[0492] In the case that the test / monitoring result is not applicable, the network side device sends second indication information to the terminal, the second indication information being used to instruct the terminal to perform a fourth operation;

[0493] The fourth operation comprises: restarting the current AI positioning related measurement, stopping the current AI positioning related measurement, prolonging the current AI positioning related measurement.

[0494] It can be understood that when the test / monitoring result of the AI model is not applicable, if the test / monitoring is completed by the network side device, the network side device sends second indication information to the terminal to instruct the terminal to perform a fourth operation, and the purpose of the fourth operation is to reduce the influence of the current AI model on the positioning performance by restarting, stopping or prolonging the current AI positioning related measurement.

[0495] In some embodiments, the AI model comprises a first model, or an optimization model of the first model;

[0496] The first model comprises at least one of:

[0497] The first model is generated based on at least one of a protocol standardized model structure, a protocol standardized model parameter and a protocol standardized data set for training the model;

[0498] The first model is determined based on the protocol standardized model structure and the received model parameter;

[0499] The received first model comprises at least part of the model structure and / or part of the model parameter;

[0500] The first model is generated or determined based on a first data set, the first data set being a protocol standardized data set or a data set autonomously generated by the network side device;

[0501] The first model is generated or determined based on a second data set and a standardized model structure, the second data set being a protocol standardized data set or a data set autonomously generated by the network side device or a test device.

[0502] The optimization model of the first model refers to an optimization model based on the first model, which can be understood as a model that does not perfectly implement the first model, but can implement a function similar to the first model, or the mapping relationship between the input and output of the optimization model is the same as or within a certain error range of the first model, or can be understood as having a certain consistency.

[0503] Optionally, the terminal can maintain L optimization models based on the first model, and the L optimization models based on the first model can be respectively applicable to different cases, scenarios, conditions, or data distributions, etc. Alternatively, the performance similar to the first model can be obtained through the L optimization models based on the first model, and the target quantity of each optimization model is less than the first model, and the target quantity includes scale, complexity, and / or storage size. L is a positive integer greater than or equal to 1.

[0504] The method provided by the embodiment of the application describes the generation or obtaining method of the AI model described in each of the above embodiments. Through the testing / monitoring method of the embodiment, it is verified or monitored whether the AI model for positioning generated or obtained by the above method can adapt to the current environment, thereby improving the reliability of the AI model generated or obtained by the above method in the actual environment, and effectively guaranteeing the performance of AI positioning.

[0505] The testing or monitoring method of the artificial intelligence AI model for positioning provided by the embodiment of the application can be executed by a testing or monitoring device for the artificial intelligence AI model for positioning. In the embodiment of the application, the testing or monitoring method of the artificial intelligence AI model for positioning is executed by the testing or monitoring device for the artificial intelligence AI model for positioning as an example to illustrate the testing or monitoring device for the artificial intelligence AI model for positioning provided by the embodiment of the application.

[0506] FIG. 4 is a structural schematic diagram of the testing or monitoring device for the artificial intelligence AI model for positioning provided by the embodiment of the application. As shown in FIG. 4, the testing or monitoring device 400 for the artificial intelligence AI model for positioning includes:

[0507] The first obtaining unit 410 is configured to obtain a reference value and an input value.

[0508] The first execution unit 420 is configured to execute a first operation, and the first operation includes one of the following:

[0509] The input value is input into the AI model for positioning deployed locally in the terminal, and an output value of the AI model is obtained. The output value is taken as a test / monitoring value, and a test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value.

[0510] The input value is taken as a test / monitoring value, and a test / monitoring result of the AI model is determined based on the reference value and the test / monitoring value.

[0511] In the embodiments of the present application, by obtaining the reference value and the input value, inputting the input value into the AI model for positioning locally deployed by the terminal, obtaining the output value of the AI model, taking the output value as the test / monitoring value, or taking the input value as the test / monitoring value, determining the test / monitoring result of the AI model based on the reference value and the test / monitoring value, the verification or monitoring of the AI model for positioning is realized, thereby improving the reliability of the AI model for positioning deployed in the actual environment, and the performance of AI positioning can be effectively guaranteed.

[0512] Optionally, the reference value and the input value include one of the following:

[0513] The reference value is reference position information, and the input value is measurement information or model input information obtained after processing the measurement information.

[0514] The reference value is a reference intermediate quantity, the reference intermediate quantity is intermediate information used to determine the position of the terminal, the reference intermediate quantity is of the same type as the output value of the AI model, and the input value is measurement information or model input information obtained after processing the measurement information.

[0515] The reference value belongs to a reference output data set, and the input value belongs to a reference input data set.

[0516] The reference value belongs to a reference input data set, and the input value is measurement information or model input information obtained after processing the measurement information.

[0517] Optionally, the reference position information and the input value are obtained in the following manner:

[0518] Receiving first information from the network side device, the first information including the reference position information and the input value, the reference position information being position information of a first communication device with known position selected by the network side device, and the input value being measurement information of the first communication device or model input information obtained after processing the measurement information of the first communication device.

[0519] Optionally, the reference intermediate quantity and the input value are obtained in the following manner:

[0520] Receiving the reference intermediate quantity and the input value from the network side device;

[0521] The reference intermediate quantity is obtained based on reference position information reported by a first communication device with known position and second information used to assist in determining the reference intermediate quantity, or the reference intermediate quantity is determined by the first communication device and reported to the network side device.

[0522] The input value is measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

[0523] Optionally, the second information includes at least one of the following: a current transmission and reception point (TRP) location participating in AI positioning, historical data of measurement information, historical data of the AI model output value, a historical location of the terminal, a historical location of the TRP, and a historical location of the first communication device.

[0524] Optionally, the reference intermediate quantity includes at least one of the following: a line-of-sight / non-line-of-sight indication; a reference signal time difference (RSTD); a relative time of arrival (RTOA); a receive-transmit (Rx-Tx) time difference; a time of arrival (TOA); an angle of arrival (AoA); an angle of departure (AoD); a reference signal received power (RSRP); and a reference signal received path power (RSRPP).

[0525] Optionally, the measurement information includes at least one of the following: power, time delay, phase, channel impulse response (CIR), power delay profile (PDP), and delay profile (DP).

[0526] Optionally, the reference output data set or the reference input data set is autonomously generated by the terminal, autonomously generated by the network side device, generated by the first device, or predefined by a protocol.

[0527] Optionally, the determination of the test / monitoring result of the AI model based on the reference value and the test / monitoring value includes:

[0528] calculating a difference between the reference value and the test / monitoring value of a first number of paths or samples;

[0529] determining the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0530] wherein the first number is greater than or equal to 1.

[0531] Optionally, the calculation of the difference between the reference value and the test / monitoring value of the first number of paths or samples includes at least one of the following:

[0532] calculating an Euclidean distance between a vector composed of the reference value and a vector composed of the test / monitoring value of the first number of paths or samples;

[0533] calculating a cosine similarity between a vector composed of the reference value and a vector composed of the test / monitoring value of the first number of paths or samples;

[0534] calculating a root mean square error between a vector composed of the reference value and a vector composed of the test / monitoring value of the first number of paths or samples.

[0535] calculating the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0536] calculating the average of the absolute value sum of the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0537] calculating the root mean square error of the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0538] calculating the straight line distance between the reference value and the test / monitoring value of the first number of paths or samples;

[0539] calculating the average of the straight line distance between the reference value and the test / monitoring value of the first number of paths or samples;

[0540] calculating the root mean square of the straight line distance between the reference value and the test / monitoring value of the first number of paths or samples.

[0541] Optionally, determining the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first number of paths or samples comprises:

[0542] determining that the test / monitoring result of the AI model is applicable in the case that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement.

[0543] determining that the test / monitoring result of the AI model is not applicable in the case that the difference between the reference value and the test / monitoring value of the first number of paths or samples does not meet the predefined accuracy requirement.

[0544] Optionally, the predefined accuracy requirement is applicable to a specified condition, and the specified condition comprises at least one of the following: signal-to-interference noise ratio, signal-to-noise ratio, channel strength, noise strength, channel model, reference signal bandwidth, reference signal configuration, subcarrier spacing, carrier frequency, frequency band information, duplexing mode, TRP configuration, UE configuration, AI model structure, AI model complexity, AI quantization information.

[0545] Optionally, in the case that the reference value is a reference intermediate quantity and the reference intermediate quantity is a line-of-sight / non-line-of-sight indication, the first probability under the specified condition is taken as the reference accuracy.

[0546] The difference between the reference value and the test / monitoring value of the first number of paths or samples meeting the predefined accuracy requirement comprises:

[0547] calculating a second probability of the reference value and the test / monitoring value being different under the first number of paths or samples;

[0548] in case that the first probability represents a maximum mismatch probability of the line-of-sight / non-line-of-sight indication, and the second probability is less than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement;

[0549] or,

[0550] a third probability that the reference value and the test / monitoring value under the first number of paths or samples are the same is calculated;

[0551] in case that the first probability represents a minimum match probability of the line-of-sight / non-line-of-sight indication, and the third probability is greater than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement.

[0552] Optionally, in case that the reference value belongs to a reference input data set, and the test / monitoring value is the input value,

[0553] the calculation of the difference between the reference value and the test / monitoring value of the first number of paths or samples comprises one of the following:

[0554] in case that the input value is power or time delay or phase of the first number of paths or samples, the difference between the input value and a reference value of the same type in the reference input data set is calculated;

[0555] in case that the input value is a first vector composed of power or time delay or phase of the first number of paths or samples, the difference between the first vector and a second vector composed of reference values of the same type in the reference input data set is calculated;

[0556] in case that the input value is a combination of power, time delay and phase of the first number of paths or samples or any two of them, the difference between the input value and a reference value of the same type in the reference input data set is calculated;

[0557] in case that the input value is a third vector composed of power, time delay and phase of the first number of paths or samples or any two of them, the difference between the third vector and a fourth vector composed of reference values of the same type in the reference input data set is calculated.

[0558] Optionally, the apparatus further comprises a first processing unit for:

[0559] in case that the test / monitoring result is not applicable, the terminal sends the test / monitoring result to the network side device;

[0560] The terminal receives first indication information from the network side device, and the first indication information is used to instruct the terminal to perform a second operation.

[0561] The second operation includes at least one of the following:

[0562] updating a current model;

[0563] fine-tuning the current model;

[0564] retraining the current model;

[0565] deactivating the current model;

[0566] the terminal falls back to a non-AI mode;

[0567] restarting a current AI positioning related measurement;

[0568] stopping a current AI positioning related measurement;

[0569] extending a current AI positioning related measurement.

[0570] Optionally, the AI model includes a first model, or an optimized model of the first model;

[0571] The first model includes at least one of the following:

[0572] a first model generated based on at least one of a protocol standardized model structure, a protocol standardized model parameter, and a protocol standardized data set for training a model;

[0573] a first model determined based on a protocol standardized model structure and a model parameter received by the terminal;

[0574] a received first model, the first model including at least part of a model structure and / or part of a model parameter;

[0575] a first model generated or determined based on a first data set, the first data set being a protocol standardized data set or a data set sent by the network side device to the terminal;

[0576] a first model generated or determined by the terminal based on a second data set and a protocol standardized model structure, the second data set being a protocol standardized data set or a data set sent by a peer device of the terminal to the terminal.

[0577] The test or monitoring apparatus for the AI model for positioning in the embodiments of the present application can be an electronic device, for example, an electronic device with an operating system, or a component in the electronic device, for example, an integrated circuit or a chip. The electronic device can be a terminal or other device than the terminal. Exemplarily, the terminal can include, but is not limited to, the types of the terminal 11 listed above, and the other device can be a server, a network attached storage (NAS), etc., which is not limited in the embodiments of the present application.

[0578] The test or monitoring apparatus for the AI model for positioning provided in the embodiments of the present application can implement the various processes implemented by the method embodiments of FIG. 2 and achieve the same technical effects. To avoid repetition, details are not described herein.

[0579] FIG. 5 is a structural schematic diagram of the test or monitoring apparatus for the AI model for positioning provided in the embodiments of the present application. As shown in FIG. 5, the test or monitoring apparatus 500 for the AI model for positioning includes:

[0580] The second obtaining unit 510 is configured to obtain a reference value and an input value.

[0581] The second execution unit 520 is configured to perform a third operation, and the third operation includes one of the following:

[0582] The reference value and the input value are sent to a terminal, and the reference value and the input value are used to determine a test / monitoring result of an AI model for positioning deployed locally by the terminal;

[0583] The input value is sent to a terminal, and an output value from the terminal is received, the output value is obtained by inputting the input value into an AI model for positioning deployed locally by the terminal, a test / monitoring result of the AI model is determined based on the reference value and a test / monitoring value, and the test / monitoring value is the output value;

[0584] The input value is input into an AI model for positioning deployed locally by the network-side device, an output value of the AI model is obtained, and a test / monitoring result of the AI model is determined based on the reference value and a test / monitoring value, and the test / monitoring value is the output value;

[0585] A test / monitoring result of the AI model is determined based on the reference value and a test / monitoring value, and the test / monitoring value is the input value.

[0586] In the embodiments of the present application, by obtaining the reference value and the input value, the verification or monitoring of the AI model for positioning is realized based on the reference value and the input value, thereby improving the reliability of the AI model for positioning in actual environment deployment, and effectively ensuring the performance of AI positioning.

[0587] Optionally, the reference value and the input value include one of the following:

[0588] The reference value is reference position information, and the input value is measurement information or model input information obtained after processing the measurement information.

[0589] The reference value is a reference intermediate quantity, the reference intermediate quantity is intermediate information used to determine the terminal position, the reference intermediate quantity is of the same type as the output value of the AI model, and the input value is measurement information or model input information obtained after processing the measurement information.

[0590] The reference value belongs to a reference output data set, and the input value belongs to a reference input data set.

[0591] The reference value belongs to a reference input data set, and the input value is measurement information or model input information obtained after processing the measurement information.

[0592] Optionally, the reference position information and the input value are obtained in the following manner:

[0593] The position information of a first communication device with known position is selected as the reference position information, and the input value is measurement information corresponding to the first communication device or model input information obtained after processing the measurement information corresponding to the first communication device.

[0594] Optionally, the reference intermediate quantity and the input value are obtained in one of the following manners:

[0595] The reference position information and the measurement information reported by a first communication device with known position are received, the reference intermediate quantity is determined based on the reference position information and second information used to assist in determining the reference intermediate quantity, and the measurement information or model input information obtained after processing the measurement information is taken as the input value.

[0596] The reference intermediate quantity and the input value determined and reported by a first communication device with known position are received, and the input value is measurement information of the first communication device or model input information obtained after processing the measurement information of the first communication device.

[0597] Optionally, the second information comprises at least one of: a current location of a transmission and reception point (TRP) participating in AI positioning, historical data of measurement information, historical data of the AI model output value, a historical location of the terminal, a historical location of the TPR, a historical location of the first communication device.

[0598] Optionally, the reference intermediate quantity comprises at least one of: a line-of-sight / non-line-of-sight indication; a reference signal time difference (RSTD); a relative time of arrival (RTOA); a receive-transmit (Rx-Tx) time difference; a time of arrival (TOA); an angle of arrival (AoA); an angle of departure (AoD); a reference signal received power (RSRP); a reference signal received path power (RSRPP).

[0599] Optionally, the measurement information comprises at least one of: power, time delay, phase, channel impulse response (CIR), power delay profile (PDP), delay profile (DP).

[0600] Optionally, the reference output data set or the reference input data set is autonomously generated by the terminal, or autonomously generated by the network side device, or generated by the first device, or predefined by a protocol.

[0601] Optionally, the determination of the test / monitoring result of the AI model based on the reference value and the test / monitoring value comprises:

[0602] calculating a difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0603] determining the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0604] wherein the first number is greater than or equal to 1.

[0605] Optionally, the calculation of the difference between the reference value and the test / monitoring value of the first number of paths or samples comprises at least one of:

[0606] calculating an Euclidean distance between a vector composed of the reference value and a vector composed of the test / monitoring value of the first number of paths or samples;

[0607] calculating a cosine similarity between a vector composed of the reference value and a vector composed of the test / monitoring value of the first number of paths or samples;

[0608] calculating a root mean square error between a vector composed of the reference value and a vector composed of the test / monitoring value of the first number of paths or samples;

[0609] calculating a difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0610] calculating an average of absolute values of differences between the reference values and the test / monitoring values of the first quantity of paths or samples;

[0611] calculating a root mean square error of differences between the reference values and the test / monitoring values of the first quantity of paths or samples;

[0612] calculating a straight line distance between the reference values and the test / monitoring values of the first quantity of paths or samples;

[0613] calculating an average of straight line distances between the reference values and the test / monitoring values of the first quantity of paths or samples;

[0614] calculating a root mean square of straight line distances between the reference values and the test / monitoring values of the first quantity of paths or samples.

[0615] Optionally, determining the test / monitoring result of the AI model based on the differences between the reference values and the test / monitoring values of the first quantity of paths or samples comprises:

[0616] determining the test / monitoring result of the AI model as applicable in a case where the differences between the reference values and the test / monitoring values of the first quantity of paths or samples satisfy a predefined accuracy requirement.

[0617] determining the test / monitoring result of the AI model as inapplicable in a case where the differences between the reference values and the test / monitoring values of the first quantity of paths or samples do not satisfy the predefined accuracy requirement.

[0618] Optionally, the predefined accuracy requirement is applicable to a specified condition, the specified condition comprising at least one of: a signal-to-interference-plus-noise ratio, a signal-to-noise ratio, a channel strength, a noise strength, a channel model, a reference signal bandwidth, a reference signal configuration, a subcarrier spacing, a carrier frequency, band information, a duplexing mode, a TRP configuration, a UE configuration, an AI model structure, an AI model complexity, AI quantization information.

[0619] Optionally, in a case where the reference value is a reference intermediate quantity and the reference intermediate quantity is a line-of-sight / non-line-of-sight indication, a first probability under the specified condition is taken as a reference accuracy.

[0620] The differences between the reference values and the test / monitoring values of the first quantity of paths or samples satisfying the predefined accuracy requirement comprises:

[0621] calculating a second probability of the reference values and the test / monitoring values being different under the first quantity of paths or samples;

[0622] in case that the first probability represents a maximum mismatch probability of the line-of-sight / non-line-of-sight indication, and the second probability is less than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement;

[0623] or,

[0624] a third probability that the reference value and the test / monitoring value under the first number of paths or samples are the same is calculated;

[0625] in case that the first probability represents a minimum match probability of the line-of-sight / non-line-of-sight indication, and the third probability is greater than or equal to the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first number of paths or samples meets the predefined accuracy requirement.

[0626] Optionally, in case that the reference value is a reference input data set, and the test / monitoring value is the input value,

[0627] the calculation of the difference between the reference value and the test / monitoring value of the first number of paths or samples comprises one of the following:

[0628] in case that the input value is power or time delay or phase of the first number of paths or samples, the difference between the input value and the reference value of the same type in the reference input data set is calculated;

[0629] in case that the input value is a first vector composed of power or time delay or phase of the first number of paths or samples, the difference between the first vector and a second vector composed of reference values of the same type in the reference input data set is calculated;

[0630] in case that the input value is a combination of power, time delay and phase of the first number of paths or samples or any two of them, the difference between the input value and the reference value of the same type in the reference input data set is calculated;

[0631] in case that the input value is a third vector composed of power, time delay and phase of the first number of paths or samples or any two of them, the difference between the third vector and a fourth vector composed of reference values of the same type in the reference input data set is calculated.

[0632] Optionally, the apparatus further comprises a second processing unit for:

[0633] receiving a test / monitoring result from a terminal, the test / monitoring result being not applicable;

[0634] sending first indication information to the terminal, the first indication information being used to instruct the terminal to perform a second operation;

[0635] The second operation includes at least one of:

[0636] updating the current model;

[0637] fine-tuning the current model;

[0638] retraining the current model;

[0639] deactivating the current model;

[0640] the terminal falling back to a non-AI mode;

[0641] restarting the current AI positioning-related measurement;

[0642] stopping the current AI positioning-related measurement;

[0643] extending the current AI positioning-related measurement.

[0644] Optionally, the apparatus further includes a third processing unit configured to:

[0645] in a case where the test / monitoring result is not applicable, sending second indication information to the terminal, the second indication information being used to instruct the terminal to perform a fourth operation;

[0646] The fourth operation includes one of restarting the current AI positioning-related measurement, stopping the current AI positioning-related measurement, and extending the current AI positioning-related measurement.

[0647] Optionally, the AI model includes a first model, or an optimized model of the first model.

[0648] The first model includes at least one of:

[0649] a first model generated based on at least one of a protocol-standardized model structure, protocol-standardized model parameters, and a protocol-standardized data set for training a model;

[0650] a first model determined based on a protocol-standardized model structure and received model parameters;

[0651] a received first model, the first model including at least part of a model structure and / or part of model parameters;

[0652] a first model generated or determined based on a first data set, the first data set being a protocol-standardized data set or a data set autonomously generated by a network-side device;

[0653] a first model generated or determined based on a second data set and a standardized model structure, the second data set being a protocol-standardized data set or a data set autonomously generated by a network-side device or a test device.

[0654] The test or monitoring apparatus for positioning AI model in the embodiments of the present application can be an electronic device, for example, an electronic device with an operating system, or a component in an electronic device, for example, an integrated circuit or a chip. The electronic device can be a network side device, or other devices other than the network side device. Exemplarily, the terminal can include but is not limited to the types of the network side device 12 listed above, and the other devices can be a server, a network attached storage (NAS), etc., which are not limited in the embodiments of the present application.

[0655] The test or monitoring apparatus for positioning AI model provided in the embodiments of the present application can realize the various processes realized by the method embodiment of FIG. 3 and achieve the same technical effects. To avoid repetition, the details are not described here.

[0656] As shown in FIG. 6, the embodiments of the present application further provide a communication device 600, which includes a processor 601 and a memory 602, and the memory 602 stores programs or instructions executable on the processor 601. For example, when the communication device 600 is a terminal, the programs or instructions are executed by the processor 601 to realize the various steps of the above-mentioned test or monitoring method for positioning AI model embodiments, and achieve the same technical effects. When the communication device 600 is a network side device, the programs or instructions are executed by the processor 601 to realize the various steps of the above-mentioned test or monitoring method for positioning AI model embodiments, and achieve the same technical effects. To avoid repetition, the details are not described here.

[0657] The embodiments of the present application further provide a terminal including a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is configured to run programs or instructions to realize the steps in the method embodiment shown in FIG. 2. The terminal embodiment corresponds to the above-mentioned terminal side method embodiment, and each implementation process and implementation manner of the above-mentioned method embodiment can be applicable to the terminal embodiment, and can achieve the same technical effects. Specifically, FIG. 7 is a schematic diagram of the hardware structure of a terminal for realizing the embodiments of the present application.

[0658] The terminal 700 includes but is not limited to at least some of the radio frequency unit 701, the network module 702, the audio output unit 703, the input unit 704, the sensor 705, the display unit 706, the user input unit 707, the interface unit 708, the memory 709, and the processor 710, etc.

[0659] Those skilled in the art can understand that the terminal 700 can further include a power supply (such as a battery) for supplying power to each component, and the power supply can be logically connected to the processor 710 through a power management system, so that the power management system can realize the functions of managing charging, discharging and power consumption management. The terminal structure shown in FIG. 7 does not constitute a limitation on the terminal, and the terminal can include more or fewer components than those shown, or combine certain components, or different component arrangements, which are not described here.

[0660] It should be understood that in the embodiments of the present application, the input unit 704 can include a graphics processing unit (GPU) 7041 and a microphone 7042. The graphics processor 7041 processes image data of a still picture or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 706 can include a display panel 7061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 707 includes at least one of a touch panel 7071 and other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 can include two parts of a touch detection device and a touch controller. The other input devices 7072 can include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), trackballs, mice, joysticks, which are not described here.

[0661] In the embodiments of the present application, after the radio frequency unit 701 receives the downlink data from the network side device, it can be transmitted to the processor 710 for processing. In addition, the radio frequency unit 701 can send uplink data to the network side device. Generally, the radio frequency unit 701 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc.

[0662] The memory 709 can be used to store software programs or instructions and various data. The memory 709 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory 709 can include a volatile memory or a non-volatile memory. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 709 in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0663] The processor 710 can include one or more processing units; optionally, the processor 710 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 710.

[0664] The processor 710 is configured to obtain a reference value and an input value; and perform a first operation, the first operation including one of the following: inputting the input value into an AI model for positioning deployed locally in the terminal, obtaining an output value of the AI model, taking the output value as a test / monitoring value, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value; and taking the input value as a test / monitoring value, and determining a test / monitoring result of the AI model based on the reference value and the test / monitoring value.

[0665] In the embodiment of the present application, by obtaining the reference value and the input value, inputting the input value into the AI model for positioning deployed locally in the terminal, obtaining the output value of the AI model, taking the output value as the test / monitoring value, or taking the input value as the test / monitoring value, determining the test / monitoring result of the AI model based on the reference value and the test / monitoring value, the verification or monitoring of the AI model for positioning is realized, thereby improving the reliability of the AI model for positioning deployed in the actual environment, and the performance of AI positioning can be effectively guaranteed.

[0666] Optionally, the reference value and the input value include one of the following:

[0667] The reference value is reference position information, and the input value is measurement information or model input information obtained after processing the measurement information.

[0668] The reference value is a reference intermediate quantity, the reference intermediate quantity is intermediate information used to determine the position of the terminal, the reference intermediate quantity is of the same type as the output value of the AI model, and the input value is measurement information or model input information obtained after processing the measurement information.

[0669] The reference value belongs to a reference output data set, and the input value belongs to a reference input data set.

[0670] The reference value belongs to a reference input data set, and the input value is measurement information or model input information obtained after processing the measurement information.

[0671] Optionally, the reference position information and the input value are obtained in the following manner:

[0672] The terminal receives first information from the network side device, the first information includes the reference position information and the input value, the reference position information is the position information of a first communication device with known position selected by the network side device, and the input value is measurement information of the first communication device or model input information obtained after processing the measurement information of the first communication device.

[0673] Optionally, the reference intermediate quantity and the input value are obtained in the following manner:

[0674] The reference intermediate quantity and the input value are received from the network side device.

[0675] The reference intermediate quantity is obtained based on reference position information reported by a first communication device with known position and second information used to assist in determining the reference intermediate quantity, or the reference intermediate quantity is determined by the first communication device and reported to the network side device.

[0676] The input value is measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

[0677] Optionally, the second information includes at least one of the following: a current transmission and reception point (TRP) location participating in AI positioning, historical data of measurement information, historical data of the AI model output value, a historical location of the terminal, a historical location of the TRP, and a historical location of the first communication device.

[0678] Optionally, the reference intermediate quantity includes at least one of the following: a line-of-sight / non-line-of-sight indication; a reference signal time difference (RSTD); a relative time of arrival (RTOA); a receive-transmit (Rx-Tx) time difference; a time of arrival (TOA); an angle of arrival (AoA); an angle of departure (AoD); a reference signal received power (RSRP); and a reference signal received path power (RSRPP).

[0679] Optionally, the measurement information includes at least one of the following: power, time delay, phase, channel impulse response (CIR), power delay profile (PDP), and delay profile (DP).

[0680] Optionally, the reference output data set or the reference input data set is autonomously generated by the terminal, autonomously generated by the network side device, generated by the first device, or predefined by a protocol.

[0681] Optionally, the determination of the test / monitoring result of the AI model based on the reference value and the test / monitoring value includes:

[0682] calculating a difference between the reference value and the test / monitoring value of a first number of paths or samples;

[0683] determining the test / monitoring result of the AI model based on the difference between the reference value and the test / monitoring value of the first number of paths or samples;

[0684] wherein the first number is greater than or equal to 1.

[0685] Optionally, the calculation of the difference between the reference value and the test / monitoring value of the first number of paths or samples includes at least one of the following:

[0686] calculating an Euclidean distance between a vector composed of the reference value and a vector composed of the test / monitoring value of the first number of paths or samples;

[0687] calculating a cosine similarity between a vector composed of the reference value and a vector composed of the test / monitoring value of the first number of paths or samples;

[0688] calculating a root mean square error between a vector of reference values of a first number of paths or samples and a vector of test / monitoring values;

[0689] calculating a difference between reference values of a first number of paths or samples and test / monitoring values;

[0690] calculating an average of absolute values of a difference between reference values of a first number of paths or samples and test / monitoring values;

[0691] calculating a root mean square error of a difference between reference values of a first number of paths or samples and test / monitoring values;

[0692] calculating a linear distance between reference values of a first number of paths or samples and test / monitoring values;

[0693] calculating an average of linear distances between reference values of a first number of paths or samples and test / monitoring values;

[0694] calculating a root mean square of linear distances between reference values of a first number of paths or samples and test / monitoring values.

[0695] Optionally, determining the test / monitoring result of the AI model based on the difference between the reference values and the test / monitoring values of the first number of paths or samples comprises:

[0696] determining the test / monitoring result of the AI model as applicable in a case where the difference between the reference values and the test / monitoring values of the first number of paths or samples meets a predefined accuracy requirement;

[0697] determining the test / monitoring result of the AI model as inapplicable in a case where the difference between the reference values and the test / monitoring values of the first number of paths or samples does not meet the predefined accuracy requirement.

[0698] Optionally, the predefined accuracy requirement is applicable to a specified condition, the specified condition comprising at least one of: a signal-to-interference-and-noise ratio, a signal-to-noise ratio, a channel strength, a noise strength, a channel model, a reference signal bandwidth, a reference signal configuration, a subcarrier spacing, a carrier frequency, band information, a duplexing mode, a TRP configuration, a UE configuration, an AI model structure, an AI model complexity, AI quantization information.

[0699] Optionally, in a case where the reference value is a reference intermediate quantity and the reference intermediate quantity is a line-of-sight / non-line-of-sight indication, a first probability under the specified condition is taken as a reference accuracy;

[0700] the difference between the reference values and the test / monitoring values of the first number of paths or samples meeting the predefined accuracy requirement comprises:

[0701] calculating a second probability that the reference value and the test / monitoring value are different under the first number of paths or samples;

[0702] in a case that the first probability represents a maximum mismatch probability of the line-of-sight / non-line-of-sight indication, and the second probability is less than or equal to the first probability, determining that the difference between the reference value and the test / monitoring value under the first number of paths or samples satisfies the predefined accuracy requirement;

[0703] or,

[0704] calculating a third probability that the reference value and the test / monitoring value are the same under the first number of paths or samples;

[0705] in a case that the first probability represents a minimum match probability of the line-of-sight / non-line-of-sight indication, and the third probability is greater than or equal to the first probability, determining that the difference between the reference value and the test / monitoring value under the first number of paths or samples satisfies the predefined accuracy requirement.

[0706] Optionally, in a case that the reference value belongs to a reference input data set, and the test / monitoring value is the input value,

[0707] the calculating the difference between the reference value and the test / monitoring value under the first number of paths or samples comprises one of:

[0708] in a case that the input value is power or time delay or phase of the first number of paths or samples, calculating the difference between the input value and a reference value of the same type in the reference input data set;

[0709] in a case that the input value is a first vector composed of power or time delay or phase of the first number of paths or samples, calculating the difference between the first vector and a second vector composed of reference values of the same type in the reference input data set;

[0710] in a case that the input value is a combination of power, time delay and phase of the first number of paths or samples or any two of them, calculating the difference between the input value and a reference value of the same type in the reference input data set;

[0711] in a case that the input value is a third vector composed of power, time delay and phase of the first number of paths or samples or any two of them, calculating the difference between the third vector and a fourth vector composed of reference values of the same type in the reference input data set.

[0712] Optionally, the processor 710 is further configured to:

[0713] in a case that the test / monitoring result is not applicable, sending the test / monitoring result to the network side device;

[0714] receiving first indication information from the network-side device, the first indication information being used for instructing the terminal to perform a second operation;

[0715] The second operation includes at least one of the following:

[0716] updating the current model;

[0717] fine-tuning the current model;

[0718] retraining the current model;

[0719] deactivating the current model;

[0720] the terminal falling back to a non-AI mode;

[0721] restarting the current AI positioning-related measurement;

[0722] stopping the current AI positioning-related measurement;

[0723] extending the current AI positioning-related measurement.

[0724] Optionally, the AI model includes a first model, or an optimized model of the first model.

[0725] The first model includes at least one of the following:

[0726] a first model generated based on at least one of a protocol-standardized model structure, a protocol-standardized model parameter, and a protocol-standardized data set for training a model;

[0727] a first model determined based on a protocol-standardized model structure and a model parameter received by the terminal;

[0728] a received first model, the first model including at least part of a model structure and / or part of a model parameter;

[0729] a first model generated or determined based on a first data set, the first data set being a protocol-standardized data set or a data set sent by the network-side device to the terminal;

[0730] a first model generated or determined by the terminal based on a second data set and a protocol-standardized model structure, the second data set being a protocol-standardized data set or a data set sent by a peer device of the terminal to the terminal.

[0731] It can be understood that the implementation processes of the implementation manners mentioned in the embodiment can refer to the related descriptions of the foregoing terminal-side method embodiments and achieve the same or corresponding technical effects. To avoid repetition, they will not be described here again.

[0732] The embodiment of the present application further provides a network side device, comprising a processor and a communication interface, the communication interface is coupled with the processor, and the processor is used to run programs or instructions to realize the steps of the method embodiment shown in FIG. 3. The network side device embodiment corresponds to the network side device method embodiment described above, and each implementation process and implementation manner of the method embodiment can be applied to the network side device embodiment and can achieve the same technical effects.

[0733] Specifically, the embodiment of the present application further provides a network side device. As shown in FIG. 8, the network side device 800 comprises a processor 801, a network interface 802 and a memory 803. The network interface 802 is, for example, a common public radio interface (CPRI).

[0734] Specifically, the network side device 800 of the embodiment of the present application further comprises instructions or programs stored in the memory 803 and executable on the processor 801, and the processor 801 invokes the instructions or programs in the memory 803 to execute the method performed by each module shown in FIG. 5 and achieve the same technical effects. To avoid repetition, the details are not described here.

[0735] Specifically, the embodiment of the present application further provides a network side device. As shown in FIG. 9, the network side device 900 comprises an antenna 91, a radio frequency device 92, a baseband device 93, a processor 94 and a memory 95. The antenna 91 is connected with the radio frequency device 92. In the uplink direction, the radio frequency device 92 receives information through the antenna 91 and sends the received information to the baseband device 93 for processing. In the downlink direction, the baseband device 93 processes the information to be sent and sends it to the radio frequency device 92, and the radio frequency device 92 processes the received information and sends it out through the antenna 91.

[0736] The method performed by the network side device in the above embodiment can be implemented in the baseband device 93, which comprises a baseband processor.

[0737] The baseband device 93 may, for example, comprise at least one baseband board provided with a plurality of chips, as shown in FIG. 9, one of which is, for example, a baseband processor connected with the memory 95 through a bus interface to invoke the programs in the memory 95 and execute the network device operations shown in the above method embodiments.

[0738] The network side device may further comprise a network interface 96, which is, for example, a common public radio interface (CPRI).

[0739] Specifically, the network side device 900 of the embodiment of the application further includes instructions or programs stored on the storage 95 and executable on the processor 94, the processor 94 invokes the instructions or programs in the storage 95 to execute the method performed by each module shown in FIG. 5, and achieves the same technical effect. To avoid repetition, details are not described herein.

[0740] The embodiment of the application further provides a readable storage medium, the readable storage medium stores programs or instructions, the programs or instructions are executed by a processor to implement each process of the above-mentioned test or monitoring method for positioning artificial intelligence AI model, and the same technical effect can be achieved. To avoid repetition, details are not described herein.

[0741] The processor is the processor in the terminal in the above-mentioned embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc. In some examples, the readable storage medium can be a non-transitory readable storage medium.

[0742] The embodiment of the application further provides a chip, the chip includes a processor and a communication interface, the communication interface and the processor are coupled, the processor is used to run programs or instructions to implement each process of the above-mentioned test or monitoring method for positioning artificial intelligence AI model, and the same technical effect can be achieved. To avoid repetition, details are not described herein.

[0743] It should be understood that the chip mentioned in the embodiment of the application can also be referred to as a system chip, a system chip, a chip system or a system on chip, etc.

[0744] The embodiment of the application further provides a computer program / program product, the computer program / program product is stored in a storage medium, the computer program / program product is executed by at least one processor to implement each process of the above-mentioned test or monitoring method for positioning artificial intelligence AI model, and the same technical effect can be achieved. To avoid repetition, details are not described herein.

[0745] The embodiment of the application further provides a test or monitoring system for positioning artificial intelligence AI model, including a terminal and a network side device, the terminal can be used to execute the steps of the above-mentioned test or monitoring method for positioning artificial intelligence AI model, and the network side device can be used to execute the steps of the above-mentioned test or monitoring method for positioning artificial intelligence AI model.

[0746] It should be noted that, in the present document, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a", "comprising", or the like does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element. Furthermore, it is to be understood that the methods and apparatuses of the present application can be carried out by specific hardware, by software, or by a combination of hardware and software. It is therefore, contemplated to this patent to cover any and all modifications, variations, or equivalents that fall within the scope of the present application. Accordingly, where a concept can have been illustrated in only one of the exemplary embodiments, various aspects of the concept can be modified and / or combined to produce a variety of other embodiments that are not specifically illustrated. Thus, for purposes of describing the present application, certain aspects of the application can be presented in terms of sequences of actions, but it should be appreciated that these sequences are examples and are not limiting. The sequences of actions could be changed, and other sequences could be implemented. Moreover, it should be appreciated that sometimes it is easier to describe one aspect of the application in terms of another aspect of the application. Therefore, the description herein of one aspect of the application in terms of another aspect of the application is used merely to more particularly exemplify the application. Further, many of the details, functions, and operations can be implemented differently without departing from the basic concept of the present application. Thus, various embodiments of the application can be practiced without the specific details (e.g., the numerical values, the exact construction of the apparatus, the precise phrases for precluding equivalents, etc.) that are presented herein.

[0747] From the above description of the embodiments, it is manifest that the above-described methods of the embodiments can be realized by means of a computer software product and general-purpose hardware platforms by using the computer software product, of course, also by means of hardware. The computer software product is stored in a storage medium (such as a ROM, a RAM, a magnetic disk, an optical disk, etc.), and includes a plurality of instructions for causing a terminal or a network-side device to execute the methods described in the embodiments of the present application.

[0748] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are merely illustrative, but are not restrictive. A person of ordinary skill in the art can make many forms of embodiments under the inspiration of the present application without departing from the scope of the present application and the scope of protection of the claims, and these embodiments all belong to the protection scope of the present application.

Claims

1. A method for testing or monitoring an artificial intelligence (AI) model for positioning, comprising: The terminal obtains the reference value and the input value; The terminal performs a first operation, where the first operation includes one of the following: Inputting the input value into an AI model for positioning deployed locally on the terminal, obtaining an output value of the AI ​​model, using the output value as a test / monitoring value, and determining a test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value; The input value is used as a test / monitoring value, and a test / monitoring result of the AI ​​model is determined based on the reference value and the test / monitoring value.

2. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 1, wherein: The reference value and input value include one of the following: The reference value is reference position information, and the input value is measurement information or model input information obtained by processing the measurement information; The reference value is a reference intermediate quantity, which is intermediate information used to determine the terminal position. The reference intermediate quantity is of the same type as the output value of the AI ​​model. The input value is measurement information or model input information obtained by processing the measurement information. The reference values ​​belong to a reference output data set, and the input values ​​belong to a reference input data set; The reference value belongs to a reference input data set, and the input value is measurement information or model input information obtained by processing the measurement information.

3. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 2, wherein: The method for obtaining the reference position information and the input value includes: The terminal receives first information from a network side device, where the first information includes the reference location information and an input value. The reference location information is location information of a first communication device with a known location selected by the network side device, and the input value is measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

4. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 2, wherein: The method for obtaining the reference intermediate quantity and the input value includes: Receive reference intermediate quantities and input values ​​from network-side devices; The reference intermediate quantity is obtained based on reference location information reported by a first communication device whose location is known and second information, where the second information is used to assist in determining the reference intermediate quantity, or the reference intermediate quantity is determined by the first communication device and reported to the network-side device; The input value is measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

5. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 4, wherein: The second information includes at least one of the following: the TRP position of the transmitting and receiving point currently participating in AI positioning, historical data of measurement information, historical data of the AI ​​model output value, the historical position of the terminal, the historical position of the TRP, and the historical position of the first communication device.

6. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 2, 4 or 5, wherein: The reference intermediate quantity includes at least one of the following: line-of-sight / non-line-of-sight indication; reference signal time difference RSTD; relative time of arrival RTOA; receive-transmit Rx-Tx time difference; arrival time TOA; arrival angle AoA; departure angle AoD; reference signal received power RSRP; reference signal received path power RSRPP.

7. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to any one of claims 2 to 6, wherein: The measurement information includes at least one of the following: power, delay, phase, channel impulse response CIR, power delay profile PDP, and delay profile DP.

8. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 2, wherein: The reference output data set or the reference input data set is autonomously generated by the terminal, or autonomously generated by a network-side device, or generated by the first device, or predefined by a protocol.

9. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to any one of claims 1 to 8, wherein: Determining the test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value includes: calculating a difference between the reference value and the test / monitoring value for a first quantity diameter or sample; determining a test / monitoring result of the AI ​​model based on a difference between the reference value and the test / monitoring value of the first quantity diameter or sample; The first number is greater than or equal to 1.

10. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 9, wherein: Calculating the difference between the reference value and the test / monitoring value of the first quantity diameter or sample comprises at least one of the following: Calculating the Euclidean distance between a vector of reference values ​​and a vector of test / monitoring values ​​for a first quantity path or sample; Calculating the cosine similarity between the vector consisting of the reference values ​​and the vector consisting of the test / monitoring values ​​of the first quantity path or sample; Calculating the root mean square error between a vector of reference values ​​and a vector of test / monitoring values ​​for a first quantity diameter or sample; calculating a difference between a reference value and a test / monitoring value for a first quantity diameter or sample; calculating the average of the sum of the absolute values ​​of the differences between the reference value and the test / monitoring value for the first quantity diameter or sample; calculating a root mean square error of the difference between the reference value and the test / monitoring value for the first quantity diameter or sample; calculating a first quantity diameter or a straight-line distance between a reference value and a test / monitoring value for the sample; calculating an average of the first number diameters or straight-line distances between the reference value and the test / monitoring value of the sample; Calculate the root mean square of the straight-line distance between the reference value and the test / monitor value for a first quantity diameter or sample.

11. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 9 or 10, wherein: The determining of the test / monitoring result of the AI ​​model based on the difference between the reference value and the test / monitoring value of the first quantity diameter or sample includes: If the difference between the reference value and the test / monitoring value of the first quantity diameter or sample meets a predefined accuracy requirement, determining the test / monitoring result of the AI ​​model as applicable; In a case where the difference between the reference value and the test / monitoring value of the first quantity diameter or sample does not meet a predefined accuracy requirement, the test / monitoring result of the AI ​​model is determined to be not applicable.

12. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 11, wherein: The predefined accuracy requirement applies to specified conditions, and the specified conditions include at least one of the following: signal-to-interference-plus-noise ratio, signal-to-noise ratio, channel strength, noise strength, channel model, reference signal bandwidth, reference signal configuration, subcarrier spacing, carrier frequency, frequency band information, duplex system, TRP configuration, UE configuration, AI model structure, AI model complexity, and AI quantization information.

13. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 12, wherein: In a case where the reference value is a reference intermediate quantity and the reference intermediate quantity is a line-of-sight / non-line-of-sight indication, taking the first probability under the specified condition as a reference accuracy; The difference between the reference value and the test / monitoring value of the first quantity diameter or sample satisfies a predefined accuracy requirement including: calculating a second probability that the reference value and the test / monitoring value for the first quantity size or sample are different; determining that a difference between a reference value and a test / monitoring value of the first quantity diameter or sample satisfies a predefined accuracy requirement when the first probability represents a maximum mismatch probability of a line-of-sight / non-line-of-sight indication and the second probability is less than or equal to the first probability; or, calculating a third probability that the reference value and the test / monitoring value are the same for the first quantity size or sample; When the first probability represents a minimum matching probability of line-of-sight / non-line-of-sight indication and the third probability is greater than or greater than the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first quantity diameter or sample meets the predefined accuracy requirement.

14. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 9, wherein: In the case where the reference value belongs to a reference input data set and the test / monitoring value is the input value, Calculating the difference between the reference value and the test / monitoring value of the first quantity diameter or sample comprises one of the following: In a case where the input value is a power or a time delay or a phase of a first quantity path or sample, calculating a difference between the input value and a reference value of the same type in the reference input data set; In a case where the input value is a first vector consisting of power or delay or phase of a first quantity path or sample, calculating the difference between the first vector and a second vector consisting of reference values ​​of the same type in the reference input data set; When the input value is a power, a delay, and a phase of a first quantity path or sample, or a combination of any two thereof, calculating a difference between the input value and a reference value of the same type in the reference input data set; When the input value is a third vector consisting of power, delay and phase of a first quantity path or sample, or any two of them, a difference between the third vector and a fourth vector consisting of reference values ​​of the same type in the reference input data set is calculated.

15. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to any one of claims 1 to 14, wherein: The method further comprises: If the test / monitoring result is not applicable, the terminal sends the test / monitoring result to the network side device; The terminal receives first instruction information from the network-side device, where the first instruction information is used to instruct the terminal to perform a second operation; The second operation includes at least one of the following: Update the current model; Fine-tune the current model; Retrain the current model; Deactivate the current model; The terminal falls back to non-AI mode; Restart the current AI positioning related measurements; Stop the current AI positioning related measurements; Extend the current AI positioning related measurements.

16. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to any one of claims 1 to 15, wherein: The AI ​​model includes a first model, or an optimized model of the first model; The first model includes at least one of the following: a first model generated based on at least one of a protocol-standardized model structure, protocol-standardized model parameters, and a protocol-standardized dataset for training the model; A first model determined based on a model structure standardized by a protocol and model parameters received by the terminal; a received first model, the first model comprising at least a portion of a model structure and / or a portion of model parameters; a first model generated or determined based on a first data set, where the first data set is a protocol-standardized data set or a data set sent to the terminal by a network-side device; The terminal generates or determines the first model based on a second data set and a protocol standardized model structure, where the second data set is a protocol standardized data set or a data set sent to the terminal by a peer device of the terminal.

17. A method for testing or monitoring an artificial intelligence (AI) model for positioning, comprising: The network side device obtains the reference value and input value; The network-side device performs a third operation, where the third operation includes one of the following: Sending the reference value and the input value to the terminal, where the reference value and the input value are used to determine a test / monitoring result of an AI model for positioning deployed locally on the terminal; Sending the input value to a terminal and receiving an output value from the terminal, wherein the output value is obtained by inputting the input value into an AI model for positioning deployed locally on the terminal, and determining a test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value, wherein the test / monitoring value is the output value; Inputting the input value into an AI model for positioning deployed locally on the network-side device, obtaining an output value of the AI ​​model, and determining a test / monitoring result of the AI ​​model based on the reference value and a test / monitoring value, where the test / monitoring value is the output value; Based on the reference value and the test / monitoring value, a test / monitoring result of the AI ​​model is determined, wherein the test / monitoring value is the input value.

18. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 17, wherein: The reference value and input value include one of the following: The reference value is reference position information, and the input value is measurement information or model input information obtained by processing the measurement information; The reference value is a reference intermediate quantity, which is intermediate information used to determine the terminal position. The reference intermediate quantity is of the same type as the output value of the AI ​​model. The input value is measurement information or model input information obtained by processing the measurement information. The reference values ​​belong to a reference output data set, and the input values ​​belong to a reference input data set; The reference value belongs to a reference input data set, and the input value is measurement information or model input information obtained by processing the measurement information.

19. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 18, wherein: The method for obtaining the reference position information and the input value includes: The network side device selects location information of a first communication device with a known location as the reference location information, and the input value is measurement information corresponding to the first communication device or model input information obtained by processing the measurement information corresponding to the first communication device.

20. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 18, wherein: The method for obtaining the reference intermediate quantity and the input value includes one of the following: receiving reference location information and measurement information reported by a first communication device with a known location, determining the reference intermediate quantity based on the reference location information and the second information, wherein the second information is used to assist in determining the reference intermediate quantity, and using the measurement information or model input information obtained by processing the measurement information as the input value; A reference intermediate quantity and an input value determined and reported by a first communication device with a known location are received, where the input value is measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

21. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 20, wherein: The second information includes at least one of the following: the TRP position of the transmitting and receiving point currently participating in AI positioning, historical data of measurement information, historical data of the AI ​​model output value, the historical position of the terminal, the historical position of the TPR, and the historical position of the first communication device.

22. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 18, 20 or 21, wherein: The reference intermediate quantity includes at least one of the following: line-of-sight / non-line-of-sight indication; reference signal time difference RSTD; relative arrival time RTOA; receive-transmit Rx-Tx time difference; arrival time TOA; arrival angle AoA; departure angle AoD; reference signal received power RSRP; reference signal received path power RSRPP.

23. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to any one of claims 17 to 22, wherein: Determining the test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value includes: calculating a difference between the reference value and the test / monitoring value for a first quantity diameter or sample; determining a test / monitoring result of the AI ​​model based on a difference between the reference value and the test / monitoring value of the first quantity diameter or sample; The first number is greater than or equal to 1.

24. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 23, wherein: Calculating the difference between the reference value and the test / monitoring value of the first quantity diameter or sample comprises at least one of the following: Calculating the Euclidean distance between a vector of reference values ​​and a vector of test / monitoring values ​​for a first quantity path or sample; Calculating the cosine similarity between the vector consisting of the reference values ​​and the vector consisting of the test / monitoring values ​​of the first quantity path or sample; Calculating the root mean square error between a vector of reference values ​​and a vector of test / monitoring values ​​for a first quantity diameter or sample; calculating a difference between a reference value and a test / monitoring value for a first quantity diameter or sample; calculating the average of the sum of the absolute values ​​of the differences between the reference value and the test / monitoring value for the first quantity diameter or sample; calculating a root mean square error of the difference between the reference value and the test / monitoring value for the first quantity diameter or sample; calculating a first quantity diameter or a straight-line distance between a reference value and a test / monitoring value for the sample; calculating an average of the first number diameters or straight-line distances between the reference value and the test / monitoring value of the sample; Calculate the root mean square of the straight-line distance between the reference value and the test / monitor value for a first quantity diameter or sample.

25. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 23 or 24, wherein: The determining of the test / monitoring result of the AI ​​model based on the difference between the reference value and the test / monitoring value of the first quantity diameter or sample includes: If the difference between the reference value and the test / monitoring value of the first quantity diameter or sample meets a predefined accuracy requirement, determining the test / monitoring result of the AI ​​model as applicable; In a case where the difference between the reference value and the test / monitoring value of the first quantity diameter or sample does not meet a predefined accuracy requirement, the test / monitoring result of the AI ​​model is determined to be not applicable.

26. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 25, wherein: The predefined accuracy requirement applies to specified conditions, and the specified conditions include at least one of the following: signal-to-interference-plus-noise ratio, signal-to-noise ratio, channel strength, noise strength, channel model, reference signal bandwidth, reference signal configuration, subcarrier spacing, carrier frequency, frequency band information, duplex system, TRP configuration, UE configuration, AI model structure, AI model complexity, and AI quantization information.

27. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 26, wherein: In a case where the reference value is a reference intermediate quantity and the reference intermediate quantity is a line-of-sight / non-line-of-sight indication, taking the first probability under the specified condition as a reference accuracy; The difference between the reference value and the test / monitoring value of the first quantity diameter or sample satisfies a predefined accuracy requirement including: calculating a second probability that the reference value and the test / monitoring value for the first quantity size or sample are different; determining that a difference between a reference value and a test / monitoring value of the first quantity diameter or sample satisfies a predefined accuracy requirement when the first probability represents a maximum mismatch probability of a line-of-sight / non-line-of-sight indication and the second probability is less than or equal to the first probability; or, calculating a third probability that the reference value and the test / monitoring value are the same for the first quantity size or sample; When the first probability represents a minimum matching probability of line-of-sight / non-line-of-sight indication and the third probability is greater than or greater than the first probability, it is determined that the difference between the reference value and the test / monitoring value of the first quantity diameter or sample meets the predefined accuracy requirement.

28. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to claim 23, wherein: In the case where the reference value is a reference input data set and the test / monitoring value is the input value, Calculating the difference between the reference value and the test / monitoring value of the first quantity diameter or sample comprises one of the following: In a case where the input value is a power or a time delay or a phase of a first quantity path or sample, calculating a difference between the input value and a reference value of the same type in the reference input data set; In a case where the input value is a first vector consisting of power or delay or phase of a first quantity path or sample, calculating a difference between the first vector and a second vector consisting of reference values ​​of the same type as the reference input data set; When the input value is a power, a delay, and a phase of a first quantity path or sample, or a combination of any two thereof, calculating a difference between the input value and a reference value of the same type in the reference input data set; When the input value is a third vector consisting of power, delay and phase of a first quantity path or sample, or any two of them, a difference between the third vector and a fourth vector consisting of reference values ​​of the same type in the reference input data set is calculated.

29. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to any one of claims 17 to 22, wherein: The method further comprises: receiving a test / monitoring result from a terminal, wherein the test / monitoring result is not applicable; Sending first instruction information to the terminal, where the first instruction information is used to instruct the terminal to perform a second operation; The second operation includes at least one of the following: Update the current model; Fine-tune the current model; Retrain the current model; Deactivate the current model; The terminal falls back to non-AI mode; Restart the current AI positioning related measurements; Stop the current AI positioning related measurements; Extend the current AI positioning related measurements.

30. The method for testing or monitoring an artificial intelligence (AI) model for positioning according to any one of claims 17 to 29, wherein: The method further comprises: If the test / monitoring result is not applicable, the network-side device sends second indication information to the terminal, where the second indication information is used to instruct the terminal to perform a fourth operation; The fourth operation includes one of the following: restarting the current AI positioning related measurement, stopping the current AI positioning related measurement, and extending the current AI positioning related measurement.

31. A device for testing or monitoring an artificial intelligence (AI) model for positioning, comprising: A first acquiring unit, configured to acquire a reference value and an input value; The first execution unit is configured to execute a first operation, where the first operation includes one of the following: Inputting the input value into an AI model for positioning deployed locally on the terminal, obtaining an output value of the AI ​​model, using the output value as a test / monitoring value, and determining a test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value; The input value is used as a test / monitoring value, and a test / monitoring result of the AI ​​model is determined based on the reference value and the test / monitoring value.

32. The testing or monitoring device for an artificial intelligence (AI) model according to claim 31, wherein: The reference value and input value include one of the following: The reference value is reference position information, and the input value is measurement information or model input information obtained by processing the measurement information; The reference value is a reference intermediate quantity, which is intermediate information used to determine the terminal position. The reference intermediate quantity is of the same type as the output value of the AI ​​model. The input value is measurement information or model input information obtained by processing the measurement information. The reference values ​​belong to a reference output data set, and the input values ​​belong to a reference input data set; The reference value belongs to a reference input data set, and the input value is measurement information or model input information obtained by processing the measurement information.

33. The testing or monitoring device for an artificial intelligence (AI) model for positioning according to claim 32, wherein: The method for obtaining the reference position information and the input value includes: Receive first information from the network side device, the first information including the reference location information and an input value, the reference location information being the location information of a first communication device with a known location selected by the network side device, and the input value being the measurement information of the first communication device or the model input information obtained by processing the measurement information of the first communication device.

34. The testing or monitoring device for an artificial intelligence (AI) model for positioning according to claim 32, wherein: The method for obtaining the reference intermediate quantity and the input value includes: receiving a reference intermediate quantity and an input value from the network-side device; The reference intermediate quantity is obtained based on reference location information reported by a first communication device whose location is known and second information, where the second information is used to assist in determining the reference intermediate quantity, or the reference intermediate quantity is determined by the first communication device and reported to the network-side device; The input value is measurement information of the first communication device or model input information obtained by processing the measurement information of the first communication device.

35. The testing or monitoring device for an artificial intelligence (AI) model for positioning according to claim 32 or 34, wherein: The reference intermediate quantity includes at least one of the following: line-of-sight / non-line-of-sight indication; reference signal time difference RSTD; relative time of arrival RTOA; receive-transmit Rx-Tx time difference; arrival time TOA; arrival angle AoA; departure angle AoD; reference signal received power RSRP; reference signal received path power RSRPP.

36. A testing or monitoring device for an artificial intelligence (AI) model for positioning according to any one of claims 31 to 35, wherein: Determining the test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value includes: calculating a difference between the reference value and the test / monitoring value for a first quantity diameter or sample; determining a test / monitoring result of the AI ​​model based on a difference between the reference value and the test / monitoring value of the first quantity diameter or sample; The first number is greater than or equal to 1.

37. The testing or monitoring device for an artificial intelligence (AI) model for positioning according to claim 36, wherein: Calculating the difference between the reference value and the test / monitoring value of the first quantity diameter or sample comprises at least one of the following: Calculating the Euclidean distance between a vector of reference values ​​and a vector of test / monitoring values ​​for a first quantity path or sample; Calculating the cosine similarity between the vector consisting of the reference values ​​and the vector consisting of the test / monitoring values ​​of the first quantity path or sample; Calculating the root mean square error between a vector of reference values ​​and a vector of test / monitoring values ​​for a first quantity diameter or sample; calculating a difference between a reference value and a test / monitoring value for a first quantity diameter or sample; calculating the average of the sum of the absolute values ​​of the differences between the reference value and the test / monitoring value for the first quantity diameter or sample; calculating a root mean square error of the difference between the reference value and the test / monitoring value for the first quantity diameter or sample; calculating a first quantity diameter or a straight-line distance between a reference value and a test / monitoring value for the sample; calculating an average of the first number diameters or straight-line distances between the reference value and the test / monitoring value of the sample; Calculate the root mean square of the straight-line distance between the reference value and the test / monitor value for a first quantity diameter or sample.

38. A device for testing or monitoring an artificial intelligence (AI) model for positioning, comprising: A second acquiring unit, configured to acquire a reference value and an input value; The second execution unit is configured to execute a third operation, where the third operation includes one of the following: Sending the reference value and the input value to the terminal, where the reference value and the input value are used to determine a test / monitoring result of an AI model for positioning deployed locally on the terminal; Sending the input value to a terminal and receiving an output value from the terminal, wherein the output value is obtained by inputting the input value into an AI model for positioning deployed locally on the terminal, and determining a test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value, wherein the test / monitoring value is the output value; Inputting the input value into an AI model for positioning deployed locally on the network-side device, obtaining an output value of the AI ​​model, and determining a test / monitoring result of the AI ​​model based on the reference value and a test / monitoring value, where the test / monitoring value is the output value; Based on the reference value and the test / monitoring value, a test / monitoring result of the AI ​​model is determined, wherein the test / monitoring value is the input value.

39. The testing or monitoring device for an artificial intelligence (AI) model for positioning according to claim 38, wherein: Determining the test / monitoring result of the AI ​​model based on the reference value and the test / monitoring value includes: calculating a difference between the reference value and the test / monitoring value for a first quantity diameter or sample; determining a test / monitoring result of the AI ​​model based on a difference between the reference value and the test / monitoring value of the first quantity diameter or sample; The first number is greater than or equal to 1.

40. The testing or monitoring device for an artificial intelligence (AI) model for positioning according to claim 38 or 39, wherein: The apparatus further includes a second processing unit, configured to: receiving a test / monitoring result from a terminal, wherein the test / monitoring result is not applicable; Sending first instruction information to the terminal, where the first instruction information is used to instruct the terminal to perform a second operation; The second operation includes at least one of the following: Update the current model; Fine-tune the current model; Retrain the current model; Deactivate the current model; The terminal falls back to non-AI mode; Restart the current AI positioning related measurements; Stop the current AI positioning related measurements; Extend the current AI positioning related measurements.

41. The testing or monitoring device for an artificial intelligence (AI) model for positioning according to claim 38 or 39, wherein: The apparatus further includes a third processing unit, configured to: If the test / monitoring result is not applicable, sending second indication information to the terminal, where the second indication information is used to instruct the terminal to perform a fourth operation; The fourth operation includes one of the following: restarting the current AI positioning related measurement, stopping the current AI positioning related measurement, and extending the current AI positioning related measurement.

42. A terminal comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the testing or monitoring method of the artificial intelligence (AI) model for positioning as described in any one of claims 1 to 16 are implemented.

43. A network-side device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the testing or monitoring method of the artificial intelligence (AI) model for positioning as described in any one of claims 17 to 30 are implemented.

44. A readable storage medium storing a program or instruction, wherein the program or instruction, when executed by a processor, implements the method for testing or monitoring an artificial intelligence (AI) model for positioning as described in any one of claims 1 to 16, or implements the steps of the method for testing or monitoring an artificial intelligence (AI) model for positioning as described in any one of claims 17 to 30.

Images