Method and device for acquiring performance information of positioning model, and network side equipment

Abstract

The invention discloses a method and device for obtaining positioning model performance information and network side equipment, and belongs to the technical field of wireless communication, and the method for obtaining the positioning model performance information comprises the steps that a first network element sends a positioning message to at least one first access network device, the positioning message is used for requesting to report one of the following information: AI measurement information generated by reasoning by using a target positioning model, the AI measurement information and real measurement information, and difference information between the AI measurement information and the real measurement information; the first network element receives a positioning result message sent by at least one first access network device, the positioning result message carries target positioning measurement information, and the target positioning measurement information comprises one of the following information: AI measurement information, AI measurement information, real measurement information, and difference information between the AI measurement information and the real measurement information; and the first network element obtains first positioning performance monitoring information of the target positioning model based on the target positioning measurement information.

Description

Technical Field

[0001] The present application belongs to the field of wireless communication technology, and specifically relates to a method, device and network-side equipment for obtaining positioning model performance information. Background Art

[0002] Artificial Intelligence (AI) positioning aims to use AI positioning models to predict measurement information to give a target location estimate, which can save a lot of measurement resources and reduce computing power consumption.

[0003] In the related art, the AI ​​positioning model can be set in the access network equipment to perform model reasoning to obtain AI measurement information, and report the AI ​​measurement information to the positioning control function network element, for example, the location management function (LMF). The positioning control function network element can perform position estimation based on the received AI measurement information.

[0004] However, in actual applications, the performance of the AI ​​positioning model is unknown, which makes it impossible to determine the accuracy and precision of the estimated results obtained by using the AI ​​measurement information inferred by the AI ​​positioning model for position estimation, and thus it is impossible to know whether the obtained estimation results are accurate. Summary of the invention

[0005] The embodiments of the present application provide a method, apparatus, and network-side equipment for obtaining positioning model performance information, which can solve the problem that the accuracy and precision of the estimation results obtained by using AI measurement information inferred by the AI ​​positioning model for position estimation cannot be determined due to the unknown performance of the AI ​​positioning model.

[0006] In a first aspect, a method for obtaining positioning model performance information is provided, including: a first network element sends a positioning message to at least one first access network device, wherein the positioning message is used to request a report of one of the following: AI measurement information generated by reasoning using the target positioning model, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information; the first network element receives a positioning result message sent by the at least one first access network device, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI ​​measurement information, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information; the first network element obtains first positioning performance monitoring information of the target positioning model based on the target positioning measurement information.

[0007] In a second aspect, a method for sending positioning monitoring information of a positioning model is provided, including: a first access network device receives a positioning message sent by a first network element, wherein the positioning message is used to request a report of one of the following: AI measurement information generated by reasoning using the target positioning model, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information; the first access network device obtains real measurement information and AI measurement information inferred using the target positioning model; the first access network device sends a positioning result message to the first network element based on the real measurement information and the AI ​​measurement information, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI ​​measurement information, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information.

[0008] In a third aspect, a device for acquiring positioning model performance information is provided, including: a first sending module, used to send a positioning message to at least one first access point and a first access network device, wherein the positioning message is used to request a report of one of the following: AI measurement information generated by reasoning using the target positioning model, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information; a first receiving module, used to receive a positioning result message sent by the at least one first access network device, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI ​​measurement information, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information; a first acquisition module, used to acquire first positioning performance monitoring information of the target positioning model based on the target positioning measurement information.

[0009] In a fourth aspect, a positioning monitoring information sending device for a positioning model is provided, including: a second receiving module, used to receive a positioning message sent by a first network element, wherein the positioning message is used to request a report of one of the following: AI measurement information generated by reasoning using the target positioning model, the AI ​​measurement information and real measurement information, and the difference information between the AI ​​measurement information and the real measurement information; a second acquisition module, used to acquire the real measurement information and the AI ​​measurement information inferred using the target positioning model; a second sending module, used to send a positioning result message to the first network element based on the real measurement information and the AI ​​measurement information, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI ​​measurement information, the AI ​​measurement information and real measurement information, and the difference information between the AI ​​measurement information and the real measurement information.

[0010] In a fifth aspect, a network side device is provided, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the second aspect are implemented.

[0011] In the sixth aspect, a network side device is provided, comprising a processor and a communication interface, wherein the processor is used to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the second aspect, and the communication interface is used to couple with the processor.

[0012] In the seventh aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the second aspect are implemented.

[0013] In an eighth aspect, a communication system is provided, comprising: a first network element and an access network device, wherein the first network element can be used to execute the steps of the method described in the first aspect, and the access network device can be used to execute the steps of the method described in the second aspect.

[0014] In the ninth aspect, a chip is provided, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect, or to implement the method described in the second aspect.

[0015] In the tenth aspect, a computer program / program product is provided, wherein the computer program / program product is stored in a storage medium, and the program / program product is executed by at least one processor to implement the method as described in the first aspect, or to implement the method as described in the second aspect.

[0016] In an embodiment of the present application, a first network element sends a positioning message to at least one first access network device, wherein the positioning message is used to request reporting of one of the following: AI measurement information generated by reasoning using the target positioning model, the AI ​​measurement information and real measurement information, and the difference information between the AI ​​measurement information and the real measurement information; the first network element receives a positioning result message sent by the at least one first access network device, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI ​​measurement information, the AI ​​measurement information and real measurement information, and the difference information between the AI ​​measurement information and the real measurement information; the first network element obtains first positioning performance monitoring information of the target positioning model based on the target positioning measurement information. Thus, the first network element can monitor the positioning performance of the target positioning model based on the target positioning measurement information reported by the first access network device, solving the problem that the accuracy and precision of the estimation result obtained by using the AI ​​measurement information inferred by the AI ​​positioning model for position estimation cannot be determined due to the unknown performance of the AI ​​positioning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A block diagram showing a wireless communication system to which the embodiments of the present application can be applied;

[0018] Figure 2 A schematic diagram showing a layout of an AI positioning model in an embodiment of the present application is shown;

[0019] Figure 3 A schematic diagram showing a flow chart of a method for obtaining positioning model performance information provided in an embodiment of the present application;

[0020] Figure 4 A schematic diagram showing a flow chart of sending positioning monitoring information of a positioning model provided in an embodiment of the present application;

[0021] Figure 5 Another schematic diagram showing a flow chart of a method for acquiring positioning model performance information provided in an embodiment of the present application;

[0022] Figure 6 Another schematic diagram showing a flow chart of a method for acquiring positioning model performance information provided in an embodiment of the present application;

[0023] Figure 7 A schematic diagram showing a structure of a device for acquiring positioning model performance information provided in an embodiment of the present application;

[0024] Figure 8 A schematic diagram showing a structure of a positioning monitoring information device of a positioning model provided in an embodiment of the present application;

[0025] Fig. 9 A schematic diagram showing the structure of a communication device provided in an embodiment of the present application is shown;

[0026] Fig.10 A schematic diagram showing the hardware structure of a network side device provided in an embodiment of the present application is shown;

[0027] Fig.11 A schematic diagram of the hardware structure of another network side device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.

[0029] The terms "first", "second", etc. of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of one type, and the number of objects is not limited, for example, the first object can be one or more. In addition, "or" in the present application represents at least one of the connected objects. For example, "A or B" covers three schemes, namely, Scheme 1: including A but not including B; Scheme 2: including B but not including A; Scheme 3: including both A and B. The character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0030] The term "indication" in this application can be a direct indication (or explicit indication) or an indirect indication (or implicit indication). A direct indication can be understood as the sender explicitly informing the receiver of specific information, operations to be performed, or request results in the sent indication; an indirect indication can be understood as the receiver determining the corresponding information according to the indication sent by the sender, or making a judgment and determining the operation to be performed or the request result according to the judgment result.

[0031] It is worth noting that the technology described in the embodiments of the present application is not limited to the Long Term Evolution (LTE) / LTE-Advanced (LTE-A) system, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA) or other systems. The terms "system" and "network" in the embodiments of the present application are often used interchangeably, and the described technology can be used for the systems and radio technologies mentioned above as well as for other systems and radio technologies. The following description describes a New Radio (NR) system for illustrative purposes, and NR terminology is used in most of the following descriptions, but these technologies can also be applied to systems other than NR systems, such as the 6th generation (6 th Generation, 6G) communication system.

[0032] Figure 1A block diagram of a wireless communication system applicable to an embodiment of the present application is shown. The wireless communication system includes a terminal 11 and a network side device 12. Among them, the terminal 11 can be a mobile phone, a tablet computer (Tablet Personal Computer), a laptop computer (Laptop Computer), a notebook computer, a personal digital assistant (Personal Digital Assistant, PDA), a handheld computer, a netbook, an ultra-mobile personal computer (Ultra-mobile Personal Computer, UMPC), a mobile Internet device (Mobile Internet Device, MID), an augmented reality (Augmented Reality, AR), a virtual reality (Virtual Reality, VR) device, a robot, a wearable device (Wearable Device), an aircraft (flight vehicle), a vehicle-mounted device (Vehicle User Equipment, VUE), a ship-mounted device, a pedestrian terminal (Pedestrian User Equipment, PUE), a smart home (home appliances with wireless communication functions, such as refrigerators, televisions, washing machines or furniture, etc.), a game console, a personal computer (Personal Computer, PC), a teller machine or a self-service machine and other terminal side devices. Wearable devices include: smart watches, smart bracelets, smart headphones, smart glasses, smart jewelry (smart bracelets, smart bracelets, smart rings, smart necklaces, smart anklets, smart anklets, etc.), smart wristbands, smart clothing, etc. Among them, the vehicle-mounted device can also be called a vehicle-mounted terminal, a vehicle-mounted controller, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip or a vehicle-mounted unit, etc. It should be noted that the specific type of the terminal 11 is not limited in the embodiment of the present application. The network side device 12 may include an access network device or a core network device, wherein the access network device may also be referred to as a radio access network (Radio Access Network, RAN) device, a radio access network function or a radio access network unit. The access network device may include a base station, a wireless local area network (Wireless Local Area Network, WLAN) access point (Access Point, AS) or a wireless fidelity (Wireless Fidelity, WiFi) node, etc.Among them, the base station may be referred to as a Node B (NB), an evolved Node B (eNB), a next generation Node B (gNB), a New Radio Node B (NR Node B), an access point, a Relay Base Station (RBS), a Serving Base Station (SBS), a Base Transceiver Station (BTS), a radio base station, a radio transceiver, a Basic Service Set (BSS), an Extended Service Set (ESS), a Home Node B (HNB), a Home Evolved Node B, a Transmission Reception Point (TRP) or other appropriate terms in the field. As long as the same technical effect is achieved, the base station is not limited to specific technical terms. It should be noted that in the embodiments of the present application, only the base station in the NR system is used as an example for introduction, and the specific type of the base station is not limited.

[0033] The core network equipment may include but is not limited to at least one of the following: core network node, core network function, mobility management entity (Mobility Management Entity, MME), access mobility management function (Access and Mobility Management Function, AMF), session management function (Session Management Function, SMF), user plane function (User Plane Function, UPF), policy control function (Policy Control Function, PCF), policy and charging rules function unit (Policy and Charging Rules Function, PCRF), edge application service discovery function (Edge Application Server Discovery Function, EASDF), unified data management (Unified Data Management, UDM), unified data storage (Unified Data Repository, UDR), home user server (Home Subscriber Server, HSS), centralized network configuration (CNC), network storage function (Network Repository Function, NRF), network exposure function (Network Exposure Function, NEF), local NEF (Local NEF, or L-NEF), binding support function (Binding Support Function, BSF), application function (Application Function, AF), Location Management Function (LMF), Gateway Mobile Location Center (GMLC), Network Data Analytics Function (NWDAF), etc. It should be noted that in the embodiments of the present application, only the core network equipment in the NR system is introduced as an example, and the specific type of the core network equipment is not limited.

[0034] In related technologies, such as Figure 2As shown, the AI ​​positioning model is set in the base station, the UE sends measurement feedback (for example, a sounding reference signal (SRS)) to the base station, the base station uses the AI ​​positioning model to calculate the AI ​​measurement information, and then sends the AI ​​measurement information to the LMF to realize the position calculation.

[0035] However, the model accuracy of the AI ​​positioning model in actual use may not be high, and the positioning accuracy may also be poor, and the performance of the AI ​​positioning model is unknown, which makes it impossible to determine the accuracy and precision of the estimated results obtained by using the AI ​​measurement information inferred by the AI ​​positioning model for position estimation, and thus it is impossible to know whether the obtained estimation results are accurate.

[0036] To address this problem, the embodiments of the present application provide a method, apparatus, and network-side device for obtaining positioning model performance information. The triggerer can trigger the network to monitor information such as positioning accuracy and model accuracy of a specific AI model. If the reporting conditions are met, the AI ​​model performance result information can be reported to the triggerer.

[0037] The technical solution provided by the embodiments of the present application is described in detail below through some embodiments and their application scenarios in combination with the accompanying drawings.

[0038] Figure 3 A flow chart of a method for obtaining positioning model performance information in an embodiment of the present application is shown, and the method 300 can be performed by a first network element. In other words, the method can be performed by software or hardware installed on a communication device. In an embodiment of the present application, the first network element can be a positioning control function network element, for example, a location management function (LMF).

[0039] like Figure 3 As shown, the method may include the following steps.

[0040] S310. A first network element sends a positioning message to at least one first access network device.

[0041] In this embodiment of the present application, the positioning message is used to request reporting of one of the following:

[0042] (1) AI measurement information generated by reasoning using the target positioning model; the first network element may request at least one first access network device to report the AI ​​measurement information generated by reasoning using the target positioning model through a positioning message. After receiving the positioning message, the first access network device may use the target positioning model for reasoning, thereby obtaining the AI ​​measurement information.

[0043] (2) The AI ​​measurement information and the real measurement information; the first network element may request at least one first access network device to report the AI ​​measurement information and the real measurement information generated by reasoning using the target positioning model through a positioning message. After receiving the positioning message, the first access network device may use the target positioning model for reasoning to obtain the AI ​​measurement information, and adopt a non-AI method, for example, by measuring the positioning reference signal, to obtain the real measurement information.

[0044] (3) Difference information between the AI ​​measurement information and the real measurement information. The first network element may request at least one first access network device to report the difference information between the AI ​​measurement information generated by reasoning using the target positioning model and the real measurement information through a positioning message. After receiving the positioning message, the first access network device may use the target positioning model for reasoning to obtain the AI ​​measurement information, and adopt a non-AI method, for example, by measuring the positioning reference signal to obtain the real measurement information, and then obtain the difference information between the AI ​​measurement information and the real measurement information through comparison.

[0045] Optionally, the positioning message may carry at least one of the following:

[0046] (1) Identification information of the target positioning model; the identification information is used to indicate the target positioning model to be monitored, wherein the target positioning model to be monitored may be one or more, which is not limited in the embodiments of the present application.

[0047] (2) First indication information, used to instruct the first access network device to use the AI ​​measurement information inferred by the target positioning model for positioning; the first indication information may also instruct the first access network device to report the AI ​​measurement information inferred by the first access network device using the target positioning model. For example, the first indication information may be an AI positioning indication (AI positioningind).

[0048] (3) second indication information, used to indicate that the first network element will perform performance monitoring on the target positioning model. The second indication information indicates that the first network element will perform performance monitoring on the target positioning model. The first access network device may determine, based on the second indication information, that it is necessary to report the AI ​​measurement information, the AI ​​measurement information and the real measurement information, and the difference information between the AI ​​measurement information and the real measurement information. For example, the second indication information may be a monitoring indication (Monitoringind).

[0049] In an optional implementation, the first network element may be triggered by the target device to at least one first access network device. Therefore, in this optional implementation, before S310, the method may also include: the first network element receives a positioning performance monitoring message sent by the target device, wherein the positioning performance monitoring message is used to request performance measurement of the target positioning model, and the positioning performance monitoring message carries identification information of the target positioning model; the target device includes one of the following: the first access network device, the second access network device, and the second network element. That is to say, in this optional implementation, the first network element may perform performance monitoring of the target positioning model under the triggering of the first access network device, the second access network device, or the second network element.

[0050] For example, the first access network device can train and generate the above-mentioned target positioning model by itself, and then request the first network element to monitor the performance of the target positioning model; or, the second access network device can train and generate the above-mentioned target positioning model by itself, and send it to the first access network device for use, and the second access network device can request the first network element to monitor the performance of the target positioning model; or, the second network element trains and generates the above-mentioned target positioning model, and sends it to the first access network device for use, and the second network element can request the first network element to monitor the performance of the target positioning model.

[0051] Alternatively, the first network element itself may trigger performance monitoring of the target positioning model. For example, the first network element trains and generates the target positioning model, and sends the target positioning model to at least one first access network device for use. In order to obtain the positioning performance of the target positioning model, the first access network device may trigger performance monitoring of the target positioning model, for example, executing the above S310.

[0052] In the above implementation, the second network element may be a network data analysis function (Network Data Analytics Function, NWDAF), or may be other network elements for training target positioning models, which is not specifically limited in the embodiments of the present application.

[0053] In the above implementation, the identification information of the target positioning model carried in the positioning performance monitoring message may be the identification of the target positioning model, i.e., Model ID, or the functional identification (functionality ID) of the target positioning model, or the analysis identification (analytics ID) of the target positioning model.

[0054] In an optional implementation, the positioning performance monitoring message may also carry at least one of the following:

[0055] (1) Identification information of the target device; for example, identification information of the first access network device, identification information of the second access network device, or identification information of the second network element, used to indicate the identity information of the target device for sending the positioning performance monitoring message. For example, if it is an access network device (including the first access network device and the second access network device), the identification information may be an identification of the access network device, such as a cell global identity (CGI), or the identification information may also be an Operation Administration and Maintenance (OAM) identification of the access network device (including the first access network device and the second access network device).

[0056] (2) Monitoring period information, which is used to indicate the time for monitoring the performance of the target positioning model; the monitoring period information can indicate the time for monitoring the performance of the model, for example, the next day, the next month, etc. Optionally, the monitoring period information may include at least one of: monitoring start time, monitoring end time, monitoring repetition mode, monitoring repetition start time, and monitoring repetition end time.

[0057] (3) A first threshold (notify threshold), which is used to indicate that the target device is notified when the performance parameter corresponding to the first positioning performance monitoring information reaches the first threshold. The first threshold may represent that the positioning performance monitoring information of the target positioning model is reported to the target device when the performance parameter reaches a certain condition. For example, if it is a scheduled report, the first threshold may include: one or more of: the reporting start time, the reporting interval time, and the number of reports. If it is a conditional report, the first threshold may be at least one of the positioning accuracy threshold and the model accuracy threshold.

[0058] In the above optional implementation, the positioning performance monitoring message can be used to request the performance monitoring data of the target positioning model once, or to request the performance monitoring data of the target positioning model multiple times. For example, the positioning performance monitoring message can be a model performance monitoring request message, and the first network element can reply with the performance monitoring data of the target positioning model once in response to the request message. Alternatively, the positioning performance monitoring message can also be a model performance monitoring subscription message, and the first network element may need to reply with the performance monitoring data of the target positioning model multiple times in response to the subscription message, for example, replying with the performance monitoring data of the target positioning model to the target device once each time the performance monitoring data of the target positioning model changes, or, periodically replying with the performance monitoring data of the target positioning model to the target device according to a certain period.

[0059] In the above optional implementation, optionally, after the first network element receives the positioning performance monitoring message sent by the target device, the method may further include: the first network element sends a positioning performance monitoring response message to the target device, wherein the positioning performance monitoring response message carries identification information of the target positioning model. Through the positioning performance monitoring response, the target device can be notified that the first network element has received the above positioning performance monitoring message.

[0060] In an optional implementation, after the first network element receives the above-mentioned positioning performance monitoring message, if the first network element has learned the second positioning performance monitoring information of the target positioning model, and the learned second positioning performance monitoring information meets the second threshold, then the positioning performance monitoring response message may also carry performance indicators corresponding to the second positioning performance monitoring information, such as positioning accuracy information and model accuracy information. The second threshold may be the same as the first threshold, for example, when the first threshold is a positioning accuracy threshold or a model accuracy threshold, the second threshold may be the first threshold, or the second threshold may be different from the first threshold, for example, when the first threshold includes one or more of the reporting start time, reporting interval time, and reporting times, the second threshold may be different from the first threshold, for example, the second threshold may be a preset value or a value indicated by a network-side device, etc. Through this optional implementation method, if the first network element has completed performance monitoring of the target positioning model and the performance meets the requirements (for example, the positioning performance monitoring message carries a first threshold, and the first threshold is a positioning accuracy threshold or a model accuracy threshold), the first network element can directly inform the target device of the performance indicators corresponding to the second positioning performance monitoring information in the response message, so there is no need to initiate the performance monitoring process again, thereby saving the process.

[0061] In the above optional implementation manner, the positioning performance monitoring response message may also carry at least one of the following:

[0062] (1) Fourth indication information, used to indicate one of the following: accepting performance monitoring of the target positioning model or refusing performance monitoring of the target positioning model; for example, when the first network element has completed performance monitoring of the target positioning model, and the performance indicator corresponding to the second positioning performance monitoring information obtained by monitoring does not meet the requirements (for example, the performance indicator corresponding to the obtained second positioning performance monitoring information does not meet the second threshold), the fourth indication information carried in the positioning performance monitoring response message may indicate refusing to perform performance monitoring on the target positioning model.

[0063] (2) A reason value, which is used to indicate the reason for refusing to perform performance monitoring on the target positioning model. For example, the reason for refusing to perform performance monitoring on the target positioning model includes at least one of the following: network congestion; the load of the first network element is greater than a threshold; the model performance of the target positioning model does not meet the requirements; and the positioning performance monitoring of the target positioning model is being performed.

[0064] After receiving the positioning performance monitoring response message, the target device can learn whether the first network element accepts the performance monitoring of the target positioning model. If not, the target device can further learn the reason for the rejection.

[0065] In an optional implementation, after receiving the above positioning performance monitoring message, the first network element may execute the above S310, and then execute the following S312 and S314 to obtain the first positioning performance monitoring information of the target positioning model, and then send a positioning performance monitoring notification message to the target device, the positioning performance monitoring notification message carrying the identification information of the target positioning model and the performance index corresponding to the first positioning performance monitoring information, for example, at least one of the positioning accuracy information and model accuracy information corresponding to the first positioning performance monitoring information. Thus, the target device can know the performance index of the target positioning model, and further determine whether to use the target positioning model for reasoning based on the performance index.

[0066] S312: The first network element receives a positioning result message sent by the at least one first access network device.

[0067] The positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI ​​measurement information, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information.

[0068] After receiving the above-mentioned positioning message, the first access network device can send a positioning result message to the first network element, and report at least one of the AI ​​measurement information, the AI ​​measurement information and the real measurement information, and the difference information between the AI ​​measurement information and the real measurement information to the first network element through the positioning result message.

[0069] In an optional implementation, the positioning result message may also carry the following information:

[0070] (1) Identification information of the target positioning model; the identification information of the target positioning model can indicate the first network element and the target positioning model associated with the target positioning measurement information.

[0071] (2) third indication information, used to indicate that the positioning result message carries the target positioning measurement information. The third indication information may indicate that the positioning result message carries information related to AI measurement information inferred using the target positioning model. For example, the third indication information may be an AI information indication (AI info ind).

[0072] Optionally, the positioning result message may also carry first timestamp information for indicating the time when the AI ​​measurement information is inferred using the target positioning model. The first timestamp information can be used to find the corresponding AI measurement information and conventional measurement information in subsequent steps, which facilitates performance comparison.

[0073] S314: The first network element obtains first positioning performance monitoring information of the target positioning model based on the target positioning measurement information.

[0074] In an optional implementation, when the target positioning measurement information includes the AI ​​measurement information and the real measurement information, in S314, the first network element may compare the difference between the AI ​​measurement information and the real measurement information, and then obtain the first positioning performance monitoring information of the target positioning model based on the difference between the AI ​​measurement information and the real measurement information. For example, when there are multiple first access network devices, the first network element may obtain the difference between the AI ​​measurement information and the real measurement information corresponding to each first access network device based on the AI ​​measurement information and the real measurement information reported by each first access network device, and then calculate the difference corresponding to each first access network device based on a predetermined algorithm, thereby obtaining the first positioning performance monitoring information, wherein the predetermined algorithm may be variance, mean square error (MSE), mean absolute error (MAE), etc., and the specific algorithm is not limited in the embodiments of the present application.

[0075] In an optional implementation manner, when the target positioning measurement information includes the difference information, in S314, the first network element may directly obtain first positioning performance monitoring information of the target positioning model based on the difference information.

[0076] In an optional implementation, when the target positioning measurement information includes the AI ​​measurement information, in S314, the first network element may obtain terminal location information, wherein the terminal location information includes: terminal location estimation and second timestamp information; then based on the AI ​​measurement information, obtain the model location estimation; finally, based on the difference between the terminal location estimation and the model location estimation, obtain the first positioning performance monitoring information of the target positioning model.

[0077] In the above optional implementation method, the first network element may obtain the terminal location information from the UE, or the first network element may also obtain the terminal location information through calculation, for example, the terminal location information obtained based on real measurement information, or the first network element may obtain the terminal location information based on another positioning task at the same time (the positioning task uses a non-AI positioning method) or a new positioning task initiated in the time period, which is not limited in the specific embodiments of the present application.

[0078] In the above optional implementation method, the first network element can match the second timestamp information in the acquired terminal location information with the time information of the AI ​​measurement information obtained by inferring the target positioning model, use the AI ​​measurement information obtained at the same time to perform position estimation, obtain a model position estimate, and compare the model position estimate with the terminal position estimate in the terminal location information to obtain the difference between the two position estimates.

[0079] In an optional implementation, the first network element may reply to the target device with a model performance detection reply message or a model performance detection notification message, and the message may carry at least one of the following model performance information:

[0080] Model identification information, model ID;

[0081] Performance Key Performance Indicators (KPIs) are used to characterize the performance parameters of the target positioning model inference, for example, they may include positioning accuracy information, model accuracy information, etc.

[0082] In the above implementation, optionally, the first network element may send the model performance information to the above target device if the local reporting policy or the above first threshold is met.

[0083] For example, the first network element may send the model performance information to the first access network device, and the first access network device may send the model performance monitoring information to the model training party (eg, LMF or NWDAF).

[0084] Through the above technical solution provided by the embodiment of the present application, the first network element sends a positioning message to at least one first access network device, wherein the positioning message is used to request reporting of one of the following: AI measurement information generated by reasoning using the target positioning model, the AI ​​measurement information and real measurement information, and the difference information between the AI ​​measurement information and the real measurement information; the first network element receives a positioning result message sent by the at least one first access network device, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI ​​measurement information, the AI ​​measurement information and real measurement information, and the difference information between the AI ​​measurement information and the real measurement information; the first network element obtains the first positioning performance monitoring information of the target positioning model based on the target positioning measurement information. Thus, the first network element can monitor the positioning performance of the target positioning model based on the target positioning measurement information reported by the first access network device, solving the problem that the accuracy and precision of the estimation result obtained by using the AI ​​measurement information inferred by the AI ​​positioning model for position estimation cannot be determined due to the unknown performance of the AI ​​positioning model.

[0085] Based on the same technical concept, an embodiment of the present application also provides a method for sending positioning monitoring information of a positioning model.

[0086] Figure 4 A flow chart of a method for sending positioning monitoring information of a positioning model provided in an embodiment of the present application is shown. The method 400 can be executed by the first access network device, such as Figure 4 As shown, the method mainly includes the following steps.

[0087] S410: A first access network device receives a positioning message sent by a first network element.

[0088] In an embodiment of the present application, the positioning message is used to request reporting of one of the following: AI measurement information generated by reasoning using the target positioning model, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information.

[0089] Among them, the positioning message is the same as the positioning message in method 300. The first network element can send the positioning message to the first access network device according to the relevant description in the above method 300. For details, please refer to the relevant description in method 300.

[0090] In an optional implementation, before S410, the method may also include: the first access network device sends a positioning performance monitoring message to the first network element, wherein the positioning performance monitoring message is used to request performance measurement of the target positioning model, and the positioning performance monitoring message carries identification information of the target positioning model. In this optional implementation, the first access network device can trigger the first network element to perform performance monitoring on the target positioning model. For example, the first access network device can trigger the first network element to perform performance monitoring on the target positioning model after receiving the target positioning model distributed by the network side device, or after the first access network device has trained and generated the target positioning model, so as to determine the positioning performance of the target positioning model.

[0091] In the above optional implementation manner, optionally, the above positioning performance monitoring message may also carry at least one of the following:

[0092] (1) identification information of the first access network device; for example, the identification information may be an identification (ID) of the first access network device, such as a cell global identity (CGI), or the identification information may be an operation administration and maintenance (OAM) identification of the first access network device (including the first access network device and the second access network device).

[0093] (2) Monitoring period information, which is used to indicate the time for monitoring the performance of the target positioning model; the monitoring period information can indicate the time for monitoring the performance of the model, for example, the next day, the next month, etc. Optionally, the monitoring period information may include at least one of: monitoring start time, monitoring end time, monitoring repetition mode, monitoring repetition start time, and monitoring repetition end time.

[0094] (3) A first threshold (notify threshold), which is used to indicate that the target device is notified when the performance parameter corresponding to the first positioning performance monitoring information reaches the first threshold. The first threshold may represent that the positioning performance monitoring information of the target positioning model is reported to the target device when the performance parameter reaches a certain condition. For example, if it is a scheduled report, the first threshold may include: one or more of: the reporting start time, the reporting interval time, and the number of reports. If it is a conditional report, the first threshold may be at least one of the positioning accuracy threshold and the model accuracy threshold.

[0095] In the above optional implementation, the positioning performance monitoring message can be used to request the performance monitoring data of the target positioning model once, or to request the performance monitoring data of the target positioning model multiple times. For example, the positioning performance monitoring message can be a model performance monitoring request message, and in response to the request message, the first access network device can receive the performance monitoring data of the target positioning model once replied by the first network element. Alternatively, the positioning performance monitoring message can also be a model performance monitoring subscription message, and in response to the subscription message, the first access network device can receive the performance monitoring data of the target positioning model multiple times replied by the first network element. For example, each time the performance monitoring data of the target positioning model changes, the first network element can reply to the first access network device with the performance monitoring data of the target positioning model once, or, according to a certain period, periodically reply to the first access network device with the performance monitoring data of the target positioning model.

[0096] In an optional implementation, after the first access network device sends a positioning performance monitoring message to the first network element, the method may further include: the first access network device receives a positioning performance monitoring response message sent by the first network element, wherein the positioning performance monitoring response message carries identification information of the target positioning model. Through the positioning performance monitoring response, the first access network device can learn that the first network element has received the above positioning performance monitoring message.

[0097] Optionally, the positioning performance monitoring response message may also carry performance indicators corresponding to the second positioning performance monitoring information, such as positioning accuracy information and model accuracy information, etc. Through this optional implementation, if the first network element has completed the performance monitoring of the target positioning model, and the performance meets the requirements (for example, the performance indicators corresponding to the acquired second positioning performance monitoring information meet the second threshold), the first network element can directly inform the target device of the performance indicators corresponding to the second positioning performance monitoring information in the response message, so that there is no need to initiate the performance monitoring process again, thereby saving the process.

[0098] In the above optional implementation manner, the positioning performance monitoring response message may also carry at least one of the following:

[0099] (1) Fourth indication information, used to indicate one of the following: accepting performance monitoring of the target positioning model or refusing performance monitoring of the target positioning model; for example, when the first network element has completed performance monitoring of the target positioning model and the performance does not meet the requirements (for example, the performance indicator corresponding to the acquired second positioning performance monitoring information does not meet the second threshold), the fourth indication information carried in the positioning performance monitoring response message may indicate refusing to perform performance monitoring on the target positioning model.

[0100] (2) Cause value, used to indicate the reason for rejecting the performance monitoring of the target positioning model. For example, the reasons for rejecting the performance monitoring of the target positioning model include at least one of the following: network congestion; the load of the first network element is greater than the threshold; the model performance of the target positioning model does not meet the requirements; the positioning performance monitoring of the target positioning model is in progress.

[0101] After receiving the positioning performance monitoring response message, the first access network device can know whether the first network element accepts the performance monitoring of the target positioning model. If not, it can further know the reason for the rejection.

[0102] In an optional implementation, the positioning message may carry at least one of the following:

[0103] (1) Identification information of the target positioning model;

[0104] (2) First indication information, used to indicate that the first access network device uses the AI measurement information inferred by the target positioning model for positioning;

[0105] (3) Second indication information, used to indicate that the first network element will perform performance monitoring on the target positioning model.

[0106] Through this positioning message, the first access network device can know that the first network element will perform performance monitoring on the target positioning model, or the information that the first network element needs to obtain.

[0107] In a specific application, the above positioning message may be a New Radio Positioning Protocol a (NRPPa) positioning message.

[0108] S412, the first access network device obtains the true measurement information and the AI measurement information inferred by using the target positioning model.

[0109] After receiving the above positioning message, the first access network device knows that it needs to report at least one of the AI measurement information generated by reasoning using the target positioning model, the difference information between the AI measurement information and the true measurement information. Therefore, the first access network device can use the target positioning model to reason to obtain the AI measurement information. In addition, the first access network device can also use conventional positioning measurements, such as the measurement of positioning reference signals, to obtain the true measurement information.

[0110] S414, the first access network device sends a positioning result message to the first network element based on the true measurement information and the AI measurement information.

[0111] In an embodiment of the present application, the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI ​​measurement information, the AI ​​measurement information and real measurement information, and the difference information between the AI ​​measurement information and the real measurement information.

[0112] The first access network device sends a positioning result message carrying target positioning measurement information to the first network element, so that the first network element can obtain positioning performance monitoring information of the target positioning model based on the target positioning measurement information, and realize monitoring of the positioning performance of the target positioning model.

[0113] In an optional implementation, the positioning result message may carry the following information:

[0114] (1) identification information of the target positioning model;

[0115] (2) third indication information, used to indicate that the positioning result message carries the target positioning measurement information.

[0116] Optionally, the positioning result message also carries first timestamp information, which is used to indicate the time when the AI ​​measurement information is inferred using the target positioning model.

[0117] In an optional implementation, when the first access network device sends the above positioning performance monitoring message to the first network element, after S414, the method may also include: the first access network device receives the positioning performance monitoring notification message sent by the first network element, and the positioning performance monitoring notification message carries the identification information of the target positioning model and the performance index corresponding to the first positioning performance monitoring information, for example, at least one of the positioning precision information and model accuracy information corresponding to the first positioning performance monitoring information. Thus, the first access network device can learn the performance index of the target positioning model, and further determine whether to use the target positioning model for positioning reasoning based on the performance index.

[0118] Through the above-mentioned technical solution provided by the embodiment of the present application, the first access network device can send a positioning result message to the first network element after receiving the above-mentioned positioning message, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI ​​measurement information, the AI ​​measurement information and the real measurement information, and the difference information between the AI ​​measurement information and the real measurement information, so that the first network element can obtain the first positioning performance monitoring information of the target positioning model based on the target positioning measurement information, and realize the monitoring of the positioning performance of the target positioning model, which solves the problem that the accuracy and precision of the estimation result obtained by using the AI ​​measurement information inferred by the AI ​​positioning model for position estimation cannot be determined due to the unknown performance of the AI ​​positioning model.

[0119] In an embodiment of the present application, after obtaining the performance parameters of the target positioning model, the above-mentioned target device can optimize the target positioning model to improve the positioning accuracy of the target positioning model and optimize the service experience of AI positioning when the model performance is poor.

[0120] The following uses the first network element being LMF and the second network element being NWDAF as an example to illustrate the technical solution provided in the embodiment of the present application.

[0121] Figure 5 Another flow chart of the method for obtaining positioning model performance information provided by the embodiment of the present application is shown as follows: Figure 5 As shown, the method mainly includes the following steps:

[0122] Step 501: Base station A obtains an AI positioning model.

[0123] exist Figure 5 In the figure, base station A is represented by access network equipment (RAN).

[0124] like Figure 5 As shown, this step includes but is not limited to the following three implementation methods:

[0125] Step 501a, base station A trains and generates an AI positioning model;

[0126] Step 501b, the AI ​​positioning model generated by LMF training is then sent to base station A;

[0127] Step 501c: After the NWDAF training generates a model, it is sent to base station A.

[0128] It should be noted that, before step 501b or step 501c, optionally, base station A may send a request or subscription message to LMF or NWDAF, requesting LMF or NWDAF to provide base station A with an AI positioning model.

[0129] Step 502: Base station A decides to perform performance monitoring on the target AI positioning model, triggering the performance monitoring process.

[0130] Step 503: Base station A sends a model performance monitoring request or a signing message to the LMF, requesting performance monitoring of the target AI positioning model and feedback of performance monitoring information.

[0131] The model performance monitoring request or contract signing message may carry at least one of the following:

[0132] Model identification information, which represents the information of the target AI positioning model. For example, it can be one of: Model ID, functionality ID, and analytics ID;

[0133] Base station identification information, representing the identity information of base station A requesting model detection, for example, it can be CGI;

[0134] Monitoring period, which represents the time for monitoring the performance of the model (e.g., the next day, the next month), and may include, for example, at least one of: monitoring start time, monitoring end time, monitoring repetition mode, monitoring repetition start time, and monitoring repetition end time;

[0135] The first threshold (notify threshold) indicates that the performance parameter is reported to base station A when a certain condition is met, for example, including scheduled reporting: reporting start time, reporting interval, reporting times; or conditional reporting: positioning accuracy threshold and model accuracy threshold.

[0136] It should be noted that the above-mentioned model performance monitoring request or signing message can be encapsulated in the NRPPa message and sent to the LMF. NRPPa is a point-to-point protocol stack between NG-RAN and LMF, which is used to transmit positioning-related signaling. A new message can also be added to transmit the request or signing message.

[0137] It should be noted that the reply corresponding to the model performance monitoring request message is one-time, while the reply corresponding to the model performance monitoring contract message is one or more times.

[0138] It should be noted that the monitoring time period and the first threshold can be optional parameters. When one or both of them are not provided, LMF can perform detection and reporting according to local policies or default monitoring parameters. For example, the local configuration defaults to a detection time period of the next day, and the first threshold is to report when the positioning accuracy is less than 1m.

[0139] It should be noted that the base station identification information may be a base station ID, or an OAM ID corresponding to the base station may be used to identify an OAM.

[0140] Step 504: LMF decides whether to perform this model detection based on the current business situation, load situation, and the known model detection performance.

[0141] If not, for example, the model ID requested for detection has been detected and the performance is poor or is being detected, then step 505 is executed and subsequent steps are ignored; if yes, then step 504 is executed and subsequent steps are executed.

[0142] Another implementation method, step 505 is not mandatory, LMF does not need to reply, LMF executes steps 506 to 513 after deciding to perform monitoring, and LMF directly executes step 513 after deciding not to perform monitoring.

[0143] Step 505: LMF sends a model performance detection reply message to base station A.

[0144] The model performance test reply message may carry at least one of the following:

[0145] Model identification information, which represents the model ID of the target model to be monitored;

[0146] Performance KPI, which represents the performance parameters of model reasoning, including positioning accuracy information, model accuracy information, etc.

[0147] The fourth indication information indicates acceptance or rejection of the performance monitoring of the target model, including acceptance and rejection;

[0148] Reason value, indicating the reason for rejecting performance monitoring;

[0149] It should be noted that, when the LMF has learned the monitoring performance of the target model or the known performance meets the first threshold in step 503, the model performance detection reply message carries the performance KPI, and reports the model performance information to base station A. Among them, the fourth indication information is an optional parameter, which can be an implicit indication information, for example, carrying a reason value indicates rejection, and not carrying a reason value indicates acceptance, or it can be an explicit indication information, which clearly represents acceptance or rejection of performance monitoring of the target model; the reason value is an optional parameter, including one of the following: network congestion, excessive load, model performance does not meet the requirements, and is being monitored.

[0150] Step 506, in the positioning requests such as Mobile Original Location Request (MO-LR), Mobile Terminated Location Request (MT-LR), and deferred MT-LR, the LMF decides that the UE needs to be positioned, selects the positioning method, and determines the base stations involved in the positioning.

[0151] In this step, LMF can locate the UE by interacting with the Access and Mobility Management Function (AMF) and the Gateway Mobile Location Centre (GMLC).

[0152] It should be noted that the base station involved in positioning may not be the aforementioned base station A, but may be other base stations. The illustrations in the subsequent steps do not distinguish for the sake of simplicity, and do not mean that the RAN here always has only one base station.

[0153] Step 507: LMF sends a positioning message (an enhanced or newly added NRPPa message) to one or more base stations B (not limited to base station A, base station B may be base station A), requesting one or more base stations B to report the measurement information of the UE, and the positioning message carries at least one of the following:

[0154] Model identification information, such as model ID, analytics ID;

[0155] The first indication information, i.e., AI positioning indication, indicates that the base station needs to report AI measurement information inferred using the target model;

[0156] It should be noted that in the related technology, LMF sends NRPPa positioning message to one or more base stations B, and one or more base stations B return conventional measurement information (i.e., non-AI measurement information) to LMF. In the embodiment of the present application, the base station is required to report AI measurement information through the second indication information.

[0157] In step 508, one or more base stations B obtain conventional positioning measurement information (ie, the above-mentioned real measurement information) and AI measurement information.

[0158] Step 509: One or more base stations B reply the measurement result to the LMF, wherein the measurement result may carry at least one of the following:

[0159] The third indication information, namely the AI ​​indication, indicates that the measurement result carries the AI ​​measurement information inferred using the target model;

[0160] Model identification information, such as model ID, analytics ID;

[0161] AI measurement information, representing AI measurement information inferred using the target model;

[0162] AI measurement and conventional measurement difference information, which represents the difference between AI measurement information and conventional measurement information, can be a matrix information;

[0163] Conventional measurement information (i.e. the real measurement information mentioned above);

[0164] Timestamp.

[0165] It should be noted that, when the third indication is carried, one of the two parameters, AI measurement information and AI measurement and conventional measurement information difference information, can be carried.

[0166] In step 510, LMF obtains UE location information from the UE, which is called label positioning result or UE location estimate calculated by non-AI positioning, or uses a positioning method different from the positioning method selected in S506 to locate the UE once and obtain the regular positioning result of the UE.

[0167] This step is optional.

[0168] One is the implementation method. S506 selects to use the uplink positioning method. This step uses the downlink positioning method to initiate a new positioning process to the UE, requiring the UE to report the UE location. It should be noted that in order for the LMF to compare the positioning estimates of the two positionings (AI positioning and conventional positioning), the UE can carry a timestamp when reporting the positioning estimate, and the LMF binds it according to the timestamp; or the LMF assigns the same task identification information (such as LCS correlation ID) to the two positionings.

[0169] Step 511: LMF performs performance comparison and calculates performance KPI information such as positioning accuracy and model accuracy.

[0170] In specific applications, the positioning accuracy and model accuracy can be calculated by selecting one of the following methods:

[0171] (1) Direct comparison: Compare the positioning estimates. Use the AI ​​measurement to calculate the AI ​​positioning estimate and compare it with the conventional positioning estimate obtained in step 510. Here, the comparison is between the AI ​​position estimate and the label positioning result.

[0172] (2) Indirect comparison 1: Comparison of measurement information, directly comparing AI measurement information with conventional measurement information;

[0173] (3) Indirect comparison 2: Directly judge the difference information between AI measurement and conventional measurement. LMF determines the model performance by judging whether the measurement difference meets the threshold.

[0174] Step 512: LMF replies to base station A with a model performance detection reply or notification message.

[0175] The model performance test reply or notification message may carry at least one of the following:

[0176] Model identification information, model ID;

[0177] Performance KPI, which represents the performance parameters of model reasoning, including positioning accuracy information, model accuracy information, etc.

[0178] Optionally, when the LMF local reporting strategy or the condition of the first threshold in step 503 is met, the LMF replies to base station A with the above-mentioned model performance detection reply or notification message, and sends the model performance information to base station A.

[0179] Step 513: Base station A sends the model performance monitoring information to the model training party (LMF or NWDAF), including the information in the above step 512.

[0180] This step is optional.

[0181] Figure 6 Another flow chart of the method for obtaining positioning model performance information provided by the embodiment of the present application is shown. Figure 6 The difference is that this method is triggered by NWDAF to monitor model performance. Figure 6 As shown, the method mainly includes the following steps:

[0182] Step 601: NWDAF decides to perform performance monitoring on the target AI positioning model.

[0183] For example, NWDAF can generate a target AI positioning model during training, send the target AI positioning model to at least one base station A for use, and then NWDAF decides to perform performance monitoring on the target AI positioning model.

[0184] Step 602, NWDAF sends a model performance monitoring request or a contract message to LMF. It should be noted that this step can occur through a direct interface between NWDAF and LMF, or can be transferred through GMLC. The subsequent step 611 is similar.

[0185] Step 603-Step 610, same as Step 504-Step 511 above, LMF performs performance monitoring of the target AI positioning model and calculates performance KPI information such as positioning accuracy and model accuracy.

[0186] Step 611, LMF sends a model performance detection reply or notification message to NWDAF.

[0187] The model performance test reply or notification message may carry at least one of the following:

[0188] Model identification information, model ID;

[0189] Performance KPI, which represents the performance parameters of model reasoning, including positioning accuracy information, model accuracy information, etc.

[0190] The above-mentioned technical solution provided in the embodiment of the present application solves the problem that the accuracy and precision of the estimation results obtained by using the AI ​​measurement information inferred by the AI ​​positioning model for position estimation due to the unknown performance of the AI ​​positioning model. As a result, the model provider can optimize the AI ​​positioning model based on the monitoring results to improve the positioning accuracy of the AI ​​positioning model.

[0191] The method for obtaining positioning model performance information provided in the embodiment of the present application can be executed by a device for obtaining positioning model performance information. In the embodiment of the present application, the device for obtaining positioning model performance information executing the method for obtaining positioning model performance information is used as an example to illustrate the device for obtaining positioning model performance information provided in the embodiment of the present application.

[0192] Figure 7 A schematic diagram showing a structure of a device for acquiring positioning model performance information provided in an embodiment of the present application is shown as follows: Figure 7 As shown, the device 700 includes: a first sending module 701, a first receiving module 702 and a first obtaining module 703.

[0193] In an embodiment of the present application, a first sending module 701 is used to send a positioning message to at least one first access point and a first access network device, wherein the positioning message is used to request a report of one of the following: AI measurement information generated by reasoning using the target positioning model, the AI ​​measurement information and the real measurement information, and the difference information between the AI ​​measurement information and the real measurement information; a first receiving module 702 is used to receive a positioning result message sent by the at least one first access network device, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI ​​measurement information, the AI ​​measurement information and the real measurement information, and the difference information between the AI ​​measurement information and the real measurement information; a first acquisition module 703 is used to acquire first positioning performance monitoring information of the target positioning model based on the target positioning measurement information.

[0194] In an optional implementation, the positioning message carries at least one of the following:

[0195] identification information of the target positioning model;

[0196] The first indication information is used to instruct the first access network device to use the AI ​​measurement information inferred by the target positioning model to perform positioning;

[0197] The second indication information is used to indicate that the first network element will perform performance monitoring on the target positioning model.

[0198] In an optional implementation, the positioning result message also carries the following information:

[0199] identification information of the target positioning model;

[0200] The third indication information is used to indicate that the positioning result message carries the target positioning measurement information.

[0201] In an optional implementation, the positioning result message also carries first timestamp information for indicating the time when the AI ​​measurement information is inferred using the target positioning model.

[0202] In an optional implementation, acquiring first positioning performance monitoring information of the target positioning model based on the target positioning measurement information includes one of the following:

[0203] In a case where the target positioning measurement information includes the AI ​​measurement information and the real measurement information, comparing a difference between the AI ​​measurement information and the real measurement information, and acquiring first positioning performance monitoring information of the target positioning model based on the difference between the AI ​​measurement information and the real measurement information;

[0204] In a case where the target positioning measurement information includes the difference information, acquiring first positioning performance monitoring information of the target positioning model based on the difference information;

[0205] In a case where the target positioning measurement information includes the AI ​​measurement information, terminal position information is obtained, wherein the terminal position information includes: a terminal position estimate and second timestamp information; based on the AI ​​measurement information, a model position estimate is obtained; and based on a difference between the terminal position estimate and the model position estimate, first positioning performance monitoring information of the target positioning model is obtained.

[0206] In an optional implementation, the first receiving module 702 is also used to receive a positioning performance monitoring message sent by a target device, wherein the positioning performance monitoring message is used to request performance measurement of the target positioning model, and the positioning performance monitoring message carries identification information of the target positioning model; the target device includes one of the following: the first access network device, the second access network device, and the second network element.

[0207] In an optional implementation, the positioning performance monitoring message further carries at least one of the following:

[0208] identification information of a target device of the second network element;

[0209] Monitoring time period information, used to indicate the time for performance monitoring of the target positioning model;

[0210] The first threshold is used to indicate that when the performance parameter corresponding to the first positioning performance monitoring information reaches the first threshold, the target device is notified.

[0211] In an optional implementation, the identification information of the target device includes one of the following: an identification ID of the first access network device or the second access network device, and an OAM ID corresponding to the first access network device or the second access network device.

[0212] In an optional implementation, the first sending module 701 is further configured to send a positioning performance monitoring response message to the target device, wherein the positioning performance monitoring response message carries identification information of the target positioning model.

[0213] In an optional implementation, when the second positioning performance monitoring information of the target positioning model has been obtained and the obtained second positioning performance monitoring information meets a second threshold, the positioning performance monitoring response message also carries a performance indicator corresponding to the second positioning performance monitoring information.

[0214] In an optional implementation, the positioning performance monitoring response message further carries at least one of the following:

[0215] The fourth indication information is used to indicate one of the following: accepting the performance monitoring of the target positioning model, rejecting the performance monitoring of the target positioning model;

[0216] A reason value is used to indicate the reason for rejecting the performance monitoring of the target positioning model.

[0217] In an optional implementation, the reason for refusing to perform performance monitoring on the target positioning model includes at least one of the following:

[0218] Network congestion;

[0219] The load of the first network element is greater than a threshold;

[0220] The model performance of the target positioning model does not meet the requirements;

[0221] The positioning performance of the target positioning model is being monitored.

[0222] In an optional implementation, the positioning performance monitoring message includes one of the following:

[0223] A positioning performance monitoring request message, used to request to obtain positioning performance monitoring data of the target positioning model once;

[0224] The positioning performance monitoring signing message is used to request to obtain the positioning performance monitoring data of the target positioning model at least once.

[0225] In an optional implementation, the first sending module 701 is also used to send a positioning performance monitoring notification message to the target device, wherein the positioning performance monitoring notification message carries identification information of the target positioning model and performance indicators corresponding to the first positioning performance monitoring information.

[0226] In an optional implementation, the performance indicator includes at least one of the following:

[0227] Positioning accuracy information;

[0228] Model accuracy information.

[0229] The device for acquiring positioning model performance information provided in the embodiment of the present application can achieve Figure 3 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0230] Figure 8A structural diagram of a positioning monitoring information sending device of a positioning model provided in an embodiment of the present application is shown, Figure 8 As shown, the device 800 includes: a second receiving module 801 , a second acquiring module 802 and a second sending module 803 .

[0231] In an embodiment of the present application, a second receiving module 801 is used to receive a positioning message sent by a first network element, wherein the positioning message is used to request a report of one of the following: AI measurement information generated by reasoning using the target positioning model, the AI ​​measurement information and the real measurement information, and the difference information between the AI ​​measurement information and the real measurement information; a second acquisition module 802 is used to obtain the real measurement information and the AI ​​measurement information inferred using the target positioning model; a second sending module 803 is used to send a positioning result message to the first network element based on the real measurement information and the AI ​​measurement information, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI ​​measurement information, the AI ​​measurement information and the real measurement information, and the difference information between the AI ​​measurement information and the real measurement information.

[0232] In an optional implementation, the positioning message carries at least one of the following:

[0233] identification information of the target positioning model;

[0234] The first indication information is used to instruct the first access network device to use the AI ​​measurement information inferred by the target positioning model to perform positioning;

[0235] The second indication information is used to indicate that the first network element will perform performance monitoring on the target positioning model.

[0236] In an optional implementation, the positioning result message carries the following information:

[0237] identification information of the target positioning model;

[0238] The third indication information is used to indicate that the positioning result message carries the target positioning measurement information.

[0239] In an optional implementation, the positioning result message also carries first timestamp information for indicating the time when the AI ​​measurement information is inferred using the target positioning model.

[0240] The positioning monitoring information sending device of the positioning model provided in the embodiment of the present application can achieve Figure 4 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0241] like Fig. 9 As shown, the embodiment of the present application also provides a communication device 900, including a processor 901 and a memory 902, and the memory 902 stores a program or instruction that can be run on the processor 901. For example, when the communication device 900 is a first network element, the program or instruction is executed by the processor 901 to implement the various steps of the embodiment of the method for obtaining the performance information of the positioning model, and can achieve the same technical effect. When the communication device 900 is an access network device, the program or instruction is executed by the processor 901 to implement the various steps of the embodiment of the method for sending positioning monitoring information of the positioning model, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0242] The embodiment of the present application also provides a network side device, including a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the following Figure 3 Or the steps of the method embodiment shown in 4. This network side device embodiment corresponds to the above network side device method embodiment, and each implementation process and implementation method of the above method embodiment can be applied to this network side device embodiment and can achieve the same technical effect.

[0243] Specifically, the embodiment of the present application also provides a network side device. Fig.10 As shown, the network side device 1000 includes: an antenna 1001, a radio frequency device 1002, a baseband device 1003, a processor 1004 and a memory 1005. The antenna 1001 is connected to the radio frequency device 1002. In the uplink direction, the radio frequency device 1002 receives information through the antenna 1001 and sends the received information to the baseband device 1003 for processing. In the downlink direction, the baseband device 1003 processes the information to be sent and sends it to the radio frequency device 1002. The radio frequency device 1002 processes the received information and sends it out through the antenna 1001.

[0244] The method executed by the network-side device in the above embodiment may be implemented in the baseband device 1003, which includes a baseband processor.

[0245] The baseband device 1003 may include, for example, at least one baseband board on which a plurality of chips are arranged. Fig.10 As shown, one of the chips is, for example, a baseband processor, which is connected to the memory 1005 through a bus interface to call the program in the memory 1005 to execute the network device operations shown in the above method embodiment.

[0246] The network side device may further include a network interface 1006, which is, for example, a Common Public Radio Interface (CPRI).

[0247] Specifically, the network side device 1000 of the embodiment of the present application further includes: instructions or programs stored in the memory 1005 and executable on the processor 1004, and the processor 1004 calls the instructions or programs in the memory 1005 to execute Figure 8 The methods executed by the modules shown achieve the same technical effects, and therefore will not be described here in detail to avoid repetition.

[0248] Specifically, the embodiment of the present application also provides a network side device. Fig.11 As shown, the network side device 1100 includes: a processor 1101, a network interface 1102 and a memory 1103. The network interface 1102 is, for example, a common public radio interface (CPRI).

[0249] Specifically, the network side device 1100 of the embodiment of the present application further includes: an instruction or program stored in the memory 1103 and executable on the processor 1101, and the processor 1101 calls the instruction or program in the memory 1103 to execute Figure 7 The methods executed by the modules shown achieve the same technical effects, and therefore will not be described here in detail to avoid repetition.

[0250] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the embodiment of the method for obtaining the performance information of the positioning model mentioned above are implemented, or the various processes of the embodiment of the method for sending the positioning monitoring information of the positioning model mentioned above are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0251] The processor is the processor in the terminal described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk. In some examples, the readable storage medium may be a non-transient readable storage medium.

[0252] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions, the various processes of the embodiment of the method for obtaining the performance information of the above-mentioned positioning model, or the various processes of the embodiment of the method for sending positioning monitoring information of the above-mentioned positioning model, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0253] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0254] The embodiments of the present application further provide a computer program / program product, which is stored in a storage medium. The computer program / program product is executed by at least one processor to perform the various processes of the embodiment of the method for obtaining the performance information of the positioning model, or to implement the various processes of the embodiment of the method for sending positioning monitoring information of the positioning model, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0255] An embodiment of the present application also provides a communication system, including: a first network element and a first access network device, wherein the first network element can be used to execute the steps of the method for acquiring positioning model performance information as described above, and the first access network device can be used to execute the steps of the method for sending positioning monitoring information of the positioning model as described above.

[0256] In one implementation, the communication system further includes a target device, and the target device is used to execute the steps executed by the target device in the method for acquiring positioning model performance information as described above.

[0257] In one implementation, the target device includes one of the following: the first access network device, the second access network device, and the second network element.

[0258] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0259] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of a computer software product plus a necessary general hardware platform, and of course, can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, disk, CD, etc.), including several instructions to enable a terminal or a network-side device to execute the methods described in each embodiment of the present application.

[0260] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms of implementation methods without departing from the purpose of the present application and the scope of protection of the claims, and these implementation methods are all within the protection of the present application.

Claims

1. A method for obtaining positioning model performance information, It is characterized in that include: The first network element sends a positioning message to at least one first access network device, wherein the positioning message is used to request reporting of one of the following: artificial intelligence AI measurement information generated by reasoning using the target positioning model, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information; The first network element receives a positioning result message sent by the at least one first access network device, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI ​​measurement information, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information; The first network element obtains first positioning performance monitoring information of the target positioning model based on the target positioning measurement information.

2. The method according to claim 1, It is characterized in that The positioning message carries at least one of the following: identification information of the target positioning model; The first indication information is used to instruct the first access network device to use the AI ​​measurement information inferred by the target positioning model to perform positioning; The second indication information is used to indicate that the first network element will perform performance monitoring on the target positioning model.

3. The method according to claim 1 or 2, It is characterized in that The positioning result message also carries the following information: identification information of the target positioning model; The third indication information is used to indicate that the positioning result message carries the target positioning measurement information.

4. The method according to any one of claims 1 to 3, It is characterized in that The positioning result message also carries first timestamp information, which is used to indicate the time when the AI ​​measurement information is inferred using the target positioning model.

5. The method according to any one of claims 1 to 4, It is characterized in that The first network element acquires first positioning performance monitoring information of the target positioning model based on the target positioning measurement information, including one of the following: In a case where the target positioning measurement information includes the AI ​​measurement information and the real measurement information, the first network element compares a difference between the AI ​​measurement information and the real measurement information, and acquires first positioning performance monitoring information of the target positioning model based on the difference between the AI ​​measurement information and the real measurement information; In a case where the target positioning measurement information includes the difference information, the first network element acquires first positioning performance monitoring information of the target positioning model based on the difference information; In the case where the target positioning measurement information includes the AI ​​measurement information, the first network element obtains terminal location information, wherein the terminal location information includes: a terminal location estimate and second timestamp information; based on the AI ​​measurement information, a model location estimate is obtained; based on a difference between the terminal location estimate and the model location estimate, first positioning performance monitoring information of the target positioning model is obtained.

6. The method according to any one of claims 1 to 5, It is characterized in that Before the first network element sends a positioning message to at least one first access network device, the method further includes: The first network element receives a positioning performance monitoring message sent by a target device, wherein the positioning performance monitoring message is used to request performance measurement of the target positioning model, and the positioning performance monitoring message carries identification information of the target positioning model; the target device includes one of the following: the first access network device, the second access network device, and the second network element.

7. The method according to claim 6, It is characterized in that The positioning performance monitoring message also carries at least one of the following: identification information of the target device; Monitoring time period information, used to indicate the time for performance monitoring of the target positioning model; The first threshold is used to indicate that when the performance parameter corresponding to the first positioning performance monitoring information reaches the first threshold, the target device is notified.

8. The method according to claim 7, It is characterized in that The identification information of the target device includes one of the following: an identification ID of the first access network device or the second access network device, and an operation, administration and maintenance OAM ID corresponding to the first access network device or the second access network device.

9. The method according to claim 7, It is characterized in that After the first network element receives the positioning performance monitoring message sent by the target device, the method further includes: The first network element sends a positioning performance monitoring response message to the target device, wherein the positioning performance monitoring response message carries identification information of the target positioning model.

10. The method according to claim 9, It is characterized in that When the first network element has obtained the second positioning performance monitoring information of the target positioning model and the performance indicator corresponding to the obtained second positioning performance monitoring information meets the second threshold, the positioning performance monitoring response message also carries the performance indicator corresponding to the second positioning performance monitoring information.

11. The method according to claim 9 or 10, It is characterized in that The positioning performance monitoring response message also carries at least one of the following: The fourth indication information is used to indicate one of the following: accepting the performance monitoring of the target positioning model or refusing the performance monitoring of the target positioning model; A reason value is used to indicate the reason for rejecting the performance monitoring of the target positioning model.

12. The method according to claim 11, It is characterized in that The reason for refusing to perform performance monitoring on the target positioning model includes at least one of the following: Network congestion; The load of the first network element is greater than a threshold; The model performance of the target positioning model does not meet the requirements; The positioning performance of the target positioning model is being monitored.

13. The method according to any one of claims 6 to 12, It is characterized in that The positioning performance monitoring message includes one of the following: A positioning performance monitoring request message, used to request to obtain positioning performance monitoring data of the target positioning model once; The positioning performance monitoring signing message is used to request to obtain the positioning performance monitoring data of the target positioning model at least once.

14. The method according to any one of claims 6 to 12, It is characterized in that After the first network element acquires first positioning performance monitoring information of the target positioning model based on the target positioning information, the method further includes: The first network element sends a positioning performance monitoring notification message to the target device, wherein the positioning performance monitoring notification message carries identification information of the target positioning model and a performance indicator corresponding to the first positioning performance monitoring information.

15. The method according to claim 10 or 14, It is characterized in that The performance indicators include at least one of the following: Positioning accuracy information; Model accuracy information.

16. A method for sending positioning monitoring information of a positioning model, It is characterized in that include: The first access network device receives a positioning message sent by the first network element, wherein the positioning message is used to request reporting of one of the following: AI measurement information generated by reasoning using the target positioning model, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information; The first access network device obtains real measurement information and AI measurement information inferred using the target positioning model; The first access network device sends a positioning result message to the first network element based on the real measurement information and the AI ​​measurement information, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI ​​measurement information, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information.

17. The method according to claim 16, It is characterized in that The positioning message carries at least one of the following: identification information of the target positioning model; The first indication information is used to instruct the first access network device to use the AI ​​measurement information inferred by the target positioning model to perform positioning; The second indication information is used to indicate that the first network element will perform performance monitoring on the target positioning model.

18. The method according to claim 17, It is characterized in that The positioning result message carries the following information: identification information of the target positioning model; The third indication information is used to indicate that the positioning result message carries the target positioning measurement information.

19. The method according to any one of claims 16 to 18, It is characterized in that The positioning result message also carries first timestamp information, which is used to indicate the time when the AI ​​measurement information is inferred using the target positioning model.

20. A device for acquiring positioning model performance information, It is characterized in that include: A first sending module is configured to send a positioning message to at least one first access point and a first access network device, wherein the positioning message is used to request reporting of one of the following: AI measurement information generated by reasoning using the target positioning model, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information; A first receiving module is configured to receive a positioning result message sent by the at least one first access network device, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI ​​measurement information, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information; The first acquisition module is used to acquire first positioning performance monitoring information of the target positioning model based on the target positioning measurement information.

21. A positioning monitoring information sending device for a positioning model, It is characterized in that include: A second receiving module is configured to receive a positioning message sent by the first network element, wherein the positioning message is used to request reporting of one of the following: AI measurement information generated by reasoning using the target positioning model, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information; A second acquisition module is used to acquire real measurement information and AI measurement information inferred using the target positioning model; A second sending module is used to send a positioning result message to the first network element based on the real measurement information and the AI ​​measurement information, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI ​​measurement information, the AI ​​measurement information and real measurement information, and difference information between the AI ​​measurement information and the real measurement information.

22. A network side device, It is characterized in that The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 19 are implemented.

23. A readable storage medium, It is characterized in that The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 19 are implemented.

Images