Model monitoring method and device and electronic equipment

Abstract

The embodiment of the invention provides a model monitoring method and device and electronic equipment, and the method comprises the steps that first equipment determines the performance of a first model according to the relation between estimation information of first terminal equipment and actual information; the estimation information of the first terminal equipment is obtained by processing according to the first model.

Description

Model monitoring method and device, and electronic equipment Technical Field

[0001] The embodiments of the present application relate to the field of mobile communication technology, and specifically to a model monitoring method and device, and electronic equipment. Background Art

[0002] Given the tremendous success of artificial intelligence (AI) or machine learning (ML) in computer vision, natural language processing, and other fields, the communications field has begun to try to use AI / ML technology to seek new technical ideas to solve technical problems that are limited by traditional methods.

[0003] How to monitor the performance of AI / ML models in the communications field has always been a concern in this field.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a model monitoring method and device, and an electronic device.

[0006] The present invention provides a model monitoring method, including:

[0007] The first device determines the performance of the first model based on the relationship between the estimated information of the first terminal device and the actual information; the estimated information of the first terminal device is obtained by processing according to the first model.

[0008] This embodiment of the present application provides a model monitoring device, applied to a first device, comprising:

[0009] The determining unit is configured to determine the performance of the first model according to the relationship between the estimated information and the actual information of the first terminal device; the estimated information of the first terminal device is obtained by processing according to the first model.

[0010] An electronic device provided in an embodiment of the present application includes a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to perform the above method.

[0011] The chip provided in the embodiment of the present application is used to implement the above-mentioned model monitoring method.

[0012] Specifically, the chip includes: a processor, which is used to call and run a computer program from a memory, so that a device equipped with the chip executes the above-mentioned model monitoring method.

[0013] The computer-readable storage medium provided in an embodiment of the present application is used to store a computer program, which enables a computer to execute the above-mentioned model monitoring method.

[0014] The computer program product provided in the embodiment of the present application includes computer program instructions, which enable a computer to execute the above-mentioned model monitoring method.

[0015] The computer program provided in the embodiment of the present application, when executed on a computer, enables the computer to execute the above-mentioned model monitoring method.

[0016] In the model monitoring method provided in the embodiments of the present application, a first device determines the performance of a first model based on the relationship between estimated information and actual information of a first terminal device; the estimated information of the first terminal device is processed according to the first model. In other words, the first device can compare the estimated information calculated by the first model with the actual information, and determine the performance of the first model based on the relationship between the estimated information and the actual information. If the performance of the first model does not meet actual requirements, the first model can be promptly switched and updated to ensure normal operation of the service. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] FIG1 is a schematic diagram of a communication process of a wireless communication system provided in an embodiment of the present application;

[0019] FIG2A is a schematic diagram showing the principle of a downlink-based positioning method provided in an embodiment of the present application;

[0020] FIG2B is a schematic diagram showing the principle of an uplink-based positioning method provided in an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a neuron structure provided by related technology

[0022] FIG4 is a schematic diagram of the structure of a neural network provided by the related art;

[0023] FIG5 is a schematic diagram of the structure of a convolutional neural network provided by the related art;

[0024] FIG6 is a schematic diagram of the structure of a Long Short-Term Memory (LSTM) network provided by the related art;

[0025] FIG7 is a flow chart of a model monitoring method according to an embodiment of the present application;

[0026] FIG8 is a second flow chart of a model monitoring method provided in an embodiment of the present application;

[0027] FIG9A is a schematic diagram of a first model direct positioning principle provided by an embodiment of the present application;

[0028] FIG9B is a schematic diagram of a first model-assisted positioning principle provided in an embodiment of the present application;

[0029] FIG10A is a third flow chart of a model monitoring method provided in an embodiment of the present application;

[0030] FIG10B is a fourth flow chart of a model monitoring method provided in an embodiment of the present application;

[0031] FIG11A is a fifth flow chart of a model monitoring method provided in an embodiment of the present application;

[0032] FIG11B is a sixth flow chart of a model monitoring method provided in an embodiment of the present application;

[0033] FIG12 is a schematic diagram of the structure of a model monitoring device 1200 provided in an embodiment of the present application;

[0034] FIG13 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0035] FIG14 is a schematic structural diagram of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The following will describe the technical solutions in the embodiments of this application in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0037] To facilitate understanding of the technical solutions of the embodiments of the present application, the relevant technologies of the embodiments of the present application are described below. The following relevant technologies can be arbitrarily combined with the technical solutions of the embodiments of the present application as optional solutions, and they all fall within the protection scope of the embodiments of the present application.

[0038] FIG1 is a schematic diagram of an application scenario of an embodiment of the present application.

[0039] As shown in Figure 1, a communication system 100 may include a terminal device 110 and a network device 120. The network device 120 may communicate with the terminal device 110 via an air interface. The terminal device 110 and the network device 120 support multi-service transmission.

[0040] It should be understood that the embodiments of the present application are only illustrative of the communication system 100, but the embodiments of the present application are not limited thereto. That is, the technical solutions of the embodiments of the present application can be applied to various communication systems, such as: Long Term Evolution (LTE) system, LTE Time Division Duplex (TDD), Universal Mobile Telecommunication System (UMTS), Internet of Things (IoT) system, Narrow Band Internet of Things (NB-IoT) system, enhanced Machine-Type Communications (eMTC) system, 5G communication system (also known as New Radio (NR) communication system), or future communication systems.

[0041] In the communication system 100 shown in Figure 1, the network device 120 may be an access network device that communicates with the terminal device 110. The access network device may provide communication coverage for a specific geographical area and may communicate with the terminal device 110 (eg, UE) located within the coverage area.

[0042] The network device 120 may be an evolved Node B (eNB or eNodeB) in a Long Term Evolution (LTE) system, or a Next Generation Radio Access Network (NG RAN) device, or a base station (gNB) in an NR system, or a wireless controller in a Cloud Radio Access Network (CRAN), or the network device 120 may be a relay station, an access point, an in-vehicle device, a wearable device, a hub, a switch, a bridge, a router, or a network device in a future evolved Public Land Mobile Network (PLMN), etc.

[0043] The terminal device 110 may be any terminal device, including but not limited to a terminal device connected to the network device 120 or other terminal devices by wire or wireless connection.

[0044] For example, the terminal device 110 may refer to an access terminal, user equipment (UE), a subscriber unit, a subscriber station, a mobile station, a mobile station, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communication device, a user agent, or a user apparatus. An access terminal may be a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, an IoT device, a satellite handheld terminal, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA), a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, an in-vehicle device, a wearable device, a terminal device in a 5G network, or a terminal device in a future evolution network, etc.

[0045] The terminal device 110 can be used for device-to-device (D2D) communication.

[0046] The wireless communication system 100 may further include a core network device 130 that communicates with the network device 120. The core network device 130 may be a 5G core network (5G Core, 5GC) device, such as an Access and Mobility Management Function (AMF), an Authentication Server Function (AUSF), a User Plane Function (UPF), or a Session Management Function (SMF). Optionally, the core network device 130 may also be an Evolved Packet Core (EPC) device of an LTE network, such as a Session Management Function + Core Packet Gateway (SMF+PGW-C) device. It should be understood that SMF+PGW-C can simultaneously implement the functions that can be implemented by SMF and PGW-C. During the network evolution process, the above-mentioned core network device may also be called other names, or a new network entity may be formed by dividing the functions of the core network, which is not limited in the embodiments of the present application.

[0047] The functional units in the communication system 100 may also establish connections and implement communication via next generation (NG) network interfaces.

[0048] For example, the terminal device establishes an air interface connection with the access network device through the NR interface for transmitting user plane data and control plane signaling; the terminal device can establish a control plane signaling connection with the AMF through the NG interface 1 (referred to as N1); the access network device, such as the next generation wireless access base station (gNB), can establish a user plane data connection with the UPF through the NG interface 3 (referred to as N3); the access network device can establish a control plane signaling connection with the AMF through the NG interface 2 (referred to as N2); the UPF can establish a control plane signaling connection with the SMF through the NG interface 4 (referred to as N4); the UPF can exchange user plane data with the data network through the NG interface 6 (referred to as N6); the AMF can establish a control plane signaling connection with the SMF through the NG interface 11 (referred to as N11); the SMF can establish a control plane signaling connection with the PCF through the NG interface 7 (referred to as N7).

[0049] Figure 1 exemplarily shows a network device, a core network device and two terminal devices. Optionally, the wireless communication system 100 may include multiple network devices and each network device may include other numbers of terminal devices within its coverage area, which is not limited in this embodiment of the present application.

[0050] It should be noted that Figure 1 is merely an example of a system applicable to this application. Of course, the methods described in the embodiments of this application can also be applied to other systems. Furthermore, the terms "system" and "network" are often used interchangeably herein. The term "and / or" herein simply describes an association relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " generally indicates that the associated objects are in an "or" relationship. It should also be understood that the "indication" mentioned in the embodiments of this application can be a direct indication, an indirect indication, or an indication of an association relationship. For example, "A indicates B" can mean that A directly indicates B, for example, B can obtain information through A; it can also mean that A indirectly indicates B, for example, A indicates C, and B can obtain information through C; or it can mean that A and B have an association relationship. It should also be understood that the "correspondence" mentioned in the embodiments of this application can mean that there is a direct or indirect correspondence between two objects, or that there is an association relationship between the two objects, or a relationship between an indicator and the indicated, a configuration and the configured, and so on. It should also be understood that the “predefined” or “predefined rules” mentioned in the embodiments of the present application can be implemented by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in devices (for example, including terminal devices and network devices), and the present application does not limit its specific implementation method. For example, predefined can refer to what is defined in the protocol. It should also be understood that in the embodiments of the present application, the “protocol” may refer to a standard protocol in the field of communications, such as LTE protocols, NR protocols, and related protocols used in future communication systems, and the present application does not limit this.

[0051] To facilitate understanding of the technical solutions of the embodiments of the present application, the relevant technologies of the embodiments of the present application are described below. The following relevant technologies can be arbitrarily combined with the technical solutions of the embodiments of the present application as optional solutions, and they all fall within the protection scope of the embodiments of the present application.

[0052] In practical applications, terminal device positioning methods may include the following categories:

[0053] UE-based positioning method: that is, the terminal device directly calculates the position of the target UE.

[0054] UE-assisted / LMF-based positioning method: The terminal device reports the measurement results to the LMF, and the LMF calculates the location of the target UE based on the collected measurement results.

[0055] Based on the NG-RAN node assisted positioning method: the base station reports the measurement results of the network transmission / reception point (TRP) to the LMF, and the LMF calculates the position of the target UE based on the collected measurement results.

[0056] It should be noted that multiple TRPs around the terminal device can participate in position positioning. A base station may be a TRP, and a base station may have multiple TRPs under it. The LMF can be the positioning server, responsible for the entire positioning process.

[0057] In traditional positioning methods, the terminal device or LMF applies traditional algorithms, such as the Chan algorithm, Taylor expansion, etc., to estimate the position of the terminal device.

[0058] The following introduces the downlink-based positioning method and the uplink-based positioning method respectively.

[0059] As shown in FIG2A , the downlink-based positioning method may include the following steps:

[0060] Step 1: The LMF network element notifies the TRP of the relevant configuration.

[0061] The relevant configuration may include PRS configuration information, and / or information such as the type of measurement results that the terminal device needs to report.

[0062] Step 2: TRP sends PRS.

[0063] Step 3: The terminal device receives the positioning signal PRS and performs measurement.

[0064] It should be noted that different positioning methods require different measurement results from the terminal device.

[0065] Step 4: The terminal device feeds back the measurement results to the LMF.

[0066] The terminal device feeds the measurement results back to the LMF through the base station.

[0067] Step 5: LMF calculates the location-related information.

[0068] It should be noted that the above is a schematic diagram of the UE-assisted positioning method. For terminal device-based positioning methods (UE-based), in step 4 above, the terminal device directly calculates the location-related information based on the measurement results, without reporting the measurement results to the LMF, which then performs the calculation. In UE-based positioning methods, the terminal device needs to know the location information corresponding to the TRP, so the LMF needs to notify the terminal device of the location information corresponding to the TRP in advance.

[0069] In addition, referring to FIG2B , the uplink-based positioning method may include the following steps:

[0070] Step 1. LMF notifies TRP of relevant configurations.

[0071] Step 2: The base station sends relevant signaling to the terminal device.

[0072] Step 3: The terminal device sends a sounding reference signal (SRS).

[0073] Step 4: TRP measures the SRS and sends the measurement results to LMF.

[0074] Step 5: LMF calculates location-related information

[0075] In recent years, artificial intelligence research, represented by neural networks, has achieved remarkable results in many fields and will continue to play an important role in people's production and daily lives for a long time to come. A neural network is a computational model composed of multiple interconnected neuron nodes. Figure 3 shows a schematic diagram of a neuron structure. As shown in Figure 3, a neuron structure can be connected to other neuron structures a1 to an. The transmission of signals between neuron structures is affected by weights (for example, the weight value of the signal input to neuron structure a1 is w1). Each neuron structure can perform a weighted summation of multiple input signals and output them through a specific activation function.

[0076] Figure 4 is a schematic diagram of a neural network structure proposed in related art. As shown in Figure 4, a neural network structure can include an input layer, a hidden layer, and an output layer. As shown in Figure 4, the input layer receives data, the hidden layer processes the data, and the output layer produces the final result. Each node represents a processing unit, which can be thought of as simulating a neuron. Multiple neurons form a layer of a neural network. Multiple layers of information transmission and processing construct a complete neural network.

[0077] With the continuous development of neural network research, neural network deep learning algorithms have been proposed in recent years. More hidden layers have been introduced, and feature learning has been performed layer by layer through multi-hidden layer neural network training, which has greatly improved the learning and processing capabilities of neural networks. It has also been widely used in pattern recognition, signal processing, optimization combination, anomaly detection and other aspects.

[0078] Similarly, with the development of deep learning, convolutional neural networks (CNN) have also been further studied.

[0079] Figure 5 is a schematic diagram of a convolutional neural network structure provided by related art. As shown in Figure 5, the structure of a convolutional neural network can include: an input layer, multiple convolutional layers, multiple pooling layers, a fully connected layer, and an output layer. The introduction of convolutional and pooling layers effectively controls the rapid increase in network parameters, limits the number of parameters, and exploits the characteristics of local structures, improving the robustness of the algorithm.

[0080] Recurrent neural networks have achieved remarkable results in natural language processing applications such as machine translation and speech recognition. A recurrent neural network models sequential data, memorizing information from past moments and applying it to current output calculations. Specifically, the nodes in the hidden layers are connected, rather than disconnected, and the hidden layer inputs include not only the input layer but also the output of the previous hidden layer.

[0081] Figure 6 is a schematic diagram of the structure of a long short-term memory (LSTM) network provided by related technologies. LSTM is a commonly used type of recurrent neural network. Unlike recurrent neural networks that only consider the most recent state, LSTM determines which states should be retained and which states should be forgotten, thus solving the defects of traditional recurrent neural networks in long-term memory.

[0082] Given the tremendous success of artificial intelligence (AI) technologies or machine learning (ML) technologies such as neural networks and deep learning in computer vision and natural language processing, the communications field has begun to try to use AI technologies to seek new technical ideas to solve technical problems that are limited by traditional methods.

[0083] In practical applications, AI / ML technologies can be used to implement functions such as terminal device positioning and channel estimation. Specifically, the terminal device's location or channel estimation results are determined based on the AI / ML model. When the AI / ML model is no longer suitable for the current scenario, meaning its processing accuracy does not meet actual requirements, the AI / ML model needs to be updated. The key to updating the AI / ML model lies in monitoring its performance.

[0084] Based on this, an embodiment of the present application provides a model monitoring method. FIG6 is a flow chart of the model monitoring method provided by an embodiment of the present application. As shown in FIG7 , the method includes the following contents.

[0085] Step 710: The first device determines the performance of the first model based on the relationship between the estimated information and the actual information of the first terminal device, wherein the estimated information of the first terminal device is obtained by processing according to the first model.

[0086] It should be understood that a first model can be deployed in the first device. The first model can be a pre-trained AI / ML model, such as a neural network model, a CNN model, an LSTM model, etc., which is not limited in this embodiment of the present application. The first model can be used to determine estimated information of the first terminal device. The first model can process input information and ultimately output estimated information of the first terminal device.

[0087] Optionally, the estimated information may be estimated position information of the first terminal device, or estimated channel state information between the first terminal device and the base station, etc., which is not limited in the embodiments of the present application.

[0088] In addition, the actual information of the first terminal device can be stored in the local storage space of the first device, or the actual information of the first terminal device can be obtained from other devices.

[0089] Optionally, the actual information of the first terminal device may be the actual location information of the first terminal device, or the actual channel state information between the first terminal device and the base station, etc., which is not limited in the embodiment of the present application.

[0090] In an embodiment of the present application, the first device may compare the estimated information of the first terminal device determined by the first model with the actual information of the first terminal device, and determine the performance of the first model based on the relationship between the estimated information and the actual information. For example, if the estimated information is different from the actual information, it can be determined that the performance of the first model is poor. If the estimated information and the actual information are the same, it can be determined that the performance of the first model is good. Alternatively, if the difference between the estimated information and the actual information is large (for example, greater than a first threshold), it is determined that the performance of the first model is poor; if the difference between the estimated information and the actual information is small (for example, less than a first threshold), it is determined that the performance of the first model is good. When it is determined that the performance of the first model is poor, the first model can be updated, or the first model can be switched.

[0091] It can be seen that in the model monitoring method provided in the embodiment of the present application, the first device can compare the estimated information calculated by the first model with the actual information, and determine the performance of the first model based on the relationship between the estimated information and the actual information, so that when the performance of the first model does not meet the actual needs, the first model can be switched and updated in time to ensure the normal operation of the business.

[0092] Optionally, the first device may be a first terminal device, a first network device, or a second network device.

[0093] The first terminal device may be a terminal device whose actual information is known. For example, the first terminal device may be a terminal device of a type with a known location, such as a Positioning Reference Unit (PRU).

[0094] It should be noted that the PRU can perform downlink positioning-related measurements (e.g., downlink signal time difference, reference signal received power, UE-side receive and transmit signal Rx-Tx time difference, etc.) and report these measurement results to the positioning server. In addition, the PRU can send an SRS, and the base station performs measurements based on the SRS and reports uplink positioning measurement results (e.g., uplink relative arrival time, uplink arrival angle, base station-side Rx-Tx time difference, etc.).

[0095] The first network device can be an access network device, such as a base station, a micro base station, a TRP, etc., and this embodiment of the present application does not limit this.

[0096] The second network device may be a core network device, such as a location management function (LMF) network element, or other network elements configured with a location management function, which is not limited in this embodiment of the present application.

[0097] In the embodiment of the present application, when the first device is a first network device, the actual information of the first terminal device may be sent by the second network device;

[0098] In the case where the first device is the second network device, the actual information of the first terminal device is known information;

[0099] In the case that the first device is a first terminal device, the actual information of the first terminal device may be known information or sent by the second network device.

[0100] For example, the second network device can locally store the actual information of the first terminal device, or the actual information of the first terminal device can be reported by the first terminal device to the second network device. In this way, when other devices need to monitor the model, the second network device can send the actual information of the first terminal device to the other devices so that the other devices can adjust the first model.

[0101] Optionally, with reference to FIG8 , in an embodiment of the present application, before the first device determines the performance of the first model based on the relationship between the estimated information and the actual information of the first terminal device in step 710, the model monitoring method provided in the embodiment of the present application may further include the following steps:

[0102] Step 720: The first device obtains a measurement result associated with the first terminal device;

[0103] Step 730: The first device determines estimation information based on the measurement result using the first model.

[0104] It should be understood that the input of the first model may be a measurement result associated with the first terminal device, and the output of the first model may be estimation information of the first terminal device.

[0105] Before determining the estimated information of the first terminal device, the first device may obtain a measurement result associated with the first terminal device. After obtaining the measurement result, the measurement result is used as input to the first model, and the output of the first model is the estimated information of the first terminal device.

[0106] Exemplarily, the measurement results associated with the first terminal device may include downlink measurement results and / or uplink measurement results. The first terminal device may perform downlink-related measurements and report the obtained downlink measurement results to the second network device. In addition, the first terminal device may also send an uplink reference signal, and the first network device measures the uplink reference signal sent by the first terminal device to obtain an uplink measurement result. In addition, the first network device may report the uplink measurement result to the second network device.

[0107] Optionally, the measurement result may include one or more of the following:

[0108] Channel Impulse Response (CIR), Power Delay Profile (PDP), Time of Arrival (ToA), Uplink Time Difference of Arrival (UL TDoA), Downlink Time Difference of Arrival (DL TDoA), Uplink Angle-of-Departure (UL AoD), Downlink Angle-of-Departure (DL AoD), Uplink Relative Time of Arrival (UL RTOA), Downlink Reference Signal Time Difference (DL RSTD), Reference Signal Receiving Power (RSRP), Azimuth, Zenith Angle.

[0109] It should be noted that the measurement result associated with the first terminal device may be a measurement result between the first terminal device and the first network device. Exemplarily, the measurement result may include CIR, PDP, ToA, UL TDoA, DL TDoA, UL AoD, DL AoD, UL RTOA, DL RSTD, RSRP, azimuth, zenith angle, etc. between the first terminal device and the TRP.

[0110] The measurement result associated with the first terminal device may be for a single first network device or for multiple first network devices. In other words, the number of measurement results may include one or more. When the number of measurement results includes multiple measurement results, the multiple measurement results may be measurement results between the first terminal device and multiple first network devices.

[0111] Exemplarily, multiple measurement results may correspond to n TRPs, where the multiple measurement results may be the CIR of the first terminal device relative to the n TRPs, the RSRP of the first terminal device relative to the n TRPs, the DL RSTD of the first terminal device relative to the n TRPs, the ToA of the first terminal device relative to the n TRPs, and the AoD of the first terminal device relative to the n TRPs. The number of DL TDOA, RSRP, DL RSTD, TOA, and AoD corresponding to each TRP may be greater than 1.

[0112] Optionally, in an embodiment of the present application, when the first device is a first network device and / or a second network device, before the first device obtains the measurement result associated with the first terminal device in step 720, the model monitoring method provided in the embodiment of the present application may further include the following steps:

[0113] The first device sends first information to the first terminal device; the first information is used to configure relevant information monitored by the first model; and the measurement result is obtained based on the relevant information.

[0114] It is understandable that before obtaining the measurement results, the first device can also configure relevant information about the first model monitoring so that the relevant devices can perform measurements based on the configured relevant information (for example, the first terminal device performs downlink-related measurements, and the first network device performs uplink-related measurements) to obtain the above-mentioned measurement results.

[0115] In one possible implementation, when the first device is a first network device, the first network device can receive relevant information about the first model monitoring sent by the second network device, and forward the relevant information to the first terminal device. Exemplarily, the first network device can configure the relevant information about the first model monitoring to the first terminal device through high-layer signaling (such as RRC signaling) or physical layer signaling (such as downlink control information DCI), and this embodiment of the present application does not limit this.

[0116] In another possible implementation, when the first device is a second network device, the second network device may configure the relevant information monitored by the first model to the first terminal device via the Lightweight Presentation Protocol (LPP). Alternatively, the second network device may send the relevant information monitored by the first model to the first network device, and the first network device may then forward the relevant information to the first terminal device.

[0117] Optionally, the first information may include one or more of the following:

[0118] Reference signal type, measurement type, trigger information.

[0119] The trigger information is used to trigger performance monitoring of the first model.

[0120] Optionally, the reference signal type may include one or more of the following:

[0121] Positioning Reference Signal (PRS), Synchronization Signal and PBCH Block (SSB), Sounding Reference Signal (SRS).

[0122] Optionally, the measurement type may include any of:

[0123] Periodic measurement, non-periodic measurement, semi-continuous measurement, and time window measurement.

[0124] Among them, for periodic measurement, the first device can configure the time period for measuring the reference signal during the first model monitoring process, as well as the number of measurements in each time period, etc., and the embodiment of the present application does not limit this.

[0125] Aperiodic measurement can be understood as measuring the reference signal only a preset number of times. For aperiodic measurement, the first device can configure the number of times to measure the reference signal.

[0126] For semi-continuous measurement, the first device may configure information such as a time period for continuous measurement, a measurement cycle within the time period for continuous measurement, and the number of measurements in each cycle.

[0127] For time window measurement, the first device may configure information such as the starting position of the time window, the length of the time window, and the number of measurements within the time window.

[0128] Optionally, in another embodiment of the present application, when the first device is a first terminal device, before the first device obtains the measurement result associated with the first terminal device in step 720, the model monitoring method provided in the embodiment of the present application may further include the following steps:

[0129] The first device receives first information sent by the second network device; the first information is used to configure relevant information monitored by the first model, and the measurement result is obtained based on the relevant information.

[0130] It is understandable that before obtaining the measurement results, the first device can also configure relevant information about the first model monitoring so that the relevant devices can perform measurements based on the configured relevant information (for example, the first terminal device performs downlink-related measurements, and the first network device performs uplink-related measurements) to obtain the above-mentioned measurement results.

[0131] Optionally, the second network device may configure the relevant information monitored by the first model to the first terminal device via LPP. Alternatively, the second network device may send the relevant information monitored by the first model to the first network device, and the first network device may forward the relevant information to the first terminal device. This embodiment of the present application does not limit this.

[0132] Optionally, the first information may include one or more of the following:

[0133] Reference signal type, measurement type, trigger information, wherein the trigger information is used to trigger performance monitoring for the first model.

[0134] Optionally, the reference signal type may include one or more of the following:

[0135] PRS, SSB, SRS.

[0136] Optionally, the measurement type may include any of:

[0137] Periodic measurement, non-periodic measurement, semi-continuous measurement, and time window measurement.

[0138] The following describes this with examples.

[0139] When the first device is a first network device (e.g., TRP), in a possible implementation, the relevant information of the first model monitoring configured by the first network device to the first terminal device may include SRS. After receiving the relevant information of the first model monitoring, the first terminal device may send SRS. In this way, the first network device can measure the uplink SRS sent by the first terminal device to obtain a measurement result related to uplink positioning. At this time, the measurement result may include one or more of CIR, PDP, ToA, UL TDoA, UL AoD, UL RTOA, RSRP, azimuth, and zenith angle.

[0140] Exemplarily, when the positioning method is a combination of UL TDOA and AI, the measurement result of the first terminal device is the UL TDOA, RSRP, UL RTOA, etc. of the first terminal device for the first network device. The measurement result may also be a combination of UL TDOA and RSRP, or a combination of UL RTOA and RSRP. When the positioning method is a combination of UL AoA and AI, the measurement result of the first terminal device is the uplink arrival angle of the first terminal device for the first network device, such as azimuth and / or zenith.

[0141] In another possible implementation, the first network device configures the first terminal device with the relevant information of the first model monitoring, which may include PRS and SSB. The first terminal device may measure the PRS or SSB sent by the first network device based on the relevant information of the first model monitoring to obtain downlink-related measurement results. Secondly, the first terminal device may report the measurement results to the first network device. At this time, the measurement results may include one or more of CIR, PDP, ToA, DL TDoA, DL AoD, DL RSTD, RSRP, azimuth, and zenith angle.

[0142] Exemplarily, when the positioning method is a combination of DL TDOA and AI, the measurement result of the first terminal device may be the DL TDOA, RSRP, DL RSTD, TOA, etc. of the first terminal device relative to the first network device. The measurement result may also be a combination of DL TDOA and RSRP, or a combination of DL RSTD and RSRP, or a combination of TOA and RSRP. When the positioning method is a combination of DL AoD angle of departure positioning and AI, the measurement result of the first terminal device is the AoD of the first terminal device relative to the first network device.

[0143] Furthermore, after the first network device obtains the measurement result associated with the first terminal device, the measurement result can be input into the first model to obtain estimation information.

[0144] It can be seen from this that when the first model is deployed on the first network device, the characteristics of the first terminal device can be applied to determine the performance of the current first model, so that the model can be updated in a timely manner.

[0145] In other embodiments, when the first device is a second network device, in a possible implementation, the relevant information that the second network device configures the first model monitoring to the first terminal device may include PRS or SSB. The first terminal device may measure the PRS or SSB sent by the first network device to obtain a measurement result. The first terminal device may send the measurement result to the second network device. In this case, the measurement result may include one or more of CIR, PDP, ToA, DL TDoA, DL AoD, DL RSTD, RSRP, azimuth, and zenith angle.

[0146] Exemplarily, when the positioning method is a combination of DL TDOA and AI, the measurement result of the first terminal device may be the DL TDOA, RSRP, DL RSTD, TOA, etc. of the first terminal device relative to the first network device. The measurement result may also be a combination of DL TDOA and RSRP, or a combination of DL RSTD and RSRP, or a combination of TOA and RSRP. When the positioning method is a combination of DL AoD angle of departure positioning and AI, the measurement result of the first terminal device is the AoD of the first terminal device relative to the first network device.

[0147] In another possible implementation, the relevant information that the second network device configures for the first terminal device to monitor the first model may include an SRS. The first terminal device may send an SRS based on the relevant information monitored by the first model. In this way, the first network device may measure the SRS sent by the first terminal device to obtain an uplink-related measurement result. Furthermore, the first network device may forward the measurement result to the second network device. In this case, the measurement result may include one or more of CIR, PDP, ToA, UL TDoA, UL AoD, UL RTOA, RSRP, azimuth, and zenith angle.

[0148] Exemplarily, when the positioning method is a combination of UL TDOA and AI, the measurement result of the first terminal device is the UL TDOA, RSRP, UL RTOA, etc. of the first terminal device relative to the first network device. The measurement result may also be a combination of UL TDOA and RSRP, or a combination of UL RTOA and RSRP. When the positioning method is a combination of UL AoA and AI, the measurement result of the first terminal device is the uplink arrival angle of the first terminal device relative to the first network device, such as the azimuth angle and / or zenith angle.

[0149] Furthermore, after obtaining the measurement result associated with the first terminal device, the second network device may input the measurement result into the first model to obtain estimation information.

[0150] Optionally, when the first device is a first terminal device, the first terminal device may measure a reference signal sent by the first network device to obtain a measurement result. The first terminal device may directly input the measurement result into the first model to obtain estimation information. In this scenario, the measurement result may include only one or more of CIR, PDP, and RSRP.

[0151] It should be noted that the measurement behavior in the above embodiment can be understood as monitoring the performance of the first model.

[0152] The monitoring of the performance of the first model by the first device may be spontaneous. For example, when the first device is a first terminal device, the first terminal device may actively measure the received reference signal, thereby realizing the monitoring of the first model. The monitoring of the performance of the first model by the first device may also be triggered by other devices. For example, when the first device is a first terminal device, the first terminal device may receive relevant information of the first model monitoring sent by the first network device and / or the second network device, thereby triggering the monitoring of the first model.

[0153] It should be noted that the first model can be used directly to determine the estimated information of the first terminal device, and can also be used to assist in determining the estimated information of the first terminal device. Exemplarily, the first model is used as the positioning model for illustration. Figure 9A shows a schematic diagram of the direct positioning principle of the first model, wherein the input of the first model is the measurement result in the above embodiment, and the output of the first model is the estimated position information of the first terminal device. Figure 9B shows a schematic diagram of the assisted positioning principle of the first model, wherein the input of the first model is the measurement result, and the output of the first model may include intermediate parameters, which are processed in combination with the positioning algorithm to obtain the estimated position information of the first terminal device.

[0154] In some embodiments, the first device is a first terminal device. In step 730, the first device determines the estimation information based on the measurement result using the first model. This can also be achieved in the following manner:

[0155] The first device determines the intermediate parameter based on the measurement result using the first model;

[0156] The first device sends the intermediate parameter to the second network device;

[0157] The first device receives the estimation information sent by the second network device.

[0158] Optionally, the intermediate parameters may include one or more of the following:

[0159] ToA, DL TDoA, DL AoD, DL RSTD, Line of Sight (LOS) / Non-Line of Sight (NLOS) recognition results.

[0160] In this embodiment of the present application, due to the limited processing resources and capabilities of the first terminal device, the first terminal device can utilize the first model to assist in determining the first terminal device's estimated information. Specifically, the first terminal device can utilize the first model to calculate intermediate parameters. The intermediate parameters output by the first model are then transmitted to the second network device. The second network device can then calculate the first terminal device's estimated information based on the intermediate parameters and transmit the resulting estimated information to the first terminal device. This allows the first terminal device to complete the determination of the estimated information, thereby reducing the processing overhead of the first terminal device.

[0161] In one embodiment of the present application, in step 710, the first device determines the performance of the first model based on the relationship between the estimated information and the actual information of the first terminal device, which can be achieved by:

[0162] When the difference value is less than or equal to the first threshold, determining that the first model is valid;

[0163] When the difference value is greater than a first threshold, switching the first model to the second model, or falling back to the traditional method of determining the estimation information;

[0164] The difference value represents the difference between the estimated information and the actual information.

[0165] It is understood that the first device can compare the difference between the estimated information and the actual information and calculate the difference value between the estimated information and the actual information of the first terminal device. For example, the estimated information is the estimated location information of the first terminal device, and the actual information is the actual location information of the first terminal device. The first device can calculate the distance between the estimated location information and the actual location information, and the difference value can be understood as the distance between the two.

[0166] If the difference value is small, for example, less than or equal to the first threshold, it means that the estimation information calculated by the first model is relatively accurate, the performance of the first model is good, there is no need to switch models, and the first model can continue to be used to process business.

[0167] If the difference value is large, for example, greater than the first threshold, it means that the estimation information calculated by the first model is inaccurate, the performance of the first model is poor, and it can no longer meet the current needs. It is necessary to switch the first model or fall back to the traditional method of determining the estimation information.

[0168] Optionally, the first threshold may be determined according to one or more of a network device configuration, a preset rule, and a protocol definition.

[0169] In the case where the first device is a first terminal device or a first network device, the second network device may configure the first threshold for the first terminal device or the first network device. The second network device may configure the first threshold for the first terminal device via the LPP protocol, or the second network device may configure the first threshold for the first network device, and the first network device may configure the first threshold for the first terminal device via high-layer signaling or physical layer signaling.

[0170] In addition, the first device may calculate the first threshold according to a preset rule. The protocol definition may be a value specified by the protocol, for example, the first threshold may be 0.1, 0.2, etc., which is not limited in the embodiment of the present application.

[0171] Optionally, the first device may switch the first model to a second model. The second model may be a new model formed after the model parameters in the first model are updated. The second model may also be a model obtained through offline training, a model obtained through online training, or a model stored in a model pool. This embodiment of the present application does not limit this.

[0172] Optionally, if the first model is a positioning model, the traditional way of determining the estimation information may be a traditional positioning method, for example, Chan algorithm, Tate-Taylor expansion algorithm, etc. to estimate the position of the terminal device.

[0173] For example, referring to FIG10A , when the first device is an LMF / TRP, the LMF / TRP can obtain measurement results of the first terminal device and input the measurement results into the first model. The first model can output estimated information of the first terminal device. Furthermore, the LMF / TRP can calculate the difference between the estimated information and the actual information. If the difference is less than or equal to a first threshold, the first model is determined to be valid. If the difference is greater than the first threshold, the first model is switched or reverted.

[0174] As shown in Figure 10B, when the first device is a first terminal device, the first terminal device can obtain measurement results and input the measurement results into the first model. The first model can output intermediate parameters. The first terminal device can send the intermediate parameters to the LMF for processing. After the LMF determines the estimated information based on the terminal parameters, it sends the estimated information to the first terminal device. In this way, the first terminal device can calculate the difference between the estimated information and the actual information. If the difference is less than or equal to the first threshold, it is determined that the first model is still valid. If the difference is greater than the first threshold, the first model is switched or rolled back.

[0175] To sum up, considering that the actual information of ordinary terminal devices is not easy to obtain and it is difficult to monitor the performance of its own model, the actual information and estimated information of the first terminal device are used to monitor the first model, and the network device forwards the model monitoring results to determine the quality of the first model.

[0176] In another embodiment of the present application, in step 710, the first device determines the performance of the first model based on the relationship between the estimated information and the actual information of the first terminal device, which can also be achieved in the following manner:

[0177] If the number of times that the difference value is less than or equal to the first threshold within the first time period is greater than the second threshold, determining that the first model is valid;

[0178] If the number of times the difference value is less than or equal to the first threshold within the first time period is less than or equal to the second threshold, switching the first model to the second model, or reverting to the traditional method of determining estimation information;

[0179] The difference value represents the difference between the estimated information and the actual information.

[0180] It should be understood that in order to avoid frequent model switching of the first device, the first device can monitor the difference value between multiple estimated information output by the first model and the actual information over a period of time. If the number of times the difference value is less than or equal to the first threshold is greater than the second threshold, the first model is determined to be valid; if the number of times the difference value is less than or equal to the first threshold is less than or equal to the second threshold, the first model is switched to the second model, or the method of determining the estimated information is returned to the traditional method.

[0181] Optionally, the first threshold and / or the second threshold may be determined according to one or more of a network device configuration, a preset rule, and a protocol definition.

[0182] Optionally, the first duration may be a time length defined by a protocol, such as 2 seconds, 5 seconds, etc.

[0183] Optionally, the first duration may also be determined based on one or more of the following:

[0184] The period of monitoring of the first model;

[0185] The time window of the first model monitoring.

[0186] It should be understood that if the second network device configures a monitoring period or monitoring time window of the first model, the first duration can be determined by the period and time window configured by the second network device. Exemplarily, the first duration can be the period monitored by the first model, or a multiple of the period, and the first duration can be the length of the time window monitored by the first model, or a multiple of the length of the time window. This embodiment of the application does not impose any restrictions on this.

[0187] For example, referring to FIG11A , when the first device is an LMF / TRP, the LMF / TRP may first initialize a counter value to 0. Next, the LMF / TRP may obtain the measurement results of the first terminal device and input the measurement results into the first model. The first model may output estimated information of the first terminal device. Furthermore, the LMF / TRP may calculate the difference between the estimated information and the actual information. If the difference is less than or equal to a first threshold, the counter value is incremented by 1. If the counter value is greater than a second threshold within a first time period, it is determined that the first model is still valid. If the counter value is less than or equal to the first threshold, the first model is switched or rolled back.

[0188] As shown in reference figure 11B, when the first device is a first terminal device, the first terminal device can initialize the counter value to 0. Secondly, the first terminal device can obtain multiple measurement results within a first time period and input each measurement result into the first model. The first model can output intermediate parameters. The first terminal device can send the intermediate parameters to the LMF for processing. After the LMF determines the estimated information based on the terminal parameters, it sends the estimated information to the first terminal device. In this way, the first terminal device can calculate the difference between the estimated information and the actual information. If the difference is less than or equal to the first threshold, the counter value is set to be plus 1. When the first time period is reached, it is determined that the counter value is greater than the second threshold, and it is determined that the first model is still valid. If the counter value is less than or equal to the first threshold, the first model is switched or rolled back.

[0189] In summary, the first device can monitor the difference between multiple estimated information output by the first model and the actual information over a period of time. If the number of times the difference is less than or equal to the first threshold is greater than a second threshold, the first model is determined to be valid. If the number of times the difference is less than or equal to the first threshold is less than or equal to the second threshold, the first model is switched to the second model, or the traditional method of determining estimated information is reverted. This prevents the first device from frequently switching models and improves stability during business processing.

[0190] Optionally, after the first device switches the first model to the second model or falls back to the mode of transmitting determined estimation information, it may also inform other devices of relevant information of the model switching, so that related devices can make timely adjustments.

[0191] In some embodiments, the first device is a first network device and / or a second network device. After switching the first model to the second model or reverting to the traditional method of determining estimation information, the following steps may be further performed:

[0192] The first device sends second information to the terminal device, where the second information is used to indicate a second model, or a traditional way of determining estimation information; the terminal device includes the first terminal device.

[0193] It should be understood that the first network device and / or the second network device can send second information to all terminal devices within the network, instructing them to subsequently use the second model or a traditional method to determine the estimated information. Accordingly, the terminal devices can adaptively adjust information such as the reference signal measured during the model monitoring process and the monitoring type based on the second information.

[0194] In some embodiments, the first device is a first terminal device. After switching the first model to the second model, or reverting to the traditional method of determining estimation information, the following steps may be further performed:

[0195] The first device sends second information to the second network device, where the second information is used to indicate a second model, or a traditional way of determining estimation information.

[0196] It is understood that when the first device is a first terminal device, the first terminal device can send the switching result of the first model to the second network device. In this way, the second network device can also forward the second information to all terminal devices in the network. Accordingly, the terminal device can adaptively adjust the reference signal measured during the model monitoring process, the monitoring type, and other information based on the second information.

[0197] Optionally, the second information may include one or more of the following:

[0198] identification information of the second model;

[0199] Reference signal configuration information associated with the second model;

[0200] Identification information of the traditional way of determining estimation information;

[0201] The conventional method of determining the estimation information is associated with the reference signal configuration information.

[0202] It should be noted that for different models and different ways of determining estimation information, the reference signals that the terminal device needs to measure are different. Based on this, when the model for determining the estimation information changes, the first network device and / or the second network device can inform the terminal device of the identification information of the second model after switching, or the reference signal configuration information. When it is determined to fall back to the transmission method, the first network device and / or the second network device can inform the terminal device of the identification information of the traditional way of determining the estimation information, or the reference signal configuration information associated with the traditional way of determining the estimation information.

[0203] The preferred embodiments of the present application are described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the above embodiments. Within the technical concept of the present application, the technical solution of the present application can be subjected to a variety of simple modifications, and these simple modifications all fall within the scope of protection of the present application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present application will no longer describe the various possible combinations separately. For another example, the various different embodiments of the present application can also be arbitrarily combined, as long as they do not violate the idea of ​​the present application, they should also be regarded as the contents disclosed in the present application. For another example, under the premise of no conflict, the various embodiments and / or the technical features in each embodiment described in the present application can be arbitrarily combined with the prior art, and the technical solution obtained after the combination should also fall within the scope of protection of the present application.

[0204] It should also be understood that in the various method embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0205] FIG12 is a schematic diagram of the structure of a model monitoring device 1200 provided in an embodiment of the present application, which is applied to a first device. As shown in FIG12 , the model monitoring device 1200 includes:

[0206] The determining unit 1201 is configured to determine the performance of the first model according to the relationship between the estimated information and the actual information of the first terminal device; the estimated information of the first terminal device is obtained by processing according to the first model.

[0207] Optionally, the model monitoring apparatus 1200 further includes an acquisition unit configured to acquire a measurement result associated with the first terminal device;

[0208] The determining unit 1201 is further configured to determine the estimation information based on the measurement result by using the first model.

[0209] Optionally, the first device is a first network device and / or a second network device, and the model monitoring device 1200 may further include a transceiver unit configured to send first information to the first terminal device; the first information is used to configure relevant information for monitoring the first model; and the measurement result is obtained based on the relevant information.

[0210] Optionally, the transceiver unit is further configured to receive first information sent by a second network device; the first information is used to configure relevant information monitored by the first model, and the measurement result is obtained based on the relevant information.

[0211] Optionally, the first information includes one or more of the following:

[0212] Reference signal type, measurement type, trigger information, where the trigger information is used to trigger performance monitoring for the first model.

[0213] Optionally, the reference signal type includes:

[0214] Positioning reference signal PRS, synchronization signal block SSB, and sounding reference signal SRS.

[0215] Optionally, the measurement type includes:

[0216] Periodic measurement, non-periodic measurement, semi-continuous measurement, and time window measurement.

[0217] Optionally, the number of the measurement results is 1, and the measurement result is a measurement result between the first terminal device and the first network device; or,

[0218] The measurement results include multiple measurement results, and the multiple measurement results are measurement results between the first terminal device and multiple first network devices.

[0219] Optionally, the measurement result includes one or more of the following:

[0220] Channel impulse response (CIR), power delay profile (PDP), time of arrival (ToA), uplink time difference of arrival (UL TDoA), downlink time difference of arrival (DL TDoA), uplink angle of departure (UL AoD), downlink angle of departure (DL AoD), uplink relative time of arrival (UL RTOA), downlink signal time difference (DL RSTD), reference signal received power (RSRP), azimuth angle, and zenith angle.

[0221] Optionally, when the first device is a second network device, the measurement result includes one or more of the following:

[0222] CIR, PDP, ToA, DL TDoA, DL AoD, DL RSTD, RSRP, azimuth, zenith angle.

[0223] Optionally, when the first device is a first network device, the measurement result includes one or more of the following:

[0224] CIR, PDP, ToA, UL TDoA, UL AoD, UL RTOA, RSRP, azimuth, zenith angle.

[0225] Optionally, when the first device is the first terminal device, the measurement result includes one or more of the following:

[0226] CIR, PDP, RSRP.

[0227] Optionally, the first device is the first terminal device, and the determining unit 1201 is further configured to determine an intermediate parameter based on the measurement result by using the first model;

[0228] The transceiver unit is further configured to send the intermediate parameter to the second network device and receive the estimation information sent by the second network device.

[0229] Optionally, the intermediate parameters include one or more of the following:

[0230] ToA, DL TDoA, DL AoD, DL RSTD, line-of-sight / non-line-of-sight LOS / NLOS identification results.

[0231] Optionally, the determining unit 1201 is further configured to, if the difference value is less than or equal to a first threshold, determine that the first model is valid; if the difference value is greater than the first threshold, switch the first model to the second model, or fall back to a traditional method of determining the estimation information;

[0232] The difference value represents the difference between the estimated information and the actual information.

[0233] Optionally, the determining unit 1201 is further configured to determine that the first model is valid if the number of times the difference value is less than or equal to the first threshold is greater than a second threshold within the first time period;

[0234] If the number of times the difference value is less than or equal to the first threshold within the first time period is less than or equal to a second threshold, switching the first model to the second model, or reverting to the traditional method of determining the estimation information;

[0235] The difference value represents the difference between the estimated information and the actual information.

[0236] Optionally, the first duration is determined based on one or more of the following:

[0237] a period monitored by the first model;

[0238] The first model monitors a time window.

[0239] Optionally, the first threshold and / or the second threshold is determined according to at least one of the following:

[0240] Network device configuration, preset rules, and protocol definitions.

[0241] Optionally, the transceiver unit is further configured to send second information to the terminal device, where the second information is used to indicate the second model, or a traditional method of determining the estimation information;

[0242] The terminal device includes the first terminal device.

[0243] Optionally, the transceiver unit is further configured to send second information to the second network device, where the second information is used to indicate the second model, or a traditional method of determining the estimation information.

[0244] Optionally, the second information includes one or more of the following:

[0245] identification information of the second model;

[0246] reference signal configuration information associated with the second model;

[0247] identification information of the conventional method for determining the estimation information;

[0248] The conventional method of determining the estimation information is associated with reference signal configuration information.

[0249] Optionally, when the first device is a first network device, the actual information of the first terminal device is sent by a second network device;

[0250] In the case where the first device is a second network device, the actual information of the first terminal device is known information;

[0251] In the case that the first device is the first terminal device, the actual information of the first terminal device is known information or is sent by the second network device.

[0252] Optionally, the first network device is an access network device, and the second network device is a core network device.

[0253] Optionally, the first network device is a TRP and the second network device is a LMF network element.

[0254] Optionally, the estimated information is estimated location information, and the actual information is actual location information.

[0255] Those skilled in the art should understand that the relevant description of the above-mentioned model monitoring device in the embodiment of the present application can be understood with reference to the relevant description of the model monitoring method in the embodiment of the present application.

[0256] FIG13 is a schematic structural diagram of an electronic device 1300 provided in an embodiment of the present application. The electronic device may be a first device. The electronic device 1300 shown in FIG13 includes a processor 1310, which can call and execute a computer program from a memory to implement the method in the embodiment of the present application.

[0257] Optionally, as shown in FIG13 , the electronic device 1300 may further include a memory 1320. The processor 1310 may call and execute a computer program from the memory 1320 to implement the method in the embodiment of the present application.

[0258] The memory 1320 may be a separate device independent of the processor 1310 , or may be integrated into the processor 1310 .

[0259] Optionally, the electronic device 1300 may specifically be the first device of the embodiment of the present application, and the electronic device 1300 may implement the corresponding processes implemented by the first device in each method of the embodiment of the present application. For the sake of brevity, they will not be repeated here.

[0260] Optionally, the electronic device 1300 may specifically be the second device of the embodiment of the present application, and the electronic device 1300 may implement the corresponding processes implemented by the second device in each method of the embodiment of the present application. For the sake of brevity, they will not be repeated here.

[0261] Optionally, the electronic device 1300 may specifically be the third device of the embodiment of the present application, and the electronic device 1300 may implement the corresponding processes implemented by the third device in each method of the embodiment of the present application. For the sake of brevity, they will not be repeated here.

[0262] Figure 14 is a schematic structural diagram of a chip according to an embodiment of the present application. The chip 1400 shown in Figure 14 includes a processor 1410, which can call and run a computer program from a memory to implement the method according to the embodiment of the present application.

[0263] Optionally, as shown in FIG14 , the chip 1400 may further include a memory 1420 , wherein the processor 1410 may call and execute a computer program from the memory 1420 to implement the method in the embodiment of the present application.

[0264] The memory 1420 may be a separate device independent of the processor 1410 , or may be integrated into the processor 1410 .

[0265] Optionally, the chip 1400 may further include an input interface 1430. The processor 1410 may control the input interface 1430 to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.

[0266] Optionally, the chip 1400 may further include an output interface 1440. The processor 1410 may control the output interface 1440 to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.

[0267] Optionally, the chip can be applied to the first device in the embodiment of the present application, and the chip can implement the corresponding processes implemented by the first device in each method of the embodiment of the present application. For the sake of brevity, it will not be repeated here.

[0268] Optionally, the chip can be applied to the second device in the embodiments of the present application, and the chip can implement the corresponding processes implemented by the second device in each method of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0269] Optionally, the chip can be applied to the third device in the embodiment of the present application, and the chip can implement the corresponding processes implemented by the third device in each method of the embodiment of the present application. For the sake of brevity, it will not be repeated here.

[0270] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0271] It should be understood that the processor of the embodiments of the present application may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiment can be completed by hardware integrated logic circuits in the processor or software instructions. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly implemented as a hardware decoding processor, or can be implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0272] It is understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0273] It should be understood that the above-mentioned memories are exemplary but not restrictive. For example, the memories in the embodiments of the present application may also be static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM RAM (DR RAM), etc. In other words, the memories in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0274] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0275] Optionally, the computer-readable storage medium can be applied to the network device in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the network device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0276] Optionally, the computer-readable storage medium can be applied to the mobile terminal / terminal device in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the mobile terminal / terminal device in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0277] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0278] Optionally, the computer program product can be applied to the network device in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the network device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0279] Optionally, the computer program product can be applied to the mobile terminal / terminal device in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the mobile terminal / terminal device in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0280] The embodiment of the present application also provides a computer program.

[0281] Optionally, the computer program can be applied to the network device in the embodiments of the present application. When the computer program runs on a computer, the computer executes the corresponding processes implemented by the network device in the various methods of the embodiments of the present application. For the sake of brevity, they are not described here.

[0282] Optionally, the computer program can be applied to the mobile terminal / terminal device in the embodiments of the present application. When the computer program runs on the computer, the computer executes the corresponding processes implemented by the mobile terminal / terminal device in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0283] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0284] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0285] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0286] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0287] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0288] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0289] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A model monitoring method, comprising: The first device determines the performance of the first model based on the relationship between the estimated information and the actual information of the first terminal device; The estimated information of the first terminal device is obtained by processing according to the first model.

2. The method according to claim 1, wherein Before the first device determines the performance of the first model according to the relationship between the estimated information and the actual information of the first terminal device, the method further includes: The first device obtains a measurement result associated with the first terminal device; The first device determines the estimation information based on the measurement result by using the first model.

3. The method according to claim 2, wherein: The first device is a first network device and / or a second network device, and the method further includes: The first device sends first information to the first terminal device; the first information is used to configure relevant information monitored by the first model; and the measurement result is obtained based on the relevant information.

4. The method according to claim 2, wherein: The first device is the first terminal device, and the method further includes: The first device receives first information sent by the second network device; the first information is used to configure relevant information monitored by the first model, and the measurement result is obtained based on the relevant information.

5. The method according to claim 3 or 4, wherein: The first information includes one or more of the following: Reference signal type, measurement type, trigger information, where the trigger information is used to trigger performance monitoring for the first model.

6. The method according to claim 5, wherein: The reference signal types include: Positioning reference signal PRS, synchronization signal block SSB, and sounding reference signal SRS.

7. The method according to claim 5, wherein: The measurement types include: Periodic measurement, non-periodic measurement, semi-continuous measurement, and time window measurement.

8. The method according to any one of claims 2 to 7, wherein: The number of the measurement results is 1, and the measurement result is a measurement result between the first terminal device and the first network device; or, The measurement results include multiple measurement results, and the multiple measurement results are measurement results between the first terminal device and multiple first network devices.

9. The method according to any one of claims 2 to 8, wherein: The measurement results include one or more of the following: Channel impulse response (CIR), power delay profile (PDP), time of arrival (ToA), uplink time difference of arrival (UL TDoA), downlink time difference of arrival (DL TDoA), uplink angle of departure (UL AoD), downlink angle of departure (DL AoD), uplink relative time of arrival (UL RTOA), downlink signal time difference (DL RSTD), reference signal received power (RSRP), azimuth angle, and zenith angle.

10. The method according to claim 8, wherein When the first device is a core network device, the measurement result includes one or more of the following: CIR, PDP, ToA, DL TDoA, DL AoD, DL RSTD, RSRP, azimuth, zenith angle.

11. The method according to claim 8, wherein When the first device is an access network device, the measurement result includes one or more of the following: CIR, PDP, ToA, UL TDoA, UL AoD, UL RTOA, RSRP, azimuth, zenith angle.

12. The method according to claim 8, wherein When the first device is the first terminal device, the measurement result includes one or more of the following: CIR, PDP, RSRP.

13. The method according to any one of claims 2 to 12, wherein: The first device is the first terminal device, and the first device determines the estimation information based on the measurement result by using the first model, including: The first device determines, by the first model, an intermediate parameter based on the measurement result; The first device sends the intermediate parameter to the second network device; The first device receives the estimation information sent by the second network device.

14. The method according to claim 13, wherein The intermediate parameters include one or more of the following: ToA, DL TDoA, DL AoD, DL RSTD, line-of-sight / non-line-of-sight LOS / NLOS identification results.

15. The method according to any one of claims 1 to 14, wherein: The first device determines the performance of the first model according to the relationship between the estimated information and the actual information of the first terminal device, including: When the difference value is less than or equal to a first threshold, determining that the first model is valid; When the difference value is greater than the first threshold, switching the first model to the second model, or falling back to the traditional method of determining the estimation information; The difference value represents the difference between the estimated information and the actual information.

16. The method according to any one of claims 1 to 14, wherein: The first device determines the performance of the first model according to the relationship between the estimated information and the actual information of the first terminal device, including: If the number of times that the difference value is less than or equal to the first threshold within the first time period is greater than the second threshold, determining that the first model is valid; If the number of times the difference value is less than or equal to the first threshold within the first time period is less than or equal to a second threshold, switching the first model to the second model, or reverting to the traditional method of determining the estimation information; The difference value represents the difference between the estimated information and the actual information.

17. The method according to claim 16, wherein The first duration is determined based on one or more of the following: a period monitored by the first model; The first model monitors a time window.

18. The method according to any one of claims 15 to 17, wherein: The first threshold and / or the second threshold is determined according to at least one of the following: Network device configuration, preset rules, and protocol definitions.

19. The method according to any one of claims 15 to 18, wherein: The first device is a first network device and / or a second network device. After switching the first model to the second model or returning to the traditional method of determining the estimation information, the method further includes: The first device sends second information to the terminal device, where the second information is used to indicate the second model, or a traditional method of determining the estimation information; The terminal device includes the first terminal device.

20. The method according to any one of claims 15 to 18, wherein: The first device is the first terminal device. After switching the first model to the second model or returning to the traditional method of determining the estimation information, the method further includes: The first device sends second information to the second network device, where the second information is used to indicate the second model, or a traditional way of determining the estimation information.

21. The method according to claim 19 or 20, wherein The second information includes one or more of the following: identification information of the second model; reference signal configuration information associated with the second model; identification information of the conventional method for determining the estimation information; The conventional method of determining the estimation information is associated with reference signal configuration information.

22. The method according to any one of claims 1 to 21, wherein: In the case where the first device is a first network device, the actual information of the first terminal device is sent by a second network device; In the case where the first device is a second network device, the actual information of the first terminal device is known information; In the case that the first device is the first terminal device, the actual information of the first terminal device is known information or is sent by the second network device.

23. The method according to claim 22, wherein The first network device is an access network device, and the second network device is a core network device.

24. The method according to claim 23, wherein The first network device is a sending transmission point TRP, and the second network device is a positioning management function LMF network element.

25. The method according to any one of claims 1 to 24, wherein: The estimated information is estimated position information, and the actual information is actual position information.

26. A model monitoring device, applied to a first device, comprising: a determining unit configured to determine performance of the first model based on a relationship between the estimated information and the actual information of the first terminal device; The estimated information of the first terminal device is obtained by processing according to the first model.

27. A device comprising: Memory, processors, and transceivers, The transceiver is used to realize communication with the terminal device; The memory stores a computer program executable on the processor. When the processor executes the program in conjunction with the transceiver, the method according to any one of claims 1 to 25 is implemented.

28. A computer storage medium storing one or more programs, wherein the one or more programs can be executed by one or more processors to implement the method according to any one of claims 1 to 25.

29. A chip, comprising: A processor, configured to call and run a computer program from a memory, so that a device equipped with the chip executes the method according to any one of claims 1 to 25.

30. A computer program product, comprising a computer storage medium storing a computer program, wherein the computer program comprises instructions executable by at least one processor, and when the instructions are executed by the at least one processor, the method according to any one of claims 1 to 25 is implemented.

31. A computer program, which enables a computer to execute the method according to any one of claims 1 to 25.

Images