Method for monitoring model performance and device for monitoring a model

Description

1 / 99 METHOD FOR MONITORING MODEL AND DEVICE PERFORMANCE FOR MONITORING A MODEL FIELD OF TECHNIQUE

[001] The modalities of the present disclosure relate to the technical field of mobile communication, particularly to a model monitoring method and device and to a communication device. BACKGROUND

[002] Given the great success of Artificial Intelligence (AI) / Machine Learning (ML) technology in computer vision, natural language processing, etc., AI / ML technology is beginning to be used in the field of communication to seek new technical ideas to solve the technical problems of limited traditional methods.

[003] How to monitor the performance of the AI / ML model in the field of communication is a matter of concern in the current field. SUMMARY

[004] The embodiments of the present disclosure provide a model monitoring method and device and a communication device to improve the feasibility and flexibility of model monitoring.

[005] One embodiment of the present disclosure provides a model monitoring method, and the method includes the following operation.

[006] The first device determines a similarity comparison result between an input dataset and a training dataset of a Petition 870250089278, dated 10 / 01 / 2025, page 9 / 252 2 / 99 first model. The result of the similarity comparison is used to determine the performance of the first model.

[007] One embodiment of the present disclosure provides a model monitoring device, which is applied to the first device, and the device includes a determination unit.

[008] The determination unit is configured to determine a similarity comparison result between an input dataset and a training dataset of a first model. The similarity comparison result is used to determine the performance of the first model.

[009] One embodiment of the present disclosure provides a communication device, and the communication device includes a processor and a memory. The memory is configured to store a computer program, and the processor is configured to invoke and execute the computer program stored in memory to accomplish the method mentioned above.

[0010] One embodiment of the present disclosure provides a chip, which is configured to implement the aforementioned model monitoring method.

[0011] Specifically, the chip includes a processor. The processor is configured to call and execute a computer program from memory to allow a device installed with the chip to perform the aforementioned model monitoring method.

[0012] One embodiment of the present disclosure provides a computer-readable storage medium that is configured to store a computer program. The Petition 870250089278, dated 10 / 01 / 2025, page 10 / 252 The 3 / 99 computer program causes the computer to execute the model monitoring method mentioned above.

[0013] One embodiment of the present disclosure provides a computer program product including computer program instructions. The computer program instructions cause a computer to execute the model monitoring method mentioned above.

[0014] One embodiment of the present disclosure provides a computer program that, when run on the computer, causes the computer to execute the model monitoring method mentioned above.

[0015] With the model monitoring method provided by the present disclosure embodiment, when the first device monitors the first model, the first device compares the similarity between the input dataset of the first model and the training dataset of the first model without collecting label information corresponding to the input data and determines the performance of the first model using the result of the similarity comparison. In this way, model monitoring is implemented in a scenario where label information corresponding to the input data cannot be collected, hence the feasibility and flexibility of model monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings described herein are used to provide a further understanding of the present disclosure and form part of the present disclosure. The schematic embodiments of the present disclosure and their description are used to explain the present disclosure and do not constitute Petition 870250089278, dated 10 / 01 / 2025, page 11 / 252 4 / 99 an undue limitation of the present disclosure. In the drawings:

[0017] Figure 1 is a schematic communication flowchart of a wireless communication system according to an embodiment of the present disclosure;

[0018] Figure 2A is a schematic principle diagram of a downward link-based positioning method according to an embodiment of the present disclosure;

[0019] Figure 2B is a schematic diagram of the principle of an upward link-based positioning method according to an embodiment of the present disclosure;

[0020] Figure 3 is a schematic structural diagram of a neuron according to the related technique;

[0021] Figure 4 is a schematic structural diagram of a neural network according to the related technique;

[0022] Figure 5 is a schematic structural diagram of a convolutional neural network according to the related technique;

[0023] Figure 6 is a schematic structural diagram of a Long Short Term Memory (LSTM) according to the state of the art;

[0024] Figure 7A is a schematic diagram of the direct positioning principle of the AI / ML model according to an embodiment of the present disclosure;

[0025] Figure 7B is a schematic principle diagram of the assisted positioning of the AI / ML model according to an embodiment of the present disclosure;

[0026] Figure 8 is a first schematic flowchart of a model monitoring method according to an embodiment of the present disclosure; Petition 870250089278, dated 10 / 01 / 2025, page 12 / 252 5 / 99

[0027] Figure 9 is a second schematic flowchart of a model monitoring method according to an embodiment of the present disclosure;

[0028] Figure 10 is a third schematic flowchart of a model monitoring method according to an embodiment of the present disclosure;

[0029] Figures 11A to 11D are schematic structural diagrams of a distribution of submodels according to an embodiment of the present disclosure;

[0030] Figure 12 is a schematic structural diagram of a composition of a model 1200 monitoring device according to an embodiment of the present disclosure;

[0031] Figure 13 is a schematic structural diagram of a communication device composition according to an embodiment of the present disclosure; and

[0032] Figure 14 is a schematic structural diagram of a chip according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] The technical solutions of the disclosure modalities are described below with reference to the attached drawings in the disclosure modalities, and it is apparent that the modalities described are part of the disclosure modalities, and not all of them. Based on the disclosure modalities, any other modalities obtained by those of common skill in the technique without making creative effort are within the scope of disclosure protection.

[0034] To facilitate understanding of the technical solutions of the modalities of the present disclosure, the technologies Petition 870250089278, dated 10 / 01 / 2025, page 13 / 252 6 / 99 related to the embodiments of this disclosure will be described below. The following related technologies, as an optional solution, may be arbitrarily combined with the technical solutions of the embodiments of this disclosure, and all of them fall within the scope of protection of the embodiments of this disclosure.

[0035] Figure 1 is a schematic diagram of an application scenario according to an embodiment of the present disclosure.

[0036] As illustrated in Figure 1, communication system 100 includes a terminal device 110 and a network device 120. The network device 120 can communicate with the terminal device 110 through an air interface. Multi-service transmission is supported between the terminal device 110 and the network device 120.

[0037] It should be understood that the embodiments of the present disclosure are illustrated only with reference to communication system 100; however, the embodiments of the present disclosure are not limited to this. That is, the technical solution of the embodiments of the present disclosure can be applied to various communication systems, such as a Long-Term Evolution (LTE) system, an LTE Time Division Duplexing (TDD) system, a Universal Mobile Telecommunication System (UMTS), an Internet of Things (IoT) system, a Narrowband Internet of Things (NB-IoT) system, an enhanced Machine-Type Communications (eMTC) system, a 5G communication system (also referred to as a New Radio (NR) communication system), or future communication systems, etc. Petition 870250089278, dated 10 / 01 / 2025, page 14 / 252 7 / 99

[0038] In the communication system 100 illustrated in Figure 1, the network device 120 can be an access network device that communicates with the terminal device 110. The access network device can provide communication coverage for a particular geographic area and can communicate with the terminal device 110 (such as UE) located within the coverage area.

[0039] Network device 120 may be an Evolutionary B Node (eNB or eNodeB) in an LTE (Long Term Evolution) system, a Next Generation Radio Access Network (NG RAN) device, a base station (gNB) in an NR system, a radio controller in a Cloud Radio Access Network (CRAN), or network device 120 may be a relay station, an access point, a vehicle-mounted device, a wearable device, a hub, a switch, a bridge, a router, or a network device in a future evolved Public Terrestrial Mobile Network (PLMN).

[0040] Terminal device 110 can be any terminal device, including but not limited to a terminal device connected to network device 120 or another terminal device using a wired or wireless connection.

[0041] For example, terminal device 110 may be referred to as an access terminal, User Equipment (UE), subscriber unit, subscriber station, mobile console, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent, or user appliance. The access terminal may be a mobile phone, a cordless phone, Petition 870250089278, dated 10 / 01 / 2025, page 15 / 252 8 / 99 a Session Initiation Protocol (SIP) phone, an IoT device, a portable satellite terminal, a Wireless Local Area Link (WLL) station, a Personal Digital Assistant (PDA), a portable device with wireless communication function, a computing device or other processing device connected to a wireless modem, an embedded device, a wearable device, a terminal device on a 5G network, a terminal device in a future evolution, etc.

[0042] Terminal device 110 can be used for device-to-device (D2D) communication.

[0043] The wireless communication system 100 may also include a core network device 130 that communicates with the network device 120, and the core network device 130 may be a Fifth Generation Core (5GC) device, for example, an Access and Mobility Management Function (AMF), an Authentication Server Function (AUSF), a User Plane Function (UPF), a Session Management Function (SMF). Alternatively, the core network device 130 may also be an Evolved Packet Core (EPC) device of an LTE network, for example, a Session Management Function + Core Packet Gateway (SMF + PGW-C) device of the core network. It should be understood that the SMF + PGW-C may implement the functions implemented by both the SMF and the PGW-C.In the process of network evolution, the aforementioned core network device may be called by other names, or a new network entity may be formed by dividing the functions of the core network, which is not limited by the modalities of the present disclosure. Petition 870250089278, dated 10 / 01 / 2025, page 16 / 252 9 / 99

[0044] The connection between the functional units of the communication system 100 can be established through a next-generation (NG) interface to implement the communication.

[0045] For example, the terminal device establishes an air interface connection with the access network device via the NR interface to transmit user plane data and control plane signaling. The terminal device can establish a control layer signaling connection with the AMF via the NG 1 interface (N1 for short). The access network device, such as a next-generation radio access base station (gNB), can establish a user plane data connection with the UPF via an NG 3 interface (N3 for short). The access network device can establish a control plane signaling connection with the AMF via the NG 2 interface (N2 for short). The UPF can establish a control plane signaling connection with the SMF via the NG 4 interface (N4 for short). The UPF can interface user plane data with the data network via the NG 6 interface (N6 for short).The AMF can establish a control plane signaling connection with the SMF through the NG 11 interface (N11 for short). The SMF can establish a control plane signaling connection with the PCF through the NG 7 interface (N7 for short).

[0046] Figure 1 illustrates, by way of example, a network device, a central network device, and two terminal devices. Alternatively, the 100 wireless communication system may include a plurality of Petition 870250089278, dated 10 / 01 / 2025, page 17 / 252 10 / 99 network devices and other numbers of terminal devices may be included within the coverage of each network device, which is not limited to the embodiments of the present disclosure.

[0047] It should be noted that Figure 1 schematically illustrates the system to which the present disclosure applies as an example, and, of course, the methods illustrated in the embodiments of the present disclosure can also be applied to other systems. Furthermore, the terms system and network in the present disclosure are frequently used interchangeably in this document. The term and / or in the present document only describes an association relationship of the associated objects and represents that three relationships can exist. For example, A and / or B can represent the following three cases: only A exists, both A and B exist, and only B exists. Additionally, the character / in the present document generally means that the associated objects have an or relationship. It should also be understood that the indication mentioned in the embodiments of this disclosure can be a direct indication or an indirect indication, and can also represent that an association relationship exists.For example, A indicates B, which may mean that A indicates B directly, for example, B can be acquired through A. It may also mean that A indicates B indirectly, for example, A indicates C, and B can be acquired through C. It may also indicate that there is an association relationship between A and B. Furthermore, it should be understood that the corresponding term mentioned in the present disclosure may indicate that there is a direct or indirect correspondence between the two items, it may also indicate that there is a correlation relationship between the two items, or it may. Petition 870250089278, dated 10 / 01 / 2025, p. 18 / 252 11 / 99 indicates a relationship between indicating and being indicated, configuring and being configured, or something similar. It should also be understood that the predefined or “predefined rule” mentioned in the embodiments of this disclosure may be implemented by storing the corresponding codes, tables, or other means that may be used to indicate relevant information in advance on the device (including, for example, the terminal device and the network device), and the specific implementations thereof are not limited in this disclosure. For example, predefined may refer to being defined in the protocol. It should also be understood that the protocol in the embodiments of this disclosure may refer to a standard protocol in the field of communication, which may include, for example, an LTE protocol, an NR protocol, and related protocols applied in future communication systems, which is not limited to this disclosure.

[0048] To facilitate understanding of the technical solutions of the embodiments of this disclosure, the technologies related to the embodiments of this disclosure will be described below. The following related technologies may, as an optional solution, be arbitrarily combined with the technical solutions of the embodiments of this disclosure, and all of them belong to the scope of protection of the embodiments of this disclosure.

[0049] In practical application, the placement method for a terminal device may include a UE-based placement method, a UE / Location Management Function (LMF) assisted network element placement method (LMF-based), and a network node assistance-based placement method. Petition 870250089278, dated 10 / 01 / 2025, page 19 / 252 12 / 99 access (assisted NG-RAN node).

[0050] In the UE-based positioning method, the terminal device directly calculates the position of the target UE.

[0051] In the UE-assisted / LMF-based positioning method, the terminal device reports the measurement result to the LMF, and the LMF calculates the target UE position based on the collected measurement result.

[0052] In the NG-RAN node assistance-based positioning method, the base station reports the Transmit / Receive Point (TRP) measurement result to the LMF, and the LMF calculates the target UE position based on the collected measurement result.

[0053] It should be noted that a plurality of TRPs around the terminal device may participate in location positioning, a base station may be a TRP, and there may be a plurality of TRPs under a base station. The LMF may be a positioning server responsible for the entire positioning process.

[0054] In the traditional positioning method, the terminal device or LMF applies traditional algorithms, such as Chan's algorithm, Taylor expansion, and other algorithms, to estimate the position of the terminal device.

[0055] The downlink-based positioning method and the uplink-based positioning method are described below, respectively.

[0056] With reference to Figure 2A, the downlink-based positioning method may include the following steps.

[0057] In step 1, the LMF network element notifies Petition 870250089278, dated 10 / 01 / 2025, page 20 / 252 13 / 99 the configuration related to TRP.

[0058] The configuration may include PRS configuration information and / or information such as a type of measurement result to be reported by the terminal device.

[0059] In stage 2, the TRP transmits the PRS.

[0060] In step 3, the terminal device receives a PRS positioning signal and performs the measurement.

[0061] It should be noted that the measurement results required by the terminal device are different in different positioning methods.

[0062] In step 4, the terminal device feeds the measurement result back to the LMF.

[0063] The terminal device sends back the measurement result to the LMF via the base station.

[0064] In step 5, the LMF calculates the position information.

[0065] It should be noted that the preceding is a schematic flow of the UE-assisted positioning method. In the UE-based positioning method, the terminal device calculates the position information in step 4 as described above, based on the measurement result, without reporting the measurement result to the LMF and without calculation by the LMF network element. In the UE-based positioning method, the terminal device must know the position information corresponding to the TRP, therefore it is necessary for the LMF to notify the terminal device of the position information corresponding to the TRP in advance.

[0066] Furthermore, referring to Figure 2B, the upward link-based positioning method may include the following steps. Petition 870250089278, dated 10 / 01 / 2025, page 21 / 252 14 / 99

[0067] In step 1, the LMF notifies the configuration related to the TRP.

[0068] In step 2, the TRP transmits the signaling related to the terminal device.

[0069] In step 3, the terminal device sends a Sound Reference Signal (SRS).

[0070] In step 4, the TRP measures the SRS and transmits the measurement result to the LMF.

[0071] In step 5, the LMF calculates the position information.

[0072] The essence of AI / ML technology is to enable the computer to learn rules from data on its own and predict future data according to the rules obtained. AI / ML research, represented by neural networks, has made great strides in many fields and will also play an important role in production and people's lives for a long time to come.

[0073] Referring to the schematic diagram of a neuronal structure illustrated in Figure 3, the neural network is an operational model composed of a plurality of interconnected neural nodes. As illustrated in Figure 3, the neuronal structure can be connected to other neuronal structures a1 to an. The transmission of signals between neuronal structures is affected by a weight (for example, the weight value of the signal entered by neuronal structure a1 is w1), and each neuronal structure can calculate a weighted sum of a plurality of input and output signals through a specific activation function.

[0074] Figure 4 is a schematic structural diagram of a neural network proposed in the related technique. Petition 870250089278, dated 10 / 01 / 2025, page 22 / 252 15 / 99 As illustrated in Figure 4, the neural network structure can include an input layer, a hidden layer, and an output layer. As illustrated in Figure 4, the input layer is used to receive data, and the hidden layer is used to process the data, generating a result in the output layer. Each node represents a processing unit, which can be considered to simulate a neuron. Several neurons form a neural network layer, and the transmission and processing of information across multiple layers builds an overall neural network.

[0075] With the continuous development of neural network research, deep learning algorithms for neural networks have been proposed even more in recent years, with more hidden layers being introduced. Feature learning is performed through layer-by-layer training across the neural network with multiple hidden layers, which greatly improves the learning and processing capabilities of the neural network and is widely used in pattern recognition, signal processing, combination optimization, anomaly detection, etc.

[0076] Similarly, with the development of deep learning, Convolutional Neural Networks (CNNs) have also been studied even more.

[0077] Figure 5 is a schematic structural diagram of a convolutional neural network provided by the related state of the art. As illustrated in Figure 5, the structure of the convolutional neural network may include an input layer, a plurality of convolutional layers, a plurality of clustering layers, a fully connected layer, and an output layer. With the introduction Petition 870250089278, dated 10 / 01 / 2025, page 23 / 252 In the 16 / 99 convolution layer and clustering layer, the dramatic increase in network parameters is effectively controlled, the number of parameters is limited, the characteristics of the local structure are extracted, and the robustness of the algorithm is improved.

[0078] Recurrent neural networks have achieved remarkable feats in the field of natural language processing, such as machine translation and speech recognition. A recurrent neural network is a neural network that models sequence data, memorizes information from the past moment, and uses it to calculate the current output. That is, the nodes between the hidden layers are no longer disconnected but connected, and the input of the hidden layer includes not only the input layer but also the output of the hidden layer at the previous moment.

[0079] Figure 6 is a schematic structural diagram of a Long Short Term Memory (LSTM) provided in the related prior art. The LSTM is a common recurrent neural network. Unlike the recurrent neural network, which considers only the most recent state, the LSTM determines which states should be retained and which should be forgotten, so that the defects of traditional recurrent neural networks in long-term memory are resolved.

[0080] In AI / ML technology, an AI / ML model can be built based on the neural networks introduced in the modalities above, and the AI / ML model can be trained so that the AI / ML model can learn the rules from a large amount of data, so that the trained AI / ML model can predict future data using the learned rules.

[0081] It should be understood that the AI / ML model can Petition 870250089278, dated 10 / 01 / 2025, page 24 / 252 17 / 99 to be trained based on a training dataset. The training dataset is a dataset with a relatively precise description of characteristics by adding / labeling (including manual and non-manual addition / labeling) label information. That is, the label information can be information that describes the characteristic of each data point in the training dataset. For example, when the AI / ML model is used for terminal device positioning, the AI / ML model can be trained using a training dataset appended with the terminal device location label.

[0082] In practical applications, the training dataset usually has the characteristics of large amount of data, diversity of data, and high quality of data.

[0083] Furthermore, the input dataset can be understood as the dataset predicted using the AI / ML model. After the input data in the input dataset is processed by the AI / ML model, the expected prediction result for a user can be obtained.

[0084] Given the great success of AI or ML technology, such as the aforementioned neural network and deep learning, in computer vision, natural language processing, etc., AI technology is beginning to be used in the field of communication to seek new technical ideas to solve technical problems limited by traditional methods.

[0085] In practical applications, the technology of AI / ML can be used to implement the estimation of Petition 870250089278, dated 10 / 01 / 2025, page 25 / 252 18 / 99 positioning / channel or other terminal device functions. That is, the AI / ML model can be combined with any positioning / channel estimation method to, as a substitute for the traditional positioning / channel estimation method, estimate the position / channel state of the terminal device.

[0086] It should be noted that the AI / ML model can directly produce the position / channel state of the terminal device, or it can help determine the position / channel state of the terminal device. For example, the description is given taking the AI / ML model as a positioning model. The AI / ML model is combined with the positioning method, which can be distinguished into direct AI / ML positioning and AI / ML-assisted positioning. In direct AI / ML positioning, as illustrated in Figure 7A, the AI / ML model can directly produce the position of the terminal device according to the model's input data. In indirect AI / ML positioning, as illustrated in Figure 7B, the AI / ML model can first produce the intermediate parameter according to the model's input data and then determine the position of the terminal device in combination with the intermediate parameter and the positioning algorithm.

[0087] When the AI / ML model is not suitable for the current scenario, i.e., the processing accuracy of the AI / ML model does not meet the real requirements, the AI / ML model needs to be updated. The key to updating the AI / ML model lies in how to monitor the AI / ML model's performance.

[0088] Monitoring the AI / ML model is Petition 870250089278, dated 10 / 01 / 2025, page 26 / 252 19 / 99 Monitoring the accuracy of AI / ML model output data. Generally, the AI / ML model monitoring method is a label information-based monitoring method. In this method, label information from the input data is collected, and the AI / ML model output result is compared with the label information to monitor the accuracy of the AI / ML model. Therefore, to monitor the performance of the AI / ML model, it is necessary to collect the input data and the corresponding label information for model monitoring. However, in practical applications, it is difficult to obtain the label information corresponding to the AI / ML model input data. Especially when the AI / ML model is a positioning model, the position label of the terminal device is required. However, the terminal device location is private information, and currently, there is no effective method to obtain the terminal device location.Therefore, monitoring the performance of an AI / ML model without the corresponding label information for the input data is an urgent problem that needs to be solved.

[0089] Based on the foregoing, the embodiments of the present disclosure provide a model monitoring method. Figure 8 is a schematic flow diagram of the model monitoring method provided by the embodiment of the present disclosure. As illustrated in Figure 8, the method includes the following operation 810.

[0090] In operation 810, the first device determines a similarity comparison result between an input dataset and a training dataset of a first model. The similarity comparison result is used to determine the performance of Petition 870250089278, dated 10 / 01 / 2025, page 27 / 252 20 / 99 first model.

[0091] It should be understood that the first device can monitor the performance of the first model. In other words, the monitoring of the first model's performance is performed by the first device. The first model can be deployed on the first device, and the first device can monitor the performance of the deployed first model, determine if the performance of the first model meets the actual requirement, and update the first model in a timely manner when the performance of the first model does not meet the actual requirement.

[0092] It is important to note that the first device may be a first terminal device or a network device. The network device may include an access network device, such as a base station, a micro base station, etc. The network device may also include a central network device, such as an LMF, or other network element with a positioning management function, which is not limited in the embodiments of the present disclosure.

[0093] The first terminal device can be a terminal device to be processed (e.g., a terminal device to be positioned, a terminal device whose channel estimation information needs to be determined), or a terminal device requiring processing (e.g., a terminal device requiring positioning, or a terminal device requiring channel estimation), or a terminal device deployed with a first model.

[0094] It should be understood that the first model can be implemented on the first terminal device and / or on Petition 870250089278, dated 10 / 01 / 2025, page 28 / 252 21 / 99 network device. When the first model is deployed on the first terminal device, the first terminal device can monitor the performance of the first model. When the first model is deployed on an access network device (such as a gNB), the access network device can monitor the performance of the first model. When the first model is deployed on a main network device (such as an LMF), the main network device can monitor the performance of the first model. When the first model is deployed on both the first terminal device and the network device (including the access network device and / or the main network device), both the first terminal device and the network device can monitor the performance of the first model.

[0095] In some embodiments, the first model can be used to directly determine the channel state or assist in determining the channel state, and in this scenario, the first model can be called a channel estimation model. Furthermore, the first model can also be used for direct positioning / assisted positioning, and in this scenario, the first model can be called a positioning model. The type of first model is not limited in the embodiments of the present disclosure.

[0096] The first model mentioned in the embodiments of this disclosure may be a pre-trained AI / ML model, such as a neural network model, a CNN model, a Deep Neural Network (DNN) model, an LSTM model, etc., which is not limited to the embodiments of this disclosure.

[0097] In the embodiments of the present disclosure, the first model can be trained on a dataset of Petition 870250089278, dated 10 / 01 / 2025, page 29 / 252 22 / 99 training of the first model. The training dataset can include a large amount of training data. Each training data point can include corresponding label information. The label information can accurately describe the features of the training data.

[0098] In one example, in a scenario where the first model is used for direct positioning / assisted positioning of the first terminal device, the training dataset for the first model may include the measurement parameter related to the position between a second terminal device and one or more network devices, and the label information corresponding to each measurement parameter.The measurement parameter related to the position between the second device and one or more network devices may include one or more of the following: a channel impulse response (CIR), a power delay profile (PDP), a time of arrival (ToA), a path phase, an uplink time difference (UL TDoA), a downlink time difference (DL TdoA), an uplink departure angle (UL AoD), a downlink departure angle (DL AoD), a relative time of arrival (UL RTOA), a reference signal time difference (DL RSTD), a reference signal reception power (RSRP), a reference signal reception quality (RSRQ), an azimuth, and a zenith between the second terminal device and the network device.

[0099] If the first model is used for direct positioning, the label information corresponding to each measurement parameter may be the position of the second terminal device associated with the parameter. Petition 870250089278, dated 10 / 01 / 2025, p. 30 / 252 23 / 99 measurement.

[00100] If the first model is used for assisted positioning, the label information corresponding to each measurement parameter may include at least one of the following: a ToA, a DL TdoA, a DL AoD, a DL RSTD, a UL TdoA, a UL AoD, a UL RTOA, an RSRP, an RSRQ, a Line of Sight / Non-Line of Sight (LOS / NLOS) identification, distance information between the second terminal device and the network device associated with the measurement parameter.

[00101] In another example, in a scenario where the first model is used to directly determine the channel state, the training dataset for the first model might include the measurement parameter related to a channel state between the second terminal device and the network device, and the label information corresponding to each measurement parameter. The measurement parameter related to the channel state between the second device and one or more network devices might include one or more of the following: an RSRP, an RSRQ, a signal-to-noise ratio (SINR), etc. Consequently, the label information corresponding to each parameter might be channel state information between the second terminal device and the network device.

[00102] It should be noted that the “training dataset” mentioned in the embodiments of this disclosure refers to the training dataset of the first model. Furthermore, the input dataset mentioned in the embodiments of this disclosure refers to the input dataset of the first model.

[00103] It should be understood that the first model is Petition 870250089278, dated 10 / 01 / 2025, p. 31 / 252 24 / 99 a model obtained by training based on the training dataset. The training process of the first model can be understood as allowing the output result obtained after processing the training data by the first model to be consistent / matched with the label information corresponding to the training data. In this way, when any input data is entered into the first trained model, the first trained model can obtain an output result of the same type as the label information in the training dataset.

[00104] In a scenario where the first model is a channel estimation model, the label information of each training data in the first model's training dataset can be channel state information, so that the first trained model can process the input data and generate the channel state information.

[00105] It should be noted that the first model can be obtained by training based on a training dataset by a first device that performs model monitoring, or it can be obtained by training based on a training dataset by another device, which is not limited to the embodiments of the present disclosure.

[00106] Note also that, when the first monitoring device for the first model and the training device for the first model are different devices, the training dataset for the first model can be acquired by the first device from the training device for the first model, or the training data for the first model can be transmitted by the device that Petition 870250089278, dated 10 / 01 / 2025, page 32 / 252 25 / 99 trains the first model for the first device. For example, the first model is deployed on the first terminal device, the first terminal device monitors the first model, and the first model can be obtained by training via LMF. In this case, LMF can transmit a training dataset to train the first model for the first terminal device, so that the first terminal device can monitor the performance of the first model based on the training dataset.

[00107] In one embodiment of the present disclosure, when the first device monitors the performance of the first model, the first device can acquire the input dataset (model input dataset) of the first model. The input dataset can include a large amount of input data. The first model can process the input data in the input dataset and generate an output result of the same type as the label information included in the training dataset.

[00108] The input dataset can be of the same type as the training dataset, and the input data in the input dataset may not include the label information.

[00109] In one example, in a scenario where the first model is used for direct positioning / assisted positioning for the first terminal device, the input dataset for the first model may include the measurement parameter related to the position between the first terminal device (i.e., a terminal device to be positioned) and one or more network devices. Petition 870250089278, dated 10 / 01 / 2025, page 33 / 252 26 / 99 Thus, in a scenario where the first model is used for direct positioning of the first terminal device, the first model can process the input data and output the position of the first terminal device. In a scenario where the first model is used for assisted positioning of the first terminal device, the first model can process input data and output an intermediate parameter of the first terminal device, and the intermediate parameter includes, but is not limited to: a ToA, a DL TdoA, a DL AoD, a DL RSTD, a UL TdoA, a UL AoD, a UL RTOA, an RSRP, an RSRQ, a LOS / NLOS identification, and distance information between the first terminal device and the network device.

[00110] In another example, in a scenario where the first model is used to directly determine the channel state, the input dataset of the first model may include the measurement parameter relating to the channel state between the first terminal device (i.e., a terminal device from which the channel state is to be determined) and the network device. Thus, after processing the input data in the input dataset, the first model can output the channel state information between the first terminal device and the network device.

[00111] It should be understood that the first model is obtained by training based on the training dataset, and the label information of each training data in the training dataset can be considered as an output result of the first model with the highest accuracy. Therefore, when the first device monitors the performance of the first model, the input dataset can be compared with the dataset of Petition 870250089278, dated 10 / 01 / 2025, page 34 / 252 27 / 99 training. If the two datasets are similar, it can be considered that, after the input dataset is processed by the first model, a highly accurate output result can also be obtained, and the performance of the first model is good. If the two datasets are not similar, it can be considered that, after the input dataset is processed by the first model, a highly accurate output result cannot be obtained, and the performance of the first model is poor.

[00112] Note that the input dataset for the first model can only include a plurality of input data and may not include an output result corresponding to each of the pluralities of input data and the label information corresponding to the input data.

[00113] Note also that, unless otherwise defined, the similarity comparison result described in the embodiments of this disclosure may be replaced by other similar meanings generally understood by persons skilled in the art. For example, the similarity comparison result described in the embodiments of this disclosure may also be replaced by comparison result, matching result, degree of matching, validity result (specifically, it may refer to the validity of the input dataset), etc. Consequently, the expression the input dataset is / is not similar to the training dataset described in the embodiments of this disclosure may also be replaced by the description how the input dataset is / does not match the set of Petition 870250089278, dated 10 / 01 / 2025, p. 35 / 252 28 / 99 training data or the input dataset is valid / invalid. It is understood that the terms used in this document are for descriptive purposes only and are not intended to limit the present disclosure.

[00114] As set out above, in the model monitoring method provided by the implementation of this disclosure, when monitoring the first model, the first device may not collect the label information corresponding to the input data, but only compare the similarity between the input dataset of the first model and the training dataset of the first model, and determine the performance of the first model using the result of the similarity comparison. In this way, model monitoring is implemented in a scenario where the label information corresponding to the input data cannot be collected, thus improving the feasibility and flexibility of model monitoring.

[00115] In one embodiment of the present disclosure, the first model can be used for direct positioning / assisted positioning for the terminal device. With reference to Figure 7A, when the first model is used for direct positioning for the terminal device, the location of the terminal device can be output. With reference to Figure 7B, when the first model is used for assisted positioning for the terminal device, the intermediate parameter of the terminal device can be output and the location of the terminal device can be obtained by combining the intermediate parameter with the positioning algorithm.

[00116] In a scenario where the first model is Petition 870250089278, dated 10 / 01 / 2025, page 36 / 252 29 / 99 used for direct positioning / assisted positioning of the first terminal device, the training dataset of the first model may include the measurement parameter between the second terminal device and one or more network devices, and the label information corresponding to each measurement parameter.

[00117] The measurement parameter between the second device and one or more network devices may include one or more of the following: a channel impulse response (CIR) between the second terminal device and the network device, a power delay profile (PDP), a time of arrival (ToA), a path phase, an uplink time difference (UL TDoA), a downlink time difference (DL TdoA), an uplink departure angle (UL AoD), a downlink departure angle (DL AoD), a UL relative time of arrival (UL RTOA), a DL reference signal time difference (DL RSTD), a reference signal reception power (RSRP), a reference signal reception quality (RSRQ), an azimuth, and a zenith.

[00118] If the first model is used for direct positioning, the label information corresponding to each measurement parameter may be the position of the second terminal device associated with the measurement parameter.

[00119] If the first model is used for assisted positioning, the label information corresponding to each measurement parameter may include at least one of the following: a ToA, a DL TdoA, a DL AoD, a DL RSTD, a UL TdoA, a UL AoD, a UL RTOA, a RSRP, an RSRQ, a Petition 870250089278, dated 10 / 01 / 2025, p. 37 / 252 30 / 99 Line of Sight / Non-Line of Sight (LOS / NLOS) identification, distance information between the second terminal device and the network device associated with the measurement parameter.

[00120] It is worth noting that the second terminal device may be a terminal device added with label information. There may be a plurality of secondary terminal devices. The measurement parameter between a second device and one or more network devices may be understood as training data from the training dataset.

[00121] In a scenario where the first model is used for direct positioning / assisted positioning for a terminal device, the input dataset of the first model may include a measurement parameter associated with the position of the terminal device.

[00122] It should be noted that the measurement parameter can be a measurement parameter between the terminal device and one or more network devices. Specifically, the measurement parameter can be a parameter that can be measured when the terminal device performs signal transmission with one or more network devices, to position the terminal device.

[00123] The measurement parameter can be measured by the terminal device or by the network device, which is not limited to the embodiments of the present disclosure.

[00124] In some embodiments, the measurement parameter includes one or more of: a CIR, a PDP, a ToA, a path phase, a UL TDoA, a DL TdoA, a UL AoD, a DL AoD, a UL RTOA, a DL RSTD, an RSRP, an RSRQ, an azimuth and a zenith. Petition 870250089278, dated 10 / 01 / 2025, p. 38 / 252 31 / 99

[00125] For example, when the first device to monitor the first model is a terminal device, the terminal device can measure a downlink reference signal (e.g., a positioning reference signal (PRS), a synchronization signal block (SSB), a channel state information reference signal (CSIRS), etc.) transmitted by one or more network devices, to obtain the downlink measurement parameter of the terminal device relative to one or more network devices, and the downlink measurement parameter includes, but is not limited to: a CIR, an RSRP, a DL RSTD, a TOA, a DL TDOA, and a DL RSTD of the terminal device relative to one or more network devices.

[00126] It should be noted that when the first device is a terminal device, the terminal device may still transmit an uplink reference signal (e.g., a sound reference signal (SRS)) to one or more network devices, and the network device performs measurements based on the uplink reference signal and transmits an obtained uplink measurement parameter (including, but not limited to, CIR, PDP, RSRP, UL RSTD, TOA, UL TDOA, UL AoD, UL RTOA, etc.) to the terminal device. In this way, the terminal device may obtain both the uplink measurement parameter and the downlink measurement parameter, and the uplink measurement parameter and the downlink measurement parameter may constitute a complete set of input data for the first model.

[00127] In another example, when the first device to monitor the first model is a device Petition 870250089278, dated 10 / 01 / 2025, p. 39 / 252 In a 32 / 99 network, the network device can measure an uplink reference signal (e.g., SRS) transmitted by the terminal device to obtain the network device's measurement parameter relative to the terminal device, and the measurement parameter includes, but is not limited to: the CIR, PDP, RSRP, UL RSTD, TOA, UL TDOA, UL AoD, and UL RTOA of the network device relative to the terminal device.

[00128] It should be noted that when the first device is a network device, the network device can transmit a downlink reference signal (e.g., a PRS, SSB, CSI-RS, etc.) to the terminal device, and the terminal device performs the measurement based on the downlink reference signal and reports the obtained downlink measurement parameter (including, but not limited to, CIR, RSRP, DL RSTD, TOA, DL TDOA, and DL RSTD) to the network device. In this way, the network device can obtain both the uplink measurement parameter and the downlink measurement parameter, and the uplink measurement parameter and the downlink measurement parameter can constitute a complete set of input data for the first model.

[00129] It should also be noted that, in the two examples above, the downlink reference signal to be measured by the terminal device may be specified in the protocol or configured by the network device, which is not limited to the embodiments of the present disclosure. Furthermore, the reference signal transmitted by the terminal device to one or more network devices may correspond to a protocol or may be configured by the network device, which is not limited to the embodiments of the present disclosure. Petition 870250089278, dated 10 / 01 / 2025, p. 40 / 252 33 / 99

[00130] As established above, when the first device monitors the first model, the first device may not collect the location label of the first device, but only collect the measurement parameter associated with the location of the first device and use the measurement parameter as the input dataset of the first model. Then, the similarity between the input dataset and the training dataset of the first model is compared, and the performance of the first model is determined using the result of the similarity comparison. In this way, model monitoring is implemented in a scenario where input data label information cannot be collected, making model monitoring feasible and flexible.

[00131] In one embodiment of the present disclosure, the result of the similarity comparison between the input dataset of the first model and the training dataset of the first model can be used to determine the performance of the first model. The first device can determine the performance of the first model according to the result of the aforementioned similarity comparison in the following two ways.

[00132] In the first form, in one embodiment of the present disclosure, the first device can determine that the performance of the first model meets a condition if the result of the similarity comparison indicates that the input dataset is similar to the training dataset. In a case where the result of the similarity comparison indicates that the input dataset is not similar to the training dataset, the first Petition 870250089278, dated 10 / 01 / 2025, page 41 / 252 34 / 99 device updates the first model.

[00133] With reference to Figure 9, when monitoring the first model, the first device can acquire the training dataset and the input dataset of the first model in advance. Then, the similarity between the training dataset and the input dataset of the first model is compared to obtain the similarity comparison result.

[00134] In some modalities, the result of the similarity comparison may indicate similarity or dissimilarity.

[00135] In one possible implementation, the result of the similarity comparison could be a fixed classification result, for example, the similarity comparison result is 0 or 1. Here, 0 indicates that the training dataset is not similar to the input dataset of the first model, and 1 indicates that the training dataset is similar to the input dataset of the first model.

[00136] In another possible implementation, the result of the similarity comparison can be a smooth classification result. In one example, the result of the similarity comparison can be a similarity between the training dataset and the input dataset with a decimal value between 0 and 1. In another example, the result of the similarity comparison is a quantized value between 0 and 1 that can be represented by n bits. For example, the quantized value is 0, 0.25, 0.75, and 1, and can be represented by 2 bits, where 00 represents that the result of the similarity comparison has a value of 0, 01 represents that the Petition 870250089278, dated 10 / 01 / 2025, page 42 / 252 35 / 99 represents the similarity comparison result with a value of 0.25, 10 represents the similarity comparison result with a value of 0.75, and 11 represents the similarity comparison result with a value of 1. It should be understood that when the similarity comparison result is greater than or equal to a specified threshold, the result may indicate that the training dataset is similar to the input dataset of the first model. When the similarity comparison result is less than the specified threshold, the result may indicate that the training dataset of the first model is not similar to the input dataset.

[00137] It should be understood that when the result of the similarity comparison indicates that the input dataset is similar to the training dataset, the output result obtained by processing the input dataset by the first model is close to the output result of the actual parameter. Therefore, in a case where the result of the similarity comparison indicates that the input dataset is similar to the training dataset, the first device can determine that the performance of the first model meets the requirement.

[00138] On the other hand, in a case where the result of the similarity comparison indicates that the input dataset is not similar to the training dataset, an output result obtained after the input data is processed by the first model is not close to the actual parameter. Therefore, when the result of the similarity comparison indicates that the input dataset is not Petition 870250089278, dated 10 / 01 / 2025, page 43 / 252 36 / 99 is similar to the training dataset; the first device may determine that the performance of the first model does not meet the requirement. In this case, the first device may update the first model so that the updated model can generate a more accurate result.

[00139] It should be noted that updating the first model may include fine-tuning or model replacement. Model fine-tuning may include fine-tuning of model parameters and / or fine-tuning of the structure.

[00140] It should be understood that model switching refers to changing from an initial model to a new model that is retrained. The amount of data updated in model switching is large, while the amount of data updated in model fine-tuning is small. Therefore, the complexity of model switching is greater than that of model fine-tuning.

[00141] In a possible implementation, the method for updating the first model can be specified by the protocol. The protocol specifies that the update of the first model can be a fine-tuning of the model or a model swap.

[00142] In another possible implementation, the way to update the first model can also be determined by the first device according to a predefined rule. For example, when the result of the similarity comparison is a flexible classification result, the first device can determine to perform fine-tuning of the model when the result of the similarity comparison is less than a threshold of 1 and greater than or equal to a threshold of 2. When the result of the similarity comparison is less than threshold 2, the first device can determine to perform the swap of Petition 870250089278, dated 10 / 01 / 2025, page 44 / 252 37 / 99 model. The first device can determine if the input dataset matches the output dataset according to the value of the smooth classification result generated by the first model and determine whether to perform fine-tuning of the model or model switching according to the corresponding case, thus improving the flexibility of the model update.

[00143] Secondly, it should be noted that, in addition to the measurement parameter associated with the location of the terminal device, the input data set may also include a measurement parameter associated with the location of a positioning reference unit (PRU).

[00144] It should be understood that a PRU is a terminal device, whose location is known, introduced into the network. The PRU can perform positioning measurements (e.g., RSTD, RSRP, UE Rx-Tx time difference, etc.) and report the measurement result to the positioning server. In addition, the PRU can also transmit an uplink reference signal (e.g., SRS), and the TRP can measure and report the uplink measurement parameter (e.g., RTOA, UL-AoA, gNB Rx-Tx time difference, etc.) according to the uplink reference signal.

[00145] In the first method, the first device can monitor the performance of the first model without obtaining label information corresponding to each input data in the input dataset. However, there may be an error in the result of the similarity comparison between the input dataset and the training dataset, resulting in a monitoring effect of Petition 870250089278, dated 10 / 01 / 2025, page 45 / 252 38 / 99 poor performance for the first model. In the second method, the easily obtainable location of the PRU can aid in monitoring the performance of the first model.

[00146] In one embodiment of the present disclosure, in the case where a verification result indicates success and the similarity comparison result indicates that the input dataset is similar to the training dataset, the first device can determine that the performance of the first model satisfies a condition. Furthermore, in a case where the verification result indicates failure and / or the similarity comparison result indicates that the input dataset is not similar to the training dataset, the first device updates the first model. The verification result indicates the result of verifying whether an estimated position of the PRU corresponds to an actual position; and the estimated position is obtained by processing, by the first model, the measurement parameter associated with the PRU position.

[00147] With reference to Figure 10, when monitoring the first model, the first device can acquire the training dataset and the input dataset of the first model in advance. Furthermore, the similarity between the training dataset and the input dataset of the first model is compared to obtain the similarity comparison result.

[00148] In addition, the first device can input the measurement parameter associated with the PRU position (e.g., CIR, RSRP, DL RSTD, TOA, DL TDOA, DL RSTD, PDP, UL RSTD, UL TDOA, UL AoD, UL RTOA, azimuth angle, zenith angle between the PRU and the network device) Petition 870250089278, dated 10 / 01 / 2025, p. 46 / 252 39 / 99 in the first model, and the first model processes the measurement parameter to obtain the estimated position of the PRU. A matching result between the estimated position of the PRU and the known actual position of the PRU is verified. Then, the first device can determine if the performance of the first model satisfies the condition in combination with the verification result and the similarity comparison result.

[00149] It should be noted that in a scene where the first model is used for direct positioning, the actual position may be coordinates of the actual PRU position, and the estimated position may be coordinates of the estimated PRU position. Whereas in a scenario where the first model is used for assisted positioning, the actual position may be an actual intermediate parameter related to the PRU position, and the estimated position may be an estimated intermediate parameter related to the PRU position.

[00150] In a possible implementation, if the estimated position of the PRU is the same as the actual position of the PRU, the verification result is considered a success and the performance (or positioning accuracy) of the first model meets the requirement. Otherwise, if the estimated position of the PRU is different from the actual position of the PRU, the verification result will be considered a failure and the performance (or positioning accuracy) of the first model will not meet the requirement.

[00151] In another possible implementation, the first device may introduce a positioning error threshold in consideration of a positioning error between the actual position and the estimated position of the PRU in practical application. If the difference between the estimated position of the PRU and the Petition 870250089278, dated 10 / 01 / 2025, page 47 / 252 40 / 99 If the actual position of the PRU is less than the positioning error threshold, the verification result is considered a success. Otherwise, if the difference between the estimated position of the PRU and the actual position of the PRU is greater than or equal to the positioning error threshold, the verification result will be considered a failure.

[00152] It should be noted that the positioning error threshold can be specified by a protocol, can be configured by a network (e.g., upper-layer network signaling), or can be predefined, which is not limited to the embodiments of the present disclosure.

[00153] In some embodiments, there may be a plurality of PRUs. When the estimated positions of the plurality of PRUs are successfully combined with the actual positions of the plurality of PRUs, the result of the verification indicates a success. Alternatively, in the case where the number of positioning reference units whose estimated positions and actual positions are successfully combined among the plurality of PRUs is greater than the first number, the result of the verification indicates a success.

[00154] That is, when there is a plurality of PRUs, the verification of the estimated positions and the actual positions of the PRUs can be successful in one of the following cases: the verification of the entire plurality of PRUs is successful, or the verification of a specified number (or a specified proportion) of PRUs in the plurality of PRUs is successful.

[00155] In one embodiment of the present disclosure, only in a case where a verification result indicates success and the similarity comparison result indicates that the training dataset is similar to the set Petition 870250089278, dated 10 / 01 / 2025, p. 48 / 252 41 / 99 of the input data, the first device determines that the performance of the first model satisfies a condition. When at least one of the cases where the verification result indicates a failure and the similarity comparison result indicates that the training dataset is not similar to the input dataset is satisfied, the first device can determine that the performance of the first model does not satisfy the condition, and the first device can update the first model.

[00156] It should be noted that, similar to the first method described above, updating the first model by the first device may include model fine-tuning or model swapping. Model fine-tuning may include fine-tuning of model parameters and / or fine-tuning of the structure.

[00157] It should be understood that model switching refers to changing from an initial model to a new model that is retrained. The amount of updated data in model switching is large, while the amount of updated data in model fine-tuning is small. Relatively speaking, the complexity of model switching is greater than that of model fine-tuning.

[00158] The method for updating the first model can be specified in the protocol or can be determined according to a predefined rule. For example, similar to the first method, when the result of the similarity comparison is a soft classification result, the first device can determine to perform fine-tuning of the model when the result of the similarity comparison is less than threshold 1 and greater than or equal to threshold 2. When the result of the similarity comparison is less than threshold 2, the first Petition 870250089278, dated 10 / 01 / 2025, page 49 / 252 The 42 / 99 device can determine whether to perform a model swap. The first device can determine a matching situation between the input dataset and the output dataset according to the smooth classification result value provided by the first model, and determine whether to perform model fine-tuning or model swapping according to the matching case, thus improving the flexibility of the model update.

[00159] As established above, when the estimated position of the PRU obtained after processing the first model is successfully matched with the actual position of the PRU, and the result of the similarity comparison between the input dataset and the training dataset of the first model indicates that the input dataset is similar to the training dataset, the performance of the first model is considered good. When the estimated position of the PRU obtained after processing the first model does not match the actual position of the PRU, and the result of the similarity comparison between the input dataset and the training dataset of the first model indicates that the input dataset is not similar to the training dataset, the performance of the first model is considered poor.Furthermore, if the estimated position of the PRU obtained after processing the first model does not correspond to the actual position of the PRU, the performance of the first model is considered poor, even if the result of the similarity comparison between the input dataset and the training dataset of the first model indicates that the input dataset is similar to the training dataset. The first device can monitor the... Petition 870250089278, dated 10 / 01 / 2025, page 50 / 252 43 / 99 first model with PRU assistance, thus improving the accuracy of the first model's performance detection.

[00160] In one embodiment of the present disclosure, the first model can be used for assisted positioning / assisted channel estimation for the terminal device. When the first assisted positioning / assisted channel estimation model is located on the network device side (i.e., the first device is a network device), the first model can be deployed respectively at different transmit and receive points (TRPs), and the first model can have different structures.

[00161] In some embodiments, the first model includes one or more submodels and one or more submodels are deployed in a plurality of TRPs. The model monitoring method according to the embodiments of the present disclosure may further include the following operation.

[00162] The first device can determine a similarity comparison result between the input dataset and the training dataset for each submodel.

[00163] If the results of the similarity comparison between the input datasets and the training datasets of the submodels deployed in a part of the plurality of TRPs indicate that the input datasets of the submodels are not similar to their respective training datasets, the submodels deployed in the part of the plurality of TRPs are updated.

[00164] It should be understood that the first device can monitor the performance of each of the various Petition 870250089278, dated 10 / 01 / 2025, p. 51 / 252 44 / 99 submodels and update only the submodels for which the similarity comparison result indicates that the submodels' input datasets are not similar to their respective training datasets.

[00165] In one example, the first model might include N submodels, and the N submodels are deployed in N TRPs, respectively. N is an integer greater than 1.

[00166] In one possible implementation, the first model may include N different submodels. With reference to Figure 11A, the N submodels are submodel 0, submodel 1 to submodel N-1. The N submodels are deployed in different TRPs, respectively. For example, submodel 0 is deployed in TRP0, submodel 1 is deployed in TRP1, and so on.

[00167] In this implementation, the N different submodels have different input datasets and different training datasets. The input dataset for each submodel can be a TRP measurement parameter on which the respective submodel is deployed. Alternatively, the input dataset for each submodel is a measurement parameter between the deployed TRP and the terminal device, and the measurement parameter includes, among others: the CIR, the RSRP, the DL RSTD, the TOA, the DL TDOA, the DL RSTD, the PDP, the UL RSTD, the UL TDOA, the UL AoD, the UL RTOA, the azimuth and the zenith between the terminal device and the TRP. With reference to Figure 11A, the input dataset for submodel 0 could be input dataset 0, which includes the measurement parameter between the terminal device and TRP 0. The input dataset for submodel 1 could be input dataset 1, which includes the measurement parameter between the terminal device and TRP 1.By analogy, Petition 870250089278, dated 10 / 01 / 2025, page 52 / 252 45 / 99 The input dataset for submodel N-1 may be the N-1 input dataset, which includes the measurement parameter between the terminal device and the N-1 TRP.

[00168] It should be understood that, in this implementation, for each submodel, the first device determines an input dataset and a training dataset for the submodel while monitoring the submodel's performance, determines the result of the similarity comparison between the input dataset and the training dataset of the submodel, and determines whether the submodel needs to be updated according to the result of the similarity comparison.

[00169] In another possible implementation, the first model may include N identical submodels. With reference to Figure 11B, submodels N are deployed in N TRPs, respectively.

[00170] In this implementation, the N identical submodels have identical training datasets, but different input datasets. The input dataset for each submodel can be a TRP measurement parameter on which the respective submodel is deployed. Alternatively, the input dataset for each submodel is a measurement parameter between the TRP where the respective submodel is deployed and the terminal device, and the measurement parameter includes, but is not limited to: the CIR, the RSRP, the DL RSTD, the TOA, the DL TDOA, the DL RSTD, the PDP, the UL RSTD, the UL TDOA, the UL AoD, the UL RTOA, the azimuth and zenith between the terminal device and the TRP. With reference to Figure 11B, the input dataset for the submodel deployed on TRP 0 can be input dataset 0, which includes the Petition 870250089278, dated 10 / 01 / 2025, p. 53 / 252 46 / 99 measurement parameter between the terminal device and TRP 0. The input dataset of the submodel deployed on TRP 1 can be input dataset 1, which includes the measurement parameter between the terminal device and TRP 1. By analogy, the input dataset of the submodel deployed on TRP N-1 can be input dataset N-1, which includes the measurement parameter between the terminal device and TRP N-1.

[00171] It should be understood that, in this implementation, the first device can determine, for each submodel, a set of input data for the submodel by monitoring the submodel's performance and understands the similarity between the input data set and the training data set of the submodel to obtain the result of the similarity comparison of the submodel. Then, the first device can determine which submodels in the N submodels need to be updated according to the result of the similarity comparison for each submodel.

[00172] In another example, the first model may include N submodels, and N is an integer greater than 1. The N submodels are deployed on M TRPs, and M is an integer greater than N. It should be understood that at least some of the M TRPs share the same submodel. For example, referring to Figure 11C, the first model may include N submodels (submodel 0 to submodel N-1), the N submodels are deployed on 2N TRPs, and every 2 TRPs are deployed with the same submodel. Submodel 0 may be deployed on TRP 0 and TRP 1, submodel 2 may be deployed on TRP 2 and TRP 3. By analogy, submodel N-1 may be deployed on TRP 2N-1 and TRP 2N. Petition 870250089278, dated 10 / 01 / 2025, page 54 / 252 47 / 99

[00173] It should be understood that the first device can determine the input dataset and the training dataset for each submodel. Different submodels have different training datasets. Since a submodel can be deployed on one or more TRPs, the submodel's input dataset can include the measurement parameter from one or more TRPs on which the submodel is deployed. For example, referring to Figure 11C, every 2 TRPs can share a submodel. The input dataset for submodel 0 can include input dataset 1 and / or input dataset 2. Input dataset 1 can include the measurement parameter between the terminal device and TRP 1, and input dataset 2 can include the measurement parameter between the terminal device and TRP 2.

[00174] In addition, the first device can determine, for each submodel, the submodel's input dataset by monitoring the submodel's performance, compare the similarity between the input dataset and the submodel's training dataset to obtain the submodel's similarity comparison result, and determine if the submodel needs to be updated according to the submodel's similarity comparison result.

[00175] In another example, with reference to Figure 11D, the first model may include only one submodel, and a plurality of TRPs share the submodel.

[00176] The submodel input dataset includes the measurement parameter of each TRP in the plurality of TRPs, and the first device can combine the measurement parameters of all TRPs as the input dataset. Petition 870250089278, dated 10 / 01 / 2025, page 55 / 252 48 / 99 of the model. By monitoring the submodel's performance, the first device can determine the combination of measured parameters from all TRPs to obtain the submodel's input dataset, compares the similarity between the input dataset and the submodel's training dataset to obtain the submodel's similarity comparison result, and determines whether the submodel needs to be updated according to the submodel's similarity comparison result.

[00177] As established above, in the model monitoring method provided by the present disclosure modalities, considering the different structures of the first model, only some of the submodels of the first model are updated, thus improving the flexibility of model monitoring. Furthermore, the first device monitors the submodel only by determining the input dataset and the training dataset of the submodel, without inserting the label information of the input dataset. In this way, model monitoring is implemented in a scenario where the label information of the input data cannot be collected, thus improving the feasibility and flexibility of model monitoring.

[00178] The comparison of similarity between the input dataset and the training dataset is described in detail below.

[00179] In one embodiment of the present disclosure, the result of the similarity comparison between the input dataset and the training dataset of the first model / submodel included in the first model may include one or more of the following: a first result of Petition 870250089278, dated 10 / 01 / 2025, page 56 / 252 49 / 99 comparison between a statistical parameter of the input dataset and a statistical parameter of the training dataset; and a second result comparing input data in the input dataset and training data in the training dataset.

[00180] It should be noted that the statistical parameter of the dataset may include one or more of the following: variance, mean, standard deviation, and a distribution of the data in the dataset. The data distribution may be a data distribution satisfied by the data in the dataset, such as a Poisson distribution, a normal distribution, a Gaussian distribution, etc., which is not limited in the embodiments of the present disclosure.

[00181] It should be understood that the first device can macroscopically compare the statistical parameter of the input dataset and the statistical parameter of the training dataset to obtain the first comparison result. In addition, the first device can also microscopically compare the internal data in the two datasets to obtain the second comparison result.

[00182] In a possible implementation, in a case where the result of the similarity comparison between the input dataset and the training dataset includes only the first comparison result between the statistical parameter of the input dataset and the statistical parameter of the training dataset, and the first comparison result indicates that the statistical parameter of the input dataset is similar to the statistical parameter of the training dataset, the Petition 870250089278, dated 10 / 01 / 2025, page 57 / 252 A 50 / 99 result from the similarity comparison between the input dataset and the training dataset indicates similarity. Otherwise, if the first comparison result indicates that the statistical parameter of the input dataset is not similar to the statistical parameter of the training dataset, the similarity result between the input dataset and the training dataset will indicate dissimilarity.

[00183] In another possible implementation, in a case where the result of the similarity comparison between the input dataset and the training dataset includes only the second comparison result between the input data in the input dataset and the training data in the training dataset, and if the second comparison result indicates that the input data in the input dataset are similar to the training data in the training dataset, the result of the similarity comparison between the input dataset and the training dataset indicates similarity. Otherwise, if the second comparison result indicates that the input data in the input dataset are not similar to the training data in the training dataset, the similarity result between the input dataset and the training dataset will indicate dissimilarity.

[00184] In another possible implementation, if the result of the similarity comparison between the input dataset and the training dataset includes the result of the first comparison and the result of the second comparison mentioned above, if both the result of the first Petition 870250089278, dated 10 / 01 / 2025, page 58 / 252 51 / 99 If the results of the second comparison indicate, respectively, that the statistical parameter of the input dataset is similar to the statistical parameter of the training dataset and that the input data in the input dataset are similar to the training data in the training dataset, the result of the similarity comparison between the input dataset and the training dataset indicates similarity. If the first comparison result and the second comparison result indicate that the statistical parameter of the input dataset is similar to the statistical parameter of the training dataset and / or that the input data in the input dataset are similar to the training data in the training dataset, the similarity result between the input dataset and the training dataset indicates dissimilarity.

[00185] In one embodiment of the present disclosure, the first result of comparison between the statistical parameter of the input dataset and the statistical parameter of the training dataset may include: a first result of determining correspondence between the statistical parameter of the input dataset and the statistical parameter of the training dataset and / or a first degree of correspondence between the statistical parameter of the input dataset and the statistical parameter of the training dataset.

[00186] It should be understood that the first result of determining correspondence may be a difficult classification result and may include two results, namely, Petition 870250089278, dated 10 / 01 / 2025, page 59 / 252 52 / 99 Matching and Incompatibility. If each statistical parameter of the input dataset is equal to the corresponding statistical parameter of the training dataset, the first result of the matching determination indicates matching. If any statistical parameter of the input dataset is different from the corresponding statistical parameter of the training dataset, the result of the first matching decision indicates incompatibility.

[00187] In one example, in the case where the statistical parameters include variance, mean, and standard deviation, when the variance, mean, and standard deviation of the input dataset are respectively equal to the variance, mean, and standard deviation of the training dataset, the first result of the matching decision indicates a match. If any of the variances, means, or standard deviations of the input dataset are different from the respective variance, mean, and standard deviation of the training dataset, the first result of the matching determination will indicate a mismatch.

[00188] In one example, in a case where the statistical parameter includes a data distribution and the data distribution of the input dataset is the same as the data distribution of the training dataset, the first match determination result indicates a match. Otherwise, if the data distribution of the input dataset is different from the data distribution of the training dataset, the first match determination result will indicate a mismatch. Petition 870250089278, dated 10 / 01 / 2025, p. 60 / 252 53 / 99

[00189] Whether the data distribution of the input dataset is the same as the data distribution of the training dataset can be determined in several ways.

[00190] In one possible implementation, the data in the training dataset may satisfy a first distribution. In a case where the number of data that satisfy the first distribution in the input dataset is greater than or equal to a first threshold, the data distribution of the input dataset can be determined to be the same as the data distribution of the training dataset. If the number of data that satisfy the first distribution in the input dataset is less than the first threshold, it can be determined that the data distribution of the input dataset is different from the data distribution of the training dataset.

[00191] In another possible implementation, the data in the training dataset satisfy a first distribution. If the ratio between the number of data that satisfy the first distribution in the input dataset and the total number of data in the input dataset is greater than or equal to a second threshold, it can be determined that the data distribution of the input dataset is the same as the data distribution of the training dataset. In the case where the ratio between the number of data that satisfy the first distribution in the input dataset and the total number of data in the input dataset is less than the second threshold, it can be determined that the data distribution of the input dataset is different from the data distribution of the training dataset. Petition 870250089278, dated 10 / 01 / 2025, p. 61 / 252 54 / 99 training data.

[00192] It should be noted that the first distribution can be any one of a Poisson distribution, a normal distribution, or a Gaussian distribution, which is not limited to the modalities of the present disclosure.

[00193] It should also be noted that the first threshold and the second threshold in the above embodiments may be specified by a protocol, may be configured by a network, may be determined according to a predefined rule, or may be predefined, which is not limited to the embodiments of the present disclosure.

[00194] Furthermore, the first degree of correspondence may be a result of smooth sorting, and the value of the first degree of correspondence may be a decimal value greater than or equal to 0.

[00195] In some embodiments, the first degree of correspondence includes one or more of the following: a difference between the variance of the input dataset and the variance of the training dataset; a difference between the mean of the input dataset and the mean of the training dataset; and a mean squared error of the input dataset and the training dataset.

[00196] It should be understood that the first result of comparison between the statistical parameter of the input dataset and the statistical parameter of the training dataset may include: a first result of determining correspondence between the statistical parameter of the input dataset and the statistical parameter of the training dataset and / or a first degree of correspondence between Petition 870250089278, dated 10 / 01 / 2025, p. 62 / 252 55 / 99 is the statistical parameter of the input dataset and the statistical parameter of the training dataset.

[00197] In one possible implementation, in the case where the first result of the comparison includes only the first result of the corresponding determination, the first result of the comparison indicates similarity if the first result of the corresponding decision indicates matching, and the first result of the comparison indicates dissimilarity if the first result of the corresponding determination indicates incompatibility.

[00198] In another possible implementation, in the case where the first comparison result includes only the first degree of correspondence, the first comparison result indicates similarity if the value of the first degree of correspondence is greater than or equal to a third threshold, and the first comparison result indicates dissimilarity if the value of the first degree of correspondence is less than the third threshold.

[00199] In another possible implementation, in the case where the first comparison result includes both the first match determination result and the first degree of match, the first comparison result indicates similarity if the first match determination result indicates a match and the first degree of match is greater than or equal to the third threshold, and the first comparison result indicates dissimilarity if the first match determination result indicates a non-match and / or the first degree of match is less than the third threshold.

[00200] It should be noted that the third threshold Petition 870250089278, dated 10 / 01 / 2025, page 63 / 252 56 / 99 can be specified by a protocol, can be configured by a network, can be determined according to a predefined rule, or can be predefined, which is not limited to the embodiments of the present disclosure.

[00201] In one embodiment of the present disclosure, the second result of comparison between the input data in the input dataset and the training data in the training dataset may include: a second result of determining correspondence between the input data in the input dataset and the training data in the training dataset; and / or a second degree of correspondence between the input data in the input dataset and the training data in the training dataset.

[00202] The second matching determination result can be a hard sorting result and includes two outcomes, namely matching and mismatch. The second degree of matching can be a soft sorting result with a decimal value between 0 and 1.

[00203] It should be noted that the second result of the matching determination can be determined in several ways, three of which are implemented as described below.

[00204] In one implementation, if the number of input data in the input dataset that are identical to the training data in the training dataset is greater than or equal to a fourth threshold, the second match determination result indicates a match. If the number of input data in the input dataset that Petition 870250089278, dated 10 / 01 / 2025, p. 64 / 252 57 / 99 are identical to the training data in the training dataset. If the value is less than the fourth threshold, the second match determination result will indicate a mismatch.

[00205] In another implementation, if the ratio between the number of input data in the input dataset that are identical to the training data in the training dataset and the total number of input data in the input dataset is greater than or equal to a fifth threshold, the second matching determination result indicates matching. If the ratio between the number of input data in the input dataset that are identical to the training data in the training dataset and the total number of input data in the input dataset is less than the fifth threshold, the result of the second matching decision indicates mismatch.

[00206] It should be understood that in the two implementations above, for each input data in the input dataset, the first device can compare the input data with each of the training data in the training dataset and determine if the data in the two datasets correspond to each other according to the number / proportion of identical data in the two datasets.

[00207] In another implementation, for each input data in the input dataset, the first device can compare the input data with a plurality of training data in the training dataset to obtain a degree of match of the input data. If the number of input data with a degree of match greater than one-sixth in the input dataset is greater or Petition 870250089278, dated 10 / 01 / 2025, p. 65 / 252 58 / 99 equals a seventh threshold, the second result of the matching determination indicates matching. If the number of input data with a degree of matching greater than the sixth threshold in the input dataset is less than the seventh threshold, the second result of the matching determination indicates mismatch.

[00208] It should be understood that, in this implementation, the first device can compare the input data in the input dataset with the training data in the training dataset, one by one, calculating the Euclidean distance, cosine similarity, etc., to determine the degree of correspondence of the respective input data. The corresponding degree can be a decimal value between 0 and 1.

[00209] If the number of input data in the input dataset that have a high degree of correspondence (i.e., the degree of correspondence is greater than the sixth threshold) with the training data in the training dataset is large (greater than or equal to the seventh threshold), the second result of determining correspondence between the input dataset and the training dataset indicates correspondence. Otherwise, if the number of input data in the input dataset that have a large degree of correspondence (i.e., the degree of correspondence is greater than the sixth threshold) with the training data in the training dataset is small (less than the seventh threshold), the second result of determining correspondence between the input dataset and the training dataset indicates mismatch.

[00210] In some modalities, the second degree of Petition 870250089278, dated 10 / 01 / 2025, p. 66 / 252 59 / 99 correspondence between the input data in the input dataset and the training data in the training dataset may include: a proportion of the number of input data with a degree of correspondence greater than the fifth threshold in the input dataset relative to the total number of input data in the input dataset; or a proportion of input data in the input dataset identical to the training data in the training dataset relative to the total number of input data in the input dataset.

[00211] It should be noted that at least one of the fourth thresholds, the fifth threshold, the sixth threshold, or the seventh threshold in the above embodiments may be specified by a protocol, may be configured by a network, may be determined according to a predefined rule, or may be predefined, which is not limited to the embodiments of the present disclosure.

[00212] It should be understood that the second result of comparison between the input data in the input dataset and the training data in the training dataset includes: a second result of determining correspondence between the input data in the input dataset and the training data in the training dataset; and / or a second degree of correspondence between the input data in the input dataset and the training data in the training dataset.

[00213] In a possible implementation, in the case where the second result of the comparison includes only the second result of the matching determination, the second result of the comparison indicates similarity if the second result of the matching determination indicates Petition 870250089278, dated 10 / 01 / 2025, page 67 / 252 60 / 99 matching, and the second result of the comparison indicates dissimilarity if the second result of the matching determination indicates no matching.

[00214] In another possible implementation, in the case where the second comparison result includes only the second degree of correspondence, the second comparison result indicates similarity if the value of the second degree of correspondence is greater than or equal to the third threshold, and the results of the second comparison indicate dissimilarity if the value of the second degree of correspondence is less than the third threshold.

[00215] In another possible implementation, in the case where the second result of the comparison includes both the first result of the matching determination and the second degree of matching, the second result of the comparison indicates similarity if the second result of the matching determination indicates matching and the second degree of matching is greater than or equal to the third threshold, and the second result of the comparison indicates dissimilarity if the second result of the matching determination indicates incompatibility and / or the second degree of matching is less than the third threshold.

[00216] As set forth above, in the model monitoring method provided by the embodiments of the present disclosure, the first device can flexibly determine the result of the similarity comparison between the input dataset and the training dataset of the first model in a plurality of different ways and determine the performance of the first model according to the result of the similarity comparison. Petition 870250089278, dated 10 / 01 / 2025, page 68 / 252 61 / 99 In this way, model monitoring is implemented in a scenario where input data label information cannot be collected, thereby improving the feasibility and flexibility of model monitoring.

[00217] It should be noted that updating the first model by the first device may include fine-tuning or model swapping. Model fine-tuning may include fine-tuning of model parameters and / or structural fine-tuning.

[00218] It should be understood that model switching refers to changing from an initial model to a model that is retrained. The amount of data updated in model switching is large, while the amount of data updated in model fine-tuning is small.

[00219] In one embodiment of the present disclosure, in the case where the result of the similarity comparison between the input dataset and the training dataset of the first model includes the result of the smooth classification, the first device can determine a way to update the first model according to the value of the smooth classification result.

[00220] In some modes, if the result of the similarity comparison includes a first degree of correspondence between the statistical parameter of the input dataset and the statistical parameter of the training dataset and / or a second degree of correspondence between the data in the input dataset and the data in the training dataset, the first device may update the first model according to the following method.

[00221] In case of the first degree of correspondence Petition 870250089278, dated 10 / 01 / 2025, p. 69 / 252 If the 62 / 99 and / or second degree of match is less than the third threshold and greater than or equal to the eighth threshold, the first device adjusts a model parameter of the first model. If the first degree of match and / or the second degree of match is less than the eighth threshold, the first device switches to the first model.

[00222] It should be understood that the eighth threshold is less than the third threshold. When the first degree of correspondence and / or the second degree of correspondence is less than the third threshold and greater than or equal to the eighth threshold, the input dataset of the first model is not similar to the training dataset of the first model. In this case, the performance of the first model is poor, and the first device can adjust or fine-tune the model parameter and / or the model structure of the first model so that the performance of the first model can be quickly recovered.

[00223] When the first degree of matching and / or the second degree of matching is less than the eighth limit, the performance of the first model is very poor and the first device needs to switch to the first model to ensure the model's performance.

[00224] In this way, the first device can determine the matching status between the input dataset and the output dataset according to the value of the smooth classification result generated by the first model and determine whether to perform fine-tuning of the model or model switching according to the matching status, thus improving the flexibility of model updating. Petition 870250089278, dated 10 / 01 / 2025, page 70 / 252 63 / 99

[00225] In one embodiment of the present disclosure, the first device may be a first terminal device. That is, the first device may be the terminal device to be processed (e.g., a terminal device to be positioned, a terminal device for which channel estimation information must be determined), or a terminal device with a processing requirement (e.g., a terminal device with a positioning requirement, or a terminal device with a channel estimation requirement), or a terminal device deployed with a first model. The first terminal device may monitor the performance of the first model under the network device configuration.

[00226] In some embodiments, in the case where the first device is a first terminal device, prior to operation 810, the model monitoring method provided by the embodiments of the present disclosure may further include the following operations.

[00227] The first terminal device receives the first information transmitted by a network device. The first information is used to configure information to monitor the first model, and the input data set is measured based on the information.

[00228] It should be understood that, before the first terminal device monitors the performance of the first model, the network device may configure further information to monitor the first model for the first terminal device, and the first terminal device measures according to the configured information (for example, the first terminal device performs downlink measurement or Petition 870250089278, dated 10 / 01 / 2025, page 71 / 252 64 / 99 transmits the uplink reference signal) to obtain the input data set from the first model.

[00229] In some embodiments, the network device may configure the information to monitor the first model for the first terminal device by means of upper layer signaling (e.g., RRC signaling) or physical layer signaling (e.g., downlink control information (DCI)), which is not limited to the embodiments of the present disclosure.

[00230] In some embodiments, the initial information may include one or more of: a reference signal type, a measurement type, and trigger information. Trigger information is used to trigger the performance monitoring of the first model.

[00231] Optionally, the reference signal type may include one or more of a Positioning Reference Signal (PRS), a PBCH Synchronization and Block Signal (SSB), and a Sounding Reference Signal (SRS).

[00232] Optionally, the measurement type may include any periodic measurement, aperiodic measurement, semi-persistent measurement, or time window measurement.

[00233] For periodic measurement, the network device can set a time period to measure the reference signal in the monitoring process of the first model, and the number of measurements of the reference signal in each time period, etc., which are not limited in the embodiments of the present disclosure.

[00234] In aperiodic measurement, the reference signal is measured only a predefined number of times. For aperiodic measurement, the network device can Petition 870250089278, dated 10 / 01 / 2025, page 72 / 252 65 / 99 configure the number of measurements of the reference signal.

[00235] For semi-persistent measurement, the network device can configure a continuous measurement duration, a measurement period within the time duration, the number of measurements within each period, or other information.

[00236] For time window measurement, the network device can configure a time window start position, time window length, and the number of measurements within the time window, or other information.

[00237] In some embodiments, the first device is a first terminal device, and the model monitoring method provided by the embodiments of the present disclosure may additionally include the following operations.

[00238] The first terminal device receives the second piece of information transmitted by a network device, and the second piece of information is used to configure the input dataset and / or the training dataset of the first model.

[00239] It should be understood that, if the first device is a first terminal device, the input data set (mainly including the downlink measurement parameter) of the first model can be measured by the first terminal device. The input data set (mainly including the uplink measurement parameter) can also be measured by the network device, and the network device configures the input data set for the first terminal device through the second information, which is not limited in the modes of Petition 870250089278, dated 10 / 01 / 2025, page 73 / 252 66 / 99 present revelation.

[00240] In addition, the training dataset for the first model can be configured by the network device for the first terminal device using the second information.

[00241] In some embodiments, if the first device is the terminal device, the model monitoring method provided by the embodiments of this disclosure may additionally include the following operation.

[00242] The first terminal device transmits the result of the similarity comparison between the input dataset and the training dataset of the first model to the network device.

[00243] It should be understood that, after determining the result of the similarity comparison between the input dataset and the training dataset of the first model, the first terminal device can transmit the result of the similarity comparison to the network device, so that the network device determines whether to update the first model according to the result of the similarity comparison.

[00244] In some embodiments, if the first device is the first terminal device, the model monitoring method provided by the embodiments of this disclosure may additionally include the following operation.

[00245] The first terminal device receives the third piece of information transmitted by the network device, and this third piece of information is used to indicate the Petition 870250089278, dated 10 / 01 / 2025, page 74 / 252 67 / 99 update of the first model.

[00246] The third information may indicate a way to update the first model. For example, the third information may instruct the first terminal device to adjust the first model, or the third information may instruct the first terminal device to switch the first model. If the third information instructs the first terminal device to adjust the first model, the third information may also include a model parameter and / or model structure to be adjusted. If the third information may instruct the first terminal device to change the first model, the third information may also include content related to the first model after the change.

[00247] As set forth above, in the model monitoring method provided by the embodiment of the present disclosure, when monitoring the first model, the first device compares the similarity between the input dataset of the first model and the training dataset of the first model and determines the performance of the first model using the result of the similarity comparison, without collecting the label information corresponding to the input data. In this way, model monitoring is implemented in a scenario where the label information of the input data cannot be collected, thus improving the feasibility and flexibility of model monitoring.

[00248] Hereafter, the model monitoring method provided by the embodiments of the present disclosure will be described in combination with specific application scenarios. Petition 870250089278, dated 10 / 01 / 2025, page 75 / 252 68 / 99

[00249] In one example, the first model can be used for direct positioning / assisted positioning. The first model is deployed in the LMF. That is, the LMF can monitor the performance of the first model. The operation in which the LMF monitors the performance of the first model can include the following operations S1 to S3.

[00250] In S1, the LMF determines an input dataset for the first model.

[00251] The input dataset for the first model may include a measurement parameter associated with the UE position. The measurement parameter may include one or more of: the CIR, the PDP, the ToA, the path phase, the UL TDOA, the UL AoD, the RSRP, the RSRQ, the UL RTOA, etc.

[00252] The measurement parameter associated with the UE position can be a measurement parameter between the UE and one or more TRPs. The measurement parameter associated with the UE position can be obtained by measuring, through one or more TRPs, the uplink reference signal (e.g., SRS) transmitted by the UE. Then, one or more TRPs can transmit the measured measurement parameter associated with the UE position to the LMF.

[00253] Note also that the uplink reference signal transmitted by the UE can be configured by the LMF to the UE via the gNB. The gNB can be one of the TRPs mentioned above, or one or more of the TRPs mentioned above can be included in the gNB's coverage. When one or more TRPs are included in the gNB's coverage, the gNB can configure the uplink reference signal-related information to the UE via any of the TRPs.

[00254] In S2, the LMF compares the similarity between the Petition 870250089278, dated 10 / 01 / 2025, page 76 / 252 69 / 99 input dataset of the first model and the training dataset of the first model to obtain a similarity comparison result.

[00255] The result of the similarity comparison may include a comparison result between the statistical parameters of the input dataset of the first model and the statistical parameters of the training dataset of the first model; and / or a comparison result, for each input data in the input dataset of the first model, between the input data and the training data in the training dataset of the first model.

[00256] It should be noted that the similarity comparison process can be understood with reference to the description in the aforementioned modalities, and will not be detailed here for the sake of brevity.

[00257] In addition, the training dataset for the first model can be acquired by LMF from a device used to train the first model. If the device used to train the first model is a third-party server, LMF can request the training dataset for the first model from the third-party server. If the device used to train the first model is LMF itself, LMF can directly acquire the training dataset for the first model.

[00258] In S3, the LMF determines whether to update the first model and a way to update the first model according to the result of the similarity comparison.

[00259] In a possible implementation, if the result of the similarity comparison indicates that the set Petition 870250089278, dated 10 / 01 / 2025, page 77 / 252 If the 70 / 99 input dataset and the training dataset of the first model are similar, the LMF may determine that the performance of the first model satisfies a condition and not update the first model. If the result of the similarity comparison indicates that the input dataset and the training dataset of the first model are not similar, the LMF may determine that the performance of the first model cannot satisfy the condition and decide to update the first model.

[00260] In another possible implementation, the input dataset of the first model may include measurement parameters associated with the positions of k PRUs and actual position coordinates corresponding to the measurement parameters of the k PRUs (direct positioning) or actual intermediate parameters related to the positions (assisted positioning). The LMF may verify the actual position coordinates and estimated position coordinates of the k PRUs emitted by the first model, or the actual intermediate parameters of the k PRUs and the estimated intermediate parameters of the k PRUs emitted by the first model. If the difference between the actual position coordinates or the actual intermediate parameters of the PRUs and the estimated position coordinates or the estimated intermediate parameters of the PRUs emitted by the first model is less than a threshold of 3, the verification is successful; otherwise, the verification fails.

[00261] Note that it can be determined that the verification is successful when the verification of all k PRUs is successful, or the verification of a predetermined proportion of the k PRUs is successful.

[00262] Furthermore, if the result of the comparison Petition 870250089278, dated 10 / 01 / 2025, p. 78 / 252 If the similarity between the input dataset and the training dataset of the first model indicates similarity and the verification is successful, the LMF can determine that the performance of the first model satisfies a condition and does not update the first model. In the case where the result of the similarity comparison indicates that the input dataset and the training dataset of the first model are not similar and / or the verification fails, the LMF can determine that the performance of the first model cannot satisfy the condition and determine to update the first model.

[00263] It should be noted that updating the first model may include fine-tuning the model and changing the model.

[00264] Optionally, the method of updating the first model can be defined by the protocol. The protocol defines that the update of the first model can be a fine-tuning or a model swap.

[00265] Optionally, the method for updating the first model can also be determined by the first device according to a predefined rule. For example, when the similarity comparison result is a soft classification result, the LMF can determine to perform model fine-tuning when the similarity comparison result is less than threshold 1 and greater than or equal to threshold 2, and determine to perform model swapping when the similarity comparison result is less than threshold 2.

[00266] In another example, the first model can be used for direct positioning / assisted positioning. The first model can be deployed on the gNB. That is, the gNB Petition 870250089278, dated 10 / 01 / 2025, page 79 / 252 72 / 99 can monitor the performance of the first model.

[00267] In a possible implementation, in a scenario where the first model is used for direct placement, the operation in which gNB monitors the performance of the first model may include the following operations.

[00268] In S1, gNB determines an input dataset for the first model.

[00269] The input dataset for the first model may include a measurement parameter associated with the UE position. The measurement parameter may include one or more of: the CIR, the PDP, the ToA, the path phase, the UL TDOA, the UL AoD, the RSRP, the RSRQ, the UL RTOA, etc.

[00270] The measurement parameter associated with the UE position can be a measurement parameter between the UE and one or more TRPs. The measurement parameter associated with the UE position can be obtained by measuring, through one or more TRPs, the uplink reference signal (e.g., SRS) transmitted by the UE. Then, one or more TRPs can transmit the measured measurement parameter associated with the UE position to the LMF.

[00271] gNB can be a TRP, or the aforementioned one or more TRPs can be included in gNB coverage. In a scenario where one or more TRPs are included in gNB coverage, gNB can acquire the measurement parameter related to the UE position obtained through measurement by one or more TRPs, to obtain the input dataset for the first model.

[00272] In S2, gNB compares the similarity between the input dataset of the first model and the training dataset of the first model to obtain a Petition 870250089278, dated 10 / 01 / 2025, page 80 / 252 73 / 99 similarity comparison result.

[00273] The result of the similarity comparison may include a comparison result between the statistical parameters of the input dataset of the first model and the statistical parameters of the training dataset of the first model; and / or a comparison result, for each input data in the input dataset of the first model, between the input data and the training data in the training dataset of the first model.

[00274] It should be noted that the similarity comparison process can be understood with reference to the description in the aforementioned modalities, and will not be detailed here for the sake of brevity.

[00275] In addition, the first model training dataset can be acquired by gNB from a device used to train the first model. If the device used to train the first model is a third-party server / LMF, gNB can request the first model training dataset from the third-party server / LMF. If the device used to train the first model is gNB itself, gNB can directly acquire the first model training dataset.

[00276] In S3, gNB determines whether to update the first model and a way to update the first model according to the result of the similarity comparison.

[00277] Note that gNB determines whether to update the first model and how to update the first model based on the result of the similarity comparison, referring to the relevant description mentioned above, where the Petition 870250089278, dated 10 / 01 / 2025, page 81 / 252 74 / 99 LMF determines whether to update the first model and how to update the first model based on the result of the similarity comparison, which will not be detailed here for the sake of brevity.

[00278] In a possible implementation, the first model may include one or more submodels in a scenario where the first model is used for assisted positioning. One or more submodels are deployed on one or more TRPs within the gNB coverage. In this scenario, gNB may obtain input results from different submodels to position the UE in conjunction with the input results. In this scenario, the operation where gNB monitors the performance of the first model may include the following operations.

[00279] In S1, gNB determines an input dataset for each submodel of the first model.

[00280] With reference to Figure 11A, the first model may include N different submodels, and the N submodels are submodel 0, submodel 1 to submodel N-1. Submodel 0 is deployed on TRP 0, submodel 1 is deployed on TRP 1, and so on. The input dataset for each submodel may be a TRP measurement parameter on which the respective submodel is deployed. In other words, the input dataset for each submodel is a measurement parameter between the TRP on which the respective submodel is deployed and the terminal device, which includes, but is not limited to: the CIR, the RSRP, the DL RSTD, the TOA, the DL TDOA, the DL RSTD, the PDP, the UL RSTD, the UL TDOA, the UL AoD, the UL RTOA, the azimuth and the zenith between the terminal device and the TRP. In other words, the input dataset for submodel 0 could be the input dataset 0, which includes the parameter of Petition 870250089278, dated 10 / 01 / 2025, p. 82 / 252 75 / 99 measurement between the terminal device and TRP 0. The input dataset for submodel 1 can be input dataset 1, which includes the measurement parameter between the terminal device and TRP 1. By analogy, the input dataset for submodel N-1 can be input dataset N-1, which includes the measurement parameter between the terminal device and TRP N-1.

[00281] With reference to Figure 11B, the first model may include N identical submodels, and the N submodels are deployed on N TRPs, respectively. The input dataset for each submodel may be a TRP measurement parameter on which the respective submodel is deployed. In other words, the input dataset for each submodel is a measurement parameter between the TRP on which the respective submodel is deployed and the terminal device, which includes, but is not limited to: the CIR, the RSRP, the DL RSTD, the TOA, the DL TDOA, the DL RSTD, the PDP, the UL RSTD, the UL TDOA, the UL AoD, the UL RTOA, the azimuth and zenith between the terminal device and the TRP.

[00282] With reference to Figure 11C, the first model can include N submodels (submodel 0 to submodel N1), the N submodels are deployed in 2N TRPs, and every 2 TRPs can be deployed with the same submodel. Submodel 0 can be deployed in TRP 0 and TRP 1, submodel 2 can be deployed in TRP 2 and TRP 3. By analogy, submodel N-1 can be deployed in TRP 2N-1 and TRP 2N. The input dataset of each submodel can include one or more measurement parameters from one or more TRPs in which the submodel is deployed. Specifically, referring to Figure 11C, every 2 TRPs can share a submodel. The dataset of Petition 870250089278, dated 10 / 01 / 2025, page 83 / 252 76 / 99 Submodel 0 input may include input dataset 1 and / or input dataset 2. Input dataset 1 may include the measurement parameter between the terminal device and TRP 1, and input dataset 2 may include the measurement parameter between the terminal device and TRP 2.

[00283] With reference to Figure 11D, the first model can include only one submodel, and a plurality of TRPs share the submodel. The input dataset includes a measurement parameter from each of the pluralities of TRPs, and the first device can combine the measurement parameters of all TRPs as the model's input dataset.

[00284] In S2, gNB compares, for each submodel, the similarity between the input dataset of the respective submodel and the training dataset of the respective submodel to obtain a similarity comparison result for the respective submodel.

[00285] It should be noted that training datasets for identical submodels can be identical, while training datasets for different submodels are different. gNB can acquire the training dataset for each submodel. The training dataset can be acquired by gNB from the device used to train the submodels. If the device used to train the submodels is a third-party server / LMF, gNB can request the training datasets for the submodels from the third-party server / LMF. If the device used to train the submodels is gNB itself, gNB can directly acquire the training datasets for the submodels. Petition 870250089278, dated 10 / 01 / 2025, p. 84 / 252 77 / 99

[00286] The result of the similarity comparison may include a comparison result between the statistical parameter of the submodel's input dataset and the statistical parameter of the submodel's training dataset; and / or a comparison result, for each input data point in the submodel's input dataset, of the input data and the training data in the submodel's training dataset.

[00287] It should be understood that the process of comparing similarity can be understood by referring to the description in the aforementioned modalities, and will not be elaborated on here for the sake of brevity.

[00288] In S3, gNB determines, for each submodel of the first model, whether to update the submodel and a way to update the submodel according to the result of the submodel similarity comparison.

[00289] It should be understood that if the results of the similarity comparison between the input datasets and the training datasets of the submodels deployed in a portion of the plurality of TRPs indicate that the input datasets of the submodels are not similar to the training datasets, gNB updates the submodels deployed in the portion of the plurality of TRPs. That is, in the scenario where the first model is used for assisted positioning, gNB can separately monitor the performance of each of the various submodels and update only the submodels for which the result of the similarity comparison between the input dataset and the output dataset indicates dissimilarity.

[00290] In another example, the first model is used Petition 870250089278, dated 10 / 01 / 2025, p. 85 / 252 78 / 99 for direct positioning / assisted positioning. The first model is deployed in the UE. That is, the UE can monitor the performance of the first model. The operation that the UE monitors the performance of the first model may include the following operations.

[00291] In S1, the UE receives the first information transmitted by a network device and the first information is used to configure information to monitor the first model.

[00292] The network device can be gNB or LMF.

[00293] It should be noted that the initial information may also be called monitoring instruction information, and the initial information may include one or more of the following: a reference signal type, a measurement type, and trigger information. Trigger information is used to trigger performance monitoring for the first model.

[00294] Optionally, the reference signal type may include one or more of: a PRS, an SSB, and an SRS.

[00295] Optionally, the measurement type may include any of the following: periodic measurement, aperiodic measurement, semi-persistent measurement, or time window measurement.

[00296] In S2, the UE receives the second piece of information transmitted by the network device, and the second piece of information is used to configure an input dataset and / or a training dataset for the first model. Petition 870250089278, dated 10 / 01 / 2025, page 86 / 252 79 / 99

[00297] It is important to note that S2 is an optional operation.

[00298] When the input dataset (mainly including the downlink measurement parameter) of the first model is measured by the UE, S2 may not run. When the input dataset (mainly including the uplink measurement parameter) of the first model is measured by the network device, the network device can configure the input dataset for the UE through the second information in S2.

[00299] Furthermore, when the UE is a device for training the first model, the UE can directly acquire the first model training dataset without performing S2. When the device for training the first model is a device other than the UE (e.g., a third-party server / LMF / gNB, etc.), the network device can configure the training dataset for the UE via the second information in S2.

[00300] In S3, UE compares the similarity between the input dataset of the first model and the training dataset of the first model to obtain a similarity comparison result.

[00301] The result of the similarity comparison may include a comparison result between the statistical parameters of the input dataset of the first model and the statistical parameters of the training dataset of the first model; and / or a comparison result, for each input data point in the input dataset of the first model, between the input data and the training data in the training dataset of the Petition 870250089278, dated 10 / 01 / 2025, page 87 / 252 80 / 99 first model.

[00302] It should be noted that the similarity comparison process can be understood with reference to the description in the aforementioned modalities, and will not be detailed here for the sake of brevity.

[00303] In S4, the UE transmits the result of the similarity comparison between the input dataset and the training dataset of the first model to a network device.

[00304] It should be understood that, after determining the result of the similarity comparison between the input dataset and the training dataset of the first model, the UE can transmit the result of the similarity comparison to the network device, so that the network device determines whether to update the first model according to the result of the similarity comparison.

[00305] In the S5, the UE receives third-party information transmitted by the network device, and this third-party information is used to indicate the first model update.

[00306] Third-party information may indicate a way to update the first model. For example, third-party information may instruct the UE to adjust the first model, or third-party information may instruct the UE to change the first model. If third-party information instructs the UE to adjust the first model, third-party information may also include a model parameter and / or a model structure to be adjusted. If third-party information may instruct the UE to change the first model, third-party information may also include content related to Petition 870250089278, dated 10 / 01 / 2025, page 88 / 252 81 / 99 first model after the exchange.

[00307] As set forth above, in the model monitoring method provided by the embodiment of the present disclosure, when different devices monitor the first model, the device can compare the similarity between the input dataset of the first model and the training dataset of the first model without collecting label information corresponding to the input data, and determine the performance of the first model using the result of the similarity comparison. In this way, model monitoring is implemented in a scenario where label information from the input data cannot be collected, thus improving the feasibility and flexibility of model monitoring.

[00308] The preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments; various simple modifications may be made to the technical solution of the present disclosure within the scope of the technical concept of the present disclosure, and these simple modifications all belong to the scope of protection of the present disclosure. For example, the specific technical features described in the specific embodiments above may be combined in any suitable manner without contradiction, and various possible combinations are not described in the present disclosure to avoid unnecessary repetition. As another example, the various embodiments of the present disclosure may be combined arbitrarily, and the combination shall also be considered as a disclosure of the present disclosure, provided that it does not deviate Petition 870250089278, dated 10 / 01 / 2025, page 89 / 252 82 / 99 of the concept of the present disclosure. As another example, under the premise that there is no conflict, the embodiments described in the present disclosure and / or the technical features in the embodiments may be arbitrarily combined with the state of the art, and the technical solution obtained after the combination must also be within the scope of protection of the present disclosure.

[00309] It should also be understood that, in various embodiments of the present disclosure method, the sequence numbers of the processes described above do not signify the order of execution, and the order of execution of each process should be determined by its function and inherent logic, and should not limit the implementation of the embodiments of the present disclosure.

[00310] Figure 12 is a schematic structural diagram of a composition of a model 1200 monitoring device according to an embodiment of the present disclosure, which is applied to the first device. As illustrated in Figure 12, the model 1200 monitoring device includes a determination unit 1201.

[00311] The determination unit 1201 is configured to determine a similarity comparison result between an input dataset and a training dataset of a first model. The similarity comparison result is used to determine the performance of the first model.

[00312] Optionally, the first model is used for direct positioning / assisted positioning for a first terminal device, and the input dataset includes a measurement parameter associated with a position of the first terminal device.

[00313] Optionally, the measurement parameter is a Petition 870250089278, dated 10 / 01 / 2025, p. 90 / 252 83 / 99 measurement parameter between the first terminal device and one or more network devices.

[00314] Optionally, the training dataset includes a measurement parameter between a second terminal device and one or more network devices and label information corresponding to each measurement parameter.

[00315] Optionally, the measurement parameter includes one or more of: a CIR, a PDP, a ToA, a path phase, a UL TDoA, a DL TdoA, a UL AoD, a DL AoD, a UL RTOA, a DL RSTD, an RSRP, an RSRQ, an azimuth, and a zenith.

[00316] Optionally, if the first model is used for direct placement of the terminal device, the label information corresponding to each training data point includes a location between the second terminal device and a network device.

[00317] Optionally, if the first model is used for assisted positioning for the terminal device, the label information corresponding to each training data point includes one or more of the following: a ToA, a DL TDoA, a DL AoD, a DL RSTD, a UL TDoA, a UL AoD, a UL RTOA, an RSRP, an RSRQ, a LOS / NLOS identification, and distance information between the second terminal device and the network device.

[00318] Optionally, the determination unit 1201 is further configured to: in the case where the result of the similarity comparison indicates that the input dataset is similar to the training dataset, determine that the performance of the first model satisfies a condition; and in the case where the result of the comparison of Petition 870250089278, dated 10 / 01 / 2025, p. 91 / 252 84 / 99 similarity indicates that the input dataset is not similar to the training dataset; therefore, update the first model.

[00319] Optionally, the input dataset also includes a measurement parameter associated with a position of the positioning reference unit.

[00320] The determination unit 1201 is further configured to: in the case where a verification result indicates a success, and the similarity comparison result indicates that the input dataset is similar to the training dataset, determine that the performance of the first model satisfies a condition; and in a case where the verification result indicates a failure and / or the similarity comparison result indicates that the input dataset is not similar to the training dataset, determine to update the first model.

[00321] The verification result indicates the result of verifying whether an estimated position of the positioning reference unit is successfully combined with an actual position of the positioning reference unit, and the estimated position is obtained by processing the measurement parameter associated with the position of the positioning reference unit by the first model.

[00322] Optionally, there is a plurality of positioning reference units.

[00323] If the estimated positions of the plurality of positioning reference units are successfully combined with the actual positions of the plurality of positioning reference units, the result of the verification indicates success; or if the number of Petition 870250089278, dated 10 / 01 / 2025, page 92 / 252 85 / 99 positioning reference units whose estimated positions are successfully combined with the actual positions in the plurality of positioning reference units is greater than a first number, the verification result indicates a success.

[00324] Optionally, the similarity comparison result includes one or more of: a first comparison result between a statistical parameter of the input dataset and a statistical parameter of the training dataset; and a second comparison result between the input data in the input dataset and the training data in the training dataset.

[00325] Optionally, the first comparison result includes: a first result of determining correspondence between the statistical parameter of the input dataset and the statistical parameter of the training dataset; and / or a first degree of correspondence between the statistical parameter of the input dataset and the statistical parameter of the training dataset.

[00326] Optionally, the statistical parameter includes one or more of: a variance, a mean, a standard deviation, and a data distribution.

[00327] Optionally, if the statistical parameters of the input dataset are the same as the respective statistical parameters of the training dataset, the first result of the match determination indicates a match. If any statistical parameter of the input dataset is different from the respective statistical parameter of the training dataset, the first result of the match determination Petition 870250089278, dated 10 / 01 / 2025, p. 93 / 252 86 / 99 correspondence indicates incompatibility.

[00328] Optionally, the data in the training dataset meet a first distribution. If the number of data that satisfy the first distribution in the input dataset is greater than or equal to a first limit, a data distribution of the input dataset is equal to a data distribution of the training dataset. If the number of data that satisfy the first distribution in the input dataset is less than the first limit, the data distribution of the input dataset is different from the data distribution of the training dataset.

[00329] Optionally, the data in the training dataset meet a first distribution. If the ratio of the number of data that satisfy the first distribution in the input dataset to the total number of data in the input dataset is greater than or equal to the second limit, the data distribution of the input dataset is the same as the data distribution of the training dataset. If the ratio of the number of data that satisfy the first distribution in the input dataset to the total number of data in the input dataset is less than the second limit, the data distribution of the input dataset is different from the data distribution of the training dataset.

[00330] Optionally, the first degree of correspondence includes one or more of: a difference between a variance of the input dataset and a variance of the training dataset; a difference between a mean of the input dataset and a mean of the training dataset. Petition 870250089278, dated 10 / 01 / 2025, p. 94 / 252 87 / 99 training data; and a mean squared error of the input dataset and the training dataset.

[00331] Optionally, if the first match determination result indicates a match and the first degree of match is greater than or equal to a third threshold, the first comparison result indicates similarity. If the first match determination result indicates incompatibility and / or the first degree of match is less than the third threshold, the first comparison result indicates dissimilarity.

[00332] Optionally, the second comparison result includes: a second result of determining the correspondence between the input data in the input dataset and the training data in the training dataset; and / or a second degree of correspondence between the input data in the input dataset and the training data in the training dataset.

[00333] Optionally, if the number of input data in the input dataset that are identical to the training data in the training dataset is greater than or equal to a fourth threshold, the second match determination result indicates a match. If the number of input data in the input dataset that are identical to the training data in the training dataset is less than the fourth threshold, the second match determination result indicates a mismatch.

[00334] Optionally, if the ratio between the number of input data in the input dataset that are identical to the training data in the dataset of Petition 870250089278, dated 10 / 01 / 2025, p. 95 / 252 If the ratio between the number of input data in the training dataset that are identical to the training data in the training dataset and the total number of input data in the training dataset is greater than or equal to one-fifth of the threshold, the second matching determination result indicates a match. If the ratio between the number of input data in the training dataset that are identical to the training data in the training dataset and the total number of input data in the training dataset is less than one-fifth of the threshold, the result of the second matching decision indicates a mismatch.

[00335] Optionally, the determination unit 1201 is further configured to, for each input data in the input dataset, compare the input data with a plurality of training data in the training dataset to obtain a degree of match of the input data.

[00336] If the number of input data with a degree of match greater than one-sixth threshold in the input dataset is greater than or equal to one-seventh threshold, the second result of the match determination indicates a match. If the number of input data with a degree of match greater than the sixth threshold in the input dataset is less than the seventh threshold, the second result of the match determination indicates a mismatch.

[00337] Optionally, the second degree of matching includes: a ratio of the number of input data with a degree of matching greater than the fifth threshold in the input dataset to the total number of input data in the input dataset; or a ratio of the number of input data in the dataset Petition 870250089278, dated 10 / 01 / 2025, p. 96 / 252 89 / 99 of the input data are identical to the training data in the training dataset for the total number of input data in the input dataset.

[00338] Optionally, in the case where the second result of the matching determination indicates matching and the second degree of matching is greater than or equal to a third threshold, the second result of the comparison indicates similarity. In a case where the second result of the matching determination indicates incompatibility and / or the second degree of matching is less than the third threshold, the second result of the comparison indicates dissimilarity.

[00339] Optionally, if each of the first comparison results and the second comparison result indicate similarity, the input dataset is similar to the training dataset. If the first comparison result and / or the second comparison result indicate dissimilarity, the input dataset is not similar to the training dataset.

[00340] Optionally, the result of the similarity comparison includes a first degree of correspondence between a statistical parameter of the input dataset and a statistical parameter of the training dataset and / or a second degree of correspondence between data from the input dataset and data from the training dataset; and

[00341] The determination unit 1201 is further configured for: in the case where the first degree of correspondence and / or the second degree of correspondence is less than a third threshold and greater than or equal to one eighth Petition 870250089278, dated 10 / 01 / 2025, p. 97 / 252 90 / 99 threshold, allow the first device to adjust a model parameter of the first model; and in the case where the first degree of matching and / or the second degree of matching is less than the eighth threshold, allow the first device to switch to the first model.

[00342] Optionally, the first device is a network device, the first model includes one or more submodels, the submodel(s) are deployed in a plurality of TRPs.

[00343] The determination unit 1201 is further configured to: determine a similarity comparison result between the input dataset and the training dataset of each submodel; and, if the similarity comparison results between the input datasets and the training datasets of the submodels deployed in a part of the plurality of TRPs indicate that the input datasets of the submodels are not similar to the training datasets of the submodels, update the submodels deployed in the part of the plurality of TRPs.

[00344] Optionally, the first model includes N submodels, the N submodels are deployed in N TRPs, and N is an integer greater than 1. The input dataset for each submodel includes a measurement parameter of the TRP in which the submodel is deployed.

[00345] Optionally, the first model includes N submodels, N is an integer greater than 1, the N submodels are deployed in M ​​TRPs, M is an integer greater than N. At least some of the M TRPs share the same submodel. The input dataset of each submodel includes the parameter of Petition 870250089278, dated 10 / 01 / 2025, page 98 / 252 91 / 99 measurement of one or more TRPs in which the submodels are deployed.

[00346] Optionally, the first model includes N different submodels, and the training datasets for different submodels are different.

[00347] Optionally, the first model includes N identical submodels, and the training datasets for the submodels are the same.

[00348] Optionally, the first model includes a submodel, and the plurality of TRPs shares the submodel. The submodel input dataset includes the measurement parameter for each of the pluralities of TRPs.

[00349] Optionally, the first device is the first terminal device. The first device also includes a transceiver unit, and the transceiver unit is configured to receive the first information transmitted by a network device. The first information is used to configure information to monitor the first model, and the input data set is measured based on the information.

[00350] Optionally, the initial information includes one or more of: a reference signal type, a measurement type, and trigger information. Trigger information is used to trigger performance monitoring for the first model.

[00351] Optionally, the reference signal type includes: a positioning reference signal (PRS), a synchronization signal block (SSB), and a sounding reference signal (SRS).

[00352] Optionally, the measurement type includes: periodic measurement, aperiodic measurement, semi-persistent measurement Petition 870250089278, dated 10 / 01 / 2025, page 99 / 252 92 / 99 or time window measurement.

[00353] Optionally, the first device is the first terminal device. The transceiver unit is also configured to receive the second piece of information transmitted by a network device. The second piece of information is used to configure the input dataset and / or the training dataset of the first model.

[00354] Optionally, the first device is the first terminal device. The transceiver unit is further configured to transmit the result of the similarity comparison between the input dataset and the training dataset of the first model to a network device.

[00355] Optionally, the first device is the first terminal device. The transceiver unit is also configured to receive third-party information transmitted by the network device. This third-party information is used to indicate the first model update.

[00356] It should be understood by those skilled in the art that the above description of the model monitoring device according to the embodiments of the present disclosure can be understood with reference to the description of the model monitoring method according to the embodiments of the present disclosure.

[00357] Figure 13 is a schematic structural diagram of a 1300 communication device according to an embodiment of the present disclosure. The communication device may be the first device. The device of Petition 870250089278, dated 10 / 01 / 2025, page 100 / 252 The 93 / 99 communication 1300 illustrated in Figure 13 includes a processor 1310, and the processor 1310 can call and execute a computer program from memory to implement the method in the embodiments of the present disclosure.

[00358] Optionally, as illustrated in Figure 13, the communication device 1300 may additionally include a memory 1320. The processor 1310 may call and execute the computer program from memory 1320 to implement the method in the disclosure modes.

[00359] The 1320 memory can be a separate device independent of the 1310 processor, or it can be integrated into the 1310 processor.

[00360] Optionally, communication device 1300 may be the first device in the embodiments of the present disclosure, and communication device 1300 may implement the corresponding operations implemented by the first device in the methods of the embodiments of the disclosure. For the sake of brevity, this will not be elaborated upon in this document.

[00361] Figure 14 is a schematic structural diagram of a chip according to an embodiment of the present disclosure. The 1400 chip illustrated in Figure 14 includes a 1410 processor, and the 1410 processor can call and execute computer programs from a memory to implement the method in the embodiments of the present disclosure.

[00362] Optionally, as illustrated in Figure 14, the 1400 chip may additionally include a 1420 memory. The 1410 processor may call and execute the computer program from the 1420 memory to implement the method in the disclosure modes. Petition 870250089278, dated 10 / 01 / 2025, pp. 101 / 252 94 / 99

[00363] The 1420 memory can be a separate device independent of the 1410 processor, or it can also be integrated into the 1410 processor.

[00364] Optionally, the 1400 chip may also include a 1430 input interface. The 1410 processor may control the 1430 input interface to communicate with other devices or chips and, in particular, may obtain information or data sent by other devices or chips.

[00365] Optionally, the 1400 chip may also include a 1440 output interface. The 1410 processor may control the 1440 output interface to communicate with other devices or chips. In particular, the processor may control the output interface to send information or data to other devices or chips.

[00366] Optionally, the chip may be applied to the first device in the embodiments of this disclosure, and the chip may implement the corresponding operations implemented by the first device in each method of the embodiments of the disclosure. For the sake of brevity, this will not be elaborated upon in this document.

[00367] It should be understood that the chip mentioned in the embodiments of the present disclosure may also be referred to as a system-level chip, a system chip, a system-on-a-chip, or a system-on-a-chip, etc.

[00368] It should be understood that the processor in the embodiment of the present disclosure may be an integrated circuit chip with signal processing capability. During implementation, the various operations of the embodiments of the above method may be implemented by hardware integrated logic circuits in the processor or instructions in the form of Petition 870250089278, dated 10 / 01 / 2025, page 102 / 252 95 / 99 software. The above processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, and a discrete hardware component. The methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure may be implemented or executed. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor or similar. The operations of the method disclosed in connection with the embodiment of this disclosure may be directly incorporated as the hardware decoding processor, or the combination of hardware and software modules in the decoding processor, for implementation.The software module can be located in random memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, a register, and other storage media mature in the art. The storage media is located in memory, and the processor reads information from memory and implements the operations of the above method in combination with its hardware.

[00369] It is understood that memory in the embodiments of the present disclosure may be volatile or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrical EPROM (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), Petition 870250089278, dated 10 / 01 / 2025, page 103 / 252 96 / 99 which is used as an external cache. By way of exemplary illustration, but not as a limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct Rambus RAM (DR RAM). It should be noted that the memory systems and methods described herein are intended to include, but are not limited to, these and any other suitable types of memory.

[00370] It should be understood that the memory described above is illustrative, but not restrictive. For example, memory in the embodiment of the present disclosure may also be static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), direct rambus RAM (DR RAM), etc. That is, memory in the embodiments of the present disclosure is intended to include, but not be limited to, these and any other suitable types of memory.

[00371] The embodiments of the present disclosure further provide a computer-readable storage medium for storing computer programs.

[00372] Optionally, computer-readable storage media may be applied to the first device in the embodiments of the present disclosure, and the computer program enables the computer to perform the corresponding operations implemented by the network device in the various methods of the embodiments of the present disclosure. For the sake of brevity, this will not be elaborated upon here.

[00373] The modality of the present disclosure provides Petition 870250089278, dated 10 / 01 / 2025, page 104 / 252 97 / 99 additionally a computer program product, including a computer program instruction.

[00374] Optionally, the computer program product may be applied to the first device in the embodiments of the present disclosure, and the instructions of the computer program enable the computer to perform the corresponding operations implemented by the first device in the various methods of the embodiments of the present disclosure. For the sake of brevity, this will not be elaborated upon here.

[00375] The present disclosure also provides a computer program.

[00376] Optionally, the computer program may be applied to the first device in the embodiments of the present disclosure. The computer program, when executed on the computer, causes the computer to perform the corresponding operations implemented by the first device in various methods of the embodiments of the disclosure. For the sake of brevity, this will not be elaborated upon here.

[00377] Those with ordinary skill in the art may perceive that the units and steps of algorithms of the various examples described in combination with the embodiments disclosed in this document can be implemented by electronic hardware or by a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A skilled technician may use different methods to implement the described functions for specific applications, but such implementation should not be considered beyond the scope of the present disclosure. Petition 870250089278, dated 10 / 01 / 2025, page 105 / 252 98 / 99

[00378] Those skilled in the art will clearly understand that, for the sake of convenience and simplicity of description, as to the specific working process of the system, device and unit described above, reference may be made to the corresponding process in the method embodiment described above, and it will not be detailed here.

[00379] In the various embodiments provided by the present disclosure, it should be understood that the systems, devices, and methods disclosed may be implemented in other ways. For example, the device embodiment described above is only schematic. For example, the division of units is only a logical function division, and there may be another method of division in the actual implementation, for example, several units or components may be combined or integrated into another system, or some features may be omitted or not implemented. On the other hand, the mutual coupling or direct coupling or communication connection illustrated or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be in electrical, mechanical, or other forms.

[00380] The unit described as part of a separation may or may not be physically separated, and the unit displayed as a unit may or may not be a physical unit, i.e., it may be located in a single location or may be distributed across multiple network units. Some or all of the units may be selected according to the actual needs to achieve the purpose of the embodiment.

[00381] Furthermore, the functional units in each embodiment of the present disclosure can be integrated into a processing unit, each functional unit can exist Petition 870250089278, dated 10 / 01 / 2025, page 106 / 252 99 / 99 physically alone, or two or more functional units can be integrated into one unit.

[00382] If the function is implemented in the form of a functional software unit and marketed or used as a standalone product, the functional software unit may be stored on a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure, in essence or the part contributing to the related technique or part of the technical solution, may be embodied in the form of a software product, which is stored on a storage medium and includes various instructions to enable a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the operations of the method according to each embodiment of the present disclosure.The storage media mentioned above includes: a U disk, a portable hard drive, read-only memory (ROM), random access memory (RAM), a disc or optical disc, and other media that can store program code.

[00383] The foregoing are only specific embodiments of the present disclosure and are not intended to limit the scope of protection of the present disclosure. Any alteration and replacement easily devised by those skilled in the art within the technical scope of the present disclosure is included in the scope of protection of the present disclosure. Therefore, the scope of protection of the embodiments of the present disclosure will be subject to the scope of protection of the claims. Petition 870250089278, dated 10 / 01 / 2025, page 107 / 252

Claims

1 / 7 CLAIMS 1. MODEL PERFORMANCE MONITORING METHOD characterized by comprising: determining, by a first device, a similarity comparison result between an input dataset and a training dataset of a first model, wherein the similarity comparison result is used to determine the performance of the first model.

2. METHOD, according to claim 1, characterized in that the first model is used for direct positioning / assisted positioning of a first terminal device, and the input data set comprises a measurement parameter associated with a position of the first terminal device.

3. METHOD, according to claim 2, characterized in that the measurement parameter is a measurement parameter between the first terminal device and one or more network devices, and / or in that the training dataset comprises a measurement parameter between a second terminal device and one or more network devices, and label information corresponding to each measurement parameter.

4. METHOD, according to claim 3, characterized by the measurement parameter comprising one or more of: a channel impulse response (CIR); a power delay spectrum (PDP); a time of arrival (ToA); a path phase; Petition 870250089278, 01 / 10 / 2025, p. 108 / 252 2 / 7 an uplink arrival time difference (UL TDoA); a downlink arrival time difference (DL TDoA); an uplink departure angle (UL AoD); a downlink departure angle (DL AoD); an uplink relative arrival time (UL RTOA); a downlink reference signal time difference (DL RSTD); a reference signal reception power (RSRP); a reference signal reception quality (RSRQ); an azimuth; and a zenith.

5. METHOD, according to claim 3, characterized in that, in the case of the first model being used for direct positioning of the terminal device, the label information corresponding to each training data point includes a position of the second terminal device relative to the network device.

6. METHOD, according to claim 3, characterized in that, in the case of the first model being used for assisted positioning of the terminal device, the label information corresponding to each training data comprises one or more of the following: an arrival time (ToA), a downlink arrival time difference (DL TDoA), a downlink departure angle (DL AoD), a signal time difference Petition 870250089278, dated 10 / 01 / 2025, p. 109 / 252 3 / 7 downlink reference (DL RSTD), uplink arrival time difference (UL TDoA), uplink departure angle (UL AoD), uplink relative arrival time (UL RTOA), reference signal reception power (RSRP), reference signal reception quality (RSRQ), line-of-sight / out-of-line-of-sight (LOS / NLOS) identification, and distance information between the second terminal device and the network device.

7. METHOD, according to any one of claims 1 to 6, characterized by further comprising: in response to the result of the similarity comparison indicating that the input dataset is similar to the training dataset, determining that the performance of the first model satisfies a condition; and in response to the result of the similarity comparison indicating that the input dataset is not similar to the training dataset, updating the first model.

8. METHOD, according to any one of claims 1 to 6, characterized by the input dataset further comprising a measurement parameter associated with a position of a positioning reference unit, and the method further comprising: in response to a verification result indicating success and to the similarity comparison result indicating that the input dataset is similar to the training dataset, determining that the performance of the first model satisfies a condition; and in response to the verification result indicating Petition 870250089278, dated 01 / 10 / 2025, p.110 / 252 4 / 7 a failure and / or the result of the similarity comparison indicating that the input dataset is not similar to the training dataset, update the first model, where the verification result is a result of verifying whether an estimated position of the positioning reference unit is successfully matched to an actual position of the positioning reference unit, and the estimated position is obtained by processing the measurement parameter associated with the position of the positioning reference unit by the first model.

9. METHOD, according to any one of claims 1 to 8, characterized in that the result of the similarity comparison comprises one or more of: a first comparison result between a statistical parameter of the input dataset and a statistical parameter of the training dataset; and a second comparison result between the input data in the input dataset and the training data in the training dataset.

10. METHOD, according to claim 9, characterized by the first comparison result comprising: a first result of determining correspondence between the statistical parameter of the input dataset and the statistical parameter of the training dataset; and / or a first degree of correspondence between the statistical parameter of the input dataset and the statistical parameter of the training dataset wherein the statistical parameter comprises one or more of the following parameters: variance, mean, standard deviation and data distribution.

11. METHOD, according to claim 9, characterized by the second comparison result comprising: a second result of determining correspondence between the input data in the input dataset and the training data in the training dataset; and / or a second degree of correspondence between the input data in the input dataset and the training data in the training dataset.

12. METHOD, according to claim 11, characterized in that in response to the number of input data in the input dataset that are identical to the training data in the training dataset being greater than or equal to a fourth threshold, the second result of the matching determination indicates matching; and in response to the number of input data in the input dataset that are identical to the training data in the training dataset being less than the fourth threshold, the second result of the matching determination indicates mismatch; or in response to a ratio between the number of input data in the input dataset that are identical to the training data in the training dataset and the total number of input data in the input dataset being greater than or equal to a fifth threshold, the second result of the matching determination indicates Petition 870250089278, dated 10 / 01 / 2025, p.112 / 252 6 / 7 matching; and in response to a ratio between the number of input data in the input dataset that are identical to the training data in the training dataset and the total number of input data in the input dataset being less than the fifth threshold, the second matching determination result indicates mismatch.

13. METHOD, according to any one of claims 1 to 12, characterized in that the first device is a network device, the first model comprises one or more submodels, one or more submodels being deployed at a plurality of transmission / reception points (TRPs), and the method further comprises: determining, by the first device, a similarity comparison result between an input dataset and a training dataset of each submodel; and in response to the similarity comparison results between input datasets and training datasets of the submodels deployed at a portion of the plurality of TRPs, indicating that the input datasets of the submodels are not similar to the training datasets of the submodels, updating the submodels deployed at a portion of the plurality of TRPs.

14. METHOD, according to any one of claims 1 to 12, wherein the first device is a first terminal device, and the method is characterized by further comprising at least one of the following: receiving, by the first device, second information transmitted by a network device, wherein the second information is used to configure the input dataset and / or the training dataset of the first model; transmitting, by the first device, the result of the similarity comparison between the input dataset and the training dataset of the first model to a network device; or receiving, by the first device, third information transmitted by a network device, wherein the third information is used to indicate the update of the first model.

15. DEVICE FOR MONITORING A MODEL characterized by being configured to implement the method as defined in any one of claims 1 to 14. Petition 870250089278, dated 10 / 01 / 2025, pp. 114 / 252

Images