Ai / ML model monitoring method and apparatus therefor
The proposed method for monitoring and managing AI/ML models in wireless communication systems addresses the limitations of existing state management by enabling detailed transitions beyond enable/disable states, enhancing performance through activation, update, and retraining based on quality indicators and environmental changes.
Patent Information
- Application Number
- PCT/KR2025/020545
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-06-09
- Filing Date
- 2025-12-03
- Publication Date
- 2026-11-19
AI Technical Summary
Existing wireless communication systems lack effective methods for monitoring and managing the performance and state of AI/ML models, particularly in managing transitions between enabled and disabled states, which limits their effectiveness in applications like AI-based services and self-driving cars.
A method and apparatus for a user device (UE) in a wireless communication system to monitor and manage the state of AI/ML models by receiving configuration information, transmitting monitoring information, and managing the model's state based on indicators, including operations such as activation, update, retraining, or fallback, using monitoring execution information.
Enhances the performance of AI/ML models by detailed state management, allowing for improved transitions beyond simple enable/disable states, optimizing model operations based on quality indicators and environmental changes.
Smart Images

Figure KR2025020545_19112026_PF_FP_ABST
Abstract
Description
AI / ML model monitoring method and device for the same
[0001] The following description relates to Artificial Intelligence (AI) and Machine Learning (ML) utilized in wireless communication systems, specifically to a method for monitoring the performance of AI / ML models and managing the model's state accordingly, as well as a device for this purpose.
[0002] Various technologies such as LTE, LTE-Advanced, and WiFi are used in wireless communication systems, and 5G is also included here.
[0003] Figure 1 shows the structure of a system for 5G communication.
[0004] Referring to FIG. 1, the NG-RAN (Next Generation - Radio Access Network) may include a base station (20) that provides user plane and control plane protocol termination to the UE (10). For example, the base station (20) may include a gNB (next generation-Node B) and / or an eNB (evolved-Node B). For example, the UE (10) may be fixed or mobile and may be referred to by other terms such as terminal, MS (Mobile Station), UT (User Terminal), SS (Subscriber Station), MT (Mobile Terminal), or Wireless Device. For example, the base station may be a fixed station communicating with the UE (10) and may be referred to by other terms such as BTS (Base Transceiver System) or Access Point.
[0005] The example in FIG. 1 illustrates a case including only gNB. Base stations (20) can be connected to each other via Xn interfaces. Base stations (20) can be connected to a 5th generation core network (5G Core Network: 5GC) via NG interfaces. More specifically, base stations (20) can be connected to an access and mobility management function (AMF) (30) via an NG-C interface and to a user plane function (UPF) (30) via an NG-U interface.
[0006]
[0007] Meanwhile, new service types such as XR (Extended reality), AI-based services, and self-driving cars are emerging in 5G. Among these services, AI-based services have been actively discussed since Release 19 of the 3GPP Work Item (WI) called AI / ML in terms of beam management, positioning improvement, and performance improvement related to CSI (Channel Status Information).
[0008] However, regarding AI / ML-based operations, the concept of monitoring model performance and managing the model's state accordingly is still in the early stages of discussion, and remains limited to the simple concept of enabling or disabling the model.
[0009] In order to solve the problem described above, one aspect of the present invention proposes a method and apparatus for a user device (UE) in a wireless communication system to monitor an AI / ML model and specifically manage the state of the AI / ML model under the management of a network.
[0010] Specifically, we propose a method for operating AI / ML models by not only distinguishing between enabled and disabled states based on monitoring information, but also defining the state in detail for further improvement.
[0011] Meanwhile, in another aspect of the present invention, a method and apparatus for managing the state of an AI / ML model of a UE in a wireless communication system are proposed.
[0012] Specifically, the network proposes a method to specifically manage the state of the UE's AI / ML model by utilizing monitoring execution information transmitted by the UE.
[0013] In this specification, the term 'AI / ML' is assumed to have the meaning of 'either AI or ML' as a term corresponding to the name of a Work Item discussed in 3GPP.
[0014] The problems to be solved by the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which the present invention belongs from the description below.
[0015] In one aspect of the present invention for solving the problem described above, a method for monitoring an AI (Artificial Intelligence) / ML (Machine Learning) model by a user device (UE) in a wireless communication system is proposed, comprising: receiving configuration information related to the monitoring of the AI / ML model from a network; transmitting information for performing the monitoring of the AI / ML model to a network according to the configuration information; receiving an indicator related to the state of the AI / ML model from the network; and managing the state of the AI / ML model based on the indicator related to the state, wherein managing the state of the AI / ML model includes performing one or more operations among activation, update, retraining, or fallback of the AI / ML model.
[0016] At this time, updating the AI / ML model includes training the AI / ML model with additional training data on the previously trained data of the AI / ML model, retraining the AI / ML model includes excluding training based on the previously trained data of the AI / ML model and training again by providing training data, and the fallback of the AI / ML model may include performing an operation without using the AI / ML model.
[0017] The above status-related indicator may be configured to indicate the activation, update, retraining, or fallback of the AI / ML model according to the quality level of the monitoring execution information.
[0018] The above UE can perform positioning based on the above AI / ML model, and in this case, additionally includes transmitting information about data for training the above AI / ML model to the network, wherein the data may include one or more of Part A information including channel measurement information or Part B information including label information.
[0019] At this time, the above state-related indicator may be determined differently depending on whether the data includes the Part B information.
[0020] The above monitoring execution information may include positioning results based on the AI / ML model, wherein the state-related indicator preferably includes auxiliary information that enables the UE to determine one or more actions among activation, updating, retraining, or fallback of the AI / ML model based on the positioning results.
[0021] The above monitoring execution information may be information that supports the Location Management Function (LMF) of the network in producing a positioning result, and in this case, it is preferable that the state-related indicator includes explicit instruction information that directs one or more actions among activation, update, retraining, or fallback of the AI / ML model in the UE.
[0022] Meanwhile, in another aspect of the present invention for solving the problem described above, a method for managing the state of an AI (Artificial Intelligence) / ML (Machine Learning) model of a user device (UE) in a wireless communication system is proposed, comprising: transmitting configuration information related to monitoring of the AI / ML model to the UE; receiving information on performing monitoring of the AI / ML model from the UE; and transmitting an indicator related to the state of the AI / ML model to the UE based on the information on performing monitoring, wherein the indicator related to the state induces one or more operations among activation, update, retraining, or fallback of the AI / ML model.
[0023] The above AI / ML model can perform positioning, and in this case, the Location Management Function (LMF) of the network additionally receives information about data for training the AI / ML model from the UE, wherein the data may include one or more of Part A information including channel measurement information and a first quality indicator for the channel measurement information, or Part B information including label information and a second quality indicator for the label information.
[0024] If the above data includes the above Part B information, the state-related indicator may induce one or more actions among updating, retraining, or fallback of the AI / ML model according to the value of the second quality indicator.
[0025] Meanwhile, if the above data contains only the Part A information, the state-related indicator may induce one or more actions, such as retraining or fallback of the AI / ML model, depending on the value of the first quality indicator.
[0026] The above monitoring execution information can be continuously received from the UE, and the status-related indicator can be determined based on changes in the monitoring execution information continuously received from the UE.
[0027] The Location Management Function (LMF) of the above network can produce a positioning result based on the monitoring execution information, and the LMF can determine the state-related indicator based on the monitoring execution information and additional information.
[0028] Meanwhile, in another aspect of the present invention, a user device (UE) for monitoring an AI (Artificial Intelligence) / ML (Machine Learning) model in a wireless communication system comprises: at least one processor; and at least one computer memory that can be operably connected to the at least one processor and stores instructions that cause the at least one processor to perform operations when executed, wherein the operations include receiving configuration information related to monitoring the AI / ML model from a network; transmitting information for performing monitoring of the AI / ML model to a network according to the configuration information; receiving an indicator related to the state of the AI / ML model from the network; and managing the state of the AI / ML model based on the indicator related to the state, wherein managing the state of the AI / ML model includes performing one or more operations among activation, update, retrain, or fallback of the AI / ML model.
[0029] In addition, in another aspect of the present invention, a network device for managing the state of an Artificial Intelligence (AI) / Machine Learning (ML) model of a User Device (UE) in a wireless communication system is proposed, comprising: at least one processor; and at least one computer memory that can be operably connected to the at least one processor and stores instructions that cause the at least one processor to perform operations when executed, wherein the operations include transmitting configuration information related to monitoring of the AI / ML model to the UE; receiving information for performing monitoring of the AI / ML model from the UE; and transmitting an indicator related to the state of the AI / ML model to the UE based on the information for performing monitoring, wherein the indicator related to the state induces one or more operations among activation, update, retraining, or fallback of the AI / ML model.
[0030] According to the embodiments of the present invention as described above, the UE can monitor an AI / ML model and specifically manage the status of the AI / ML model under the management of the network.
[0031] Specifically, by not only distinguishing between enabled and disabled states based on monitoring information but also defining the state for further improvement in detail, performance can be improved in a simpler way depending on the state of the AI / ML model.
[0032] In addition, according to the embodiment, the network can efficiently manage the state of the UE's AI / ML model from an overall network perspective by utilizing additional information as well as monitoring execution information transmitted by the UE.
[0033] The effects obtainable from the present invention may not be limited to those described above. Furthermore, other unmentioned effects can be clearly understood by those skilled in the art from the following description.
[0034] Figure 1 shows the structure of a system for 5G communication.
[0035] Figure 2 is a diagram illustrating the concept of LCM for 3GPP AI / ML models.
[0036] FIG. 3 is a diagram illustrating a method for a user device (UE) to perform monitoring of an AI / ML model according to an embodiment of the present invention.
[0037] FIG. 4 is a diagram illustrating the state management concept of an AI / ML model according to one embodiment of the present invention.
[0038] FIG. 5 is a diagram illustrating a concept in which a data sample, positioned using an AI / ML model of a UE according to an embodiment of the present invention, is provided to a network so that the network can utilize it for AI / ML model state management.
[0039] FIG. 6 is a diagram illustrating a case in which monitoring results of an AI / ML model are derived from the LMF side according to an embodiment of the present invention.
[0040] FIG. 7 illustrates a wireless device that can be applied to the present technology.
[0041] Hereinafter, embodiments of the present invention are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.
[0042] Throughout the specification, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.
[0043]
[0044] As described above, in one aspect of the present invention, a method and apparatus for monitoring the performance of an AI / ML model and managing the state of the model accordingly are proposed. To this end, the concept of LCM (Lyfe Cycle Management) for managing the performance of an AI / ML model is first explained.
[0045]
[0046] AI / ML LCM
[0047] Figure 2 is a diagram illustrating the concept of LCM for 3GPP AI / ML models.
[0048] Figure 2 illustrates the LCM situation of an AI / ML model specifically for positioning, but this can be similarly applied to the utilization of AI / ML models other than positioning. The LCM of an AI / ML model involves a process of managing whether the model operates normally throughout the entire process, from deployment to utilization.
[0049] First, the LCM (210) can be configured to account for the complexity of model management. It can function to adapt AI / ML models to various deployment scenarios and can be configured to account for the complexity associated with equipment from various manufacturers. Additionally, the LCM (210) can manage the training (230), validation (240), development (250), deployment (260), inference (270), monitoring, and management (280) of each model, and FIG. 2 illustrates an exemplary form that can provide monitoring configuration information to the relevant entity, particularly for model monitoring and management (280).
[0050] Meanwhile, data management (220) refers to the role of obtaining accurate data for training (230) of AI / ML models. Data for positioning is discussed as follows, categorized as follows.
[0051] <Part A Information>
[0052] Channel measurement information,
[0053] A first quality indicator for the above channel measurement information,
[0054] First time stamp information for the above channel measurement
[0055] Part B Information
[0056] Ground truth-based labels (or estimates thereof)
[0057] Second quality indicator for the above label,
[0058] Second timestamp information for the above label
[0059] Model training (240) can be performed based on this data. Model training (240) may consider combining and using training data from various sources to improve the robustness of the model. Additionally, if label data is limited, semi-supervised learning techniques may be utilized.
[0060] Meanwhile, model verification (240) can be seen as a function that tests the accuracy of the model using a separate dataset. Additionally, it can evaluate the efficiency of model switching during the batch process.
[0061] Model deployment (250) can be seen as deploying the verified model for actual use through model verification (240). To this end, the models can be optimized to be time-efficient in inference and suitable for the device.
[0062] Through this process, real-time environmental data can be input into the model during the model inference (270) process to derive a positioning estimate value. In this process, the performance of the model can be managed to meet the predetermined accuracy requirements.
[0063] Meanwhile, the model monitoring and management (280) procedure can continuously manage the performance of the model. Through this function, switching and updating of the model can also be managed if necessary. In addition, it is desirable to implement a protocol for retraining in response to severe environmental changes.
[0064] As illustrated in FIG. 2, the entity managing the LCM (210) transmits configuration information for monitoring to the entity performing monitoring and management (280) of the AI / ML model, and based on this, receives a report on whether the KPI (Key Performance Index) is satisfied.
[0065]
[0066] FIG. 3 is a diagram illustrating a method for a user device (UE) to perform monitoring of an AI / ML model according to an embodiment of the present invention.
[0067] First, the UE can receive configuration information related to the monitoring of an AI / ML model from the network (S310) as described above in relation to FIG. 2. Based on this configuration information, the UE can transmit information on the monitoring of the AI / ML model to the network (S320).
[0068] In embodiments of the present invention, it is assumed that the UE is not allowed to voluntarily switch from an AI / ML method to a non-AI / ML method even if it is determined that the model does not satisfy the KPI as a result of monitoring the AI / ML model. That is, it is assumed that a network receiving the UE's monitoring performance information determines and transmits a status-related indicator of the UE's AI / ML model in consideration of this (S330), and the UE manages the state of the AI / ML model (S340) based on this status-related indicator. This is because, as described above in relation to FIG. 2, from the perspective of a network managing the deployment of AI / ML models, a method of managing the state of the AI / ML model by considering the monitoring performance information of each UE and additional network conditions may be advantageous.
[0069] In addition, in one embodiment of the present invention, managing the state of such an AI / ML model (S340) is proposed not simply by selecting whether to activate or deactivate the AI / ML model as in the conventional method, but by inducing the model's performance to be maintained through methods such as updating or retraining even when deactivated.
[0070] FIG. 4 is a diagram illustrating the state management concept of an AI / ML model according to one embodiment of the present invention.
[0071] As illustrated in FIG. 4, in one embodiment of the present invention, the state of an AI / ML model may be divided into an active state (410) and an inactive state (440), and the operation defined in the inactive state (440) may include one or more operations of updating (420), retraining (430), or fall-back (430).
[0072] In the diagram conceptualized in FIG. 4, the update (420) of the AI / ML model is assumed to be training the AI / ML model with additional training data on the previously trained data, and the retraining (430) of the AI / ML model is assumed to exclude training based on the previously trained data of the AI / ML model and to train it again by providing training data. Meanwhile, the fallback (430) of the AI / ML model is assumed to be operating without using the AI / ML model.
[0073] That is, based on the monitoring performance information of the UE's AI / ML model and / or the quality level determined by the network, the model's state indicator may direct one of activation, update, retraining, or fallback in order of highest quality.
[0074]
[0075] Below, the monitoring / state management method of the proposed model as described above is explained using the case of positioning based on an AI / ML model as an example.
[0076]
[0077] AI / ML-based positioning
[0078] To improve positioning performance based on AI / ML, 3GPP is discussing the following structure assuming the following.
[0079] Direct AI / ML Positioning
[0080] Case 1: UE-based positioning using a UE-side model
[0081] Case 2b: LMF-based positioning using an LMF-side model with UE support
[0082] Case 3b: Positioning using an LMF-side model with NG-RAN node support
[0083] <AI / ML 지원 기반 포지셔닝>
[0084] Case 2a: LMF-based positioning using a UE-side model with UE support
[0085] Case 3a: NG-RAN node-support-based positioning using a gNB-side model
[0086]
[0087] Below, focusing on Case 1 or Case 2a which utilizes the UE-side model among the aforementioned cases, we will explain how to monitor the UE's model and how the network manages the model's state accordingly.
[0088] When considering Case 1, the entity producing the monitoring results may be the target UE or the LMF. In this regard, it will be described in detail by distinguishing based on the entity producing the monitoring results.
[0089]
[0090] Target UE side
[0091] When the monitoring result is produced at the target UE, the monitoring report (S320) that the UE transmits to the network (LMF) in FIG. 3 may be the monitoring result, and the UE may receive instructions for the target UE from the network (LMF).
[0092] In this case, it can be viewed as a situation where the UE is aware of the monitoring results, and it is possible to implicitly inform the UE of the action to be taken based on them.
[0093]
[0094] However, even in this case, the LMF may explicitly inform the UE of the action to be taken as follows.
[0095] As described above, we propose that the retraining mode, update mode, fallback pod, etc., be determined based on the monitoring results of the AI / ML model, rather than simply deciding whether to disable or enable it. In particular, when the distribution of the actual input data differs from that of the training data, retraining may be more advantageous than continued training, and accordingly, selecting the update mode may be beneficial.
[0096]
[0097] Meanwhile, in one embodiment of the present invention, the instruction information of the network (LMF) may be instruction information to set based on a monitoring result value (e.g., accuracy, RMSE, etc.) transmitted by the target UE to the LMF. For example, as a 1-bit indicator, it may instruct to operate according to a result value known to the UE through 0 or 1, or to maintain the current operation regardless of this. This can be seen as a method possible because the UE knows the monitoring result.
[0098] Depending on the performance of the model, it is also possible to set the order as Activate > Deactivate (Update) > Deactivate (Retrain) > Deactivate (Fallback). However, the four states of Activate, Deactivate (Update), Deactivate (Retrain), and Deactivate (Fallback) are exemplary, and only some of these states may be included.
[0099] Therefore, based on the monitoring result value transmitted by the target UE to the LMF, the activation > deactivation (update) > deactivation (retraining) > deactivation (fallback) state can be determined according to a specific value or the value set as the upper signaling.
[0100] If a directive is received indicating that the setting is not configured based on the monitoring results, the UE may transmit the result value to the LMF once again.
[0101] It goes without saying that the above-described embodiments can be operated in combination with one another.
[0102]
[0103] Meanwhile, as described above, instructions for the UE's operation can be set based on existing parameters to efficiently instruct the UE when it is aware of the monitoring results.
[0104] Key factors to consider when deciding whether to fall back, retrain, or update include the quality of training samples, environmental changes, and changes in data distribution. Since the target UE is the primary agent of the AI / ML model's learning, it can be assumed that it is fully aware of the quality of the existing training samples.
[0105] Accordingly, in one embodiment of the present invention, it is proposed to manage the state of an AI / ML model of a UE using a 1-bit (0 (operation based on the monitoring result transmitted by the UE) or 1 (maintenance)) method as the direct indication method described above and indirect parameters (e.g., Part A or Part B of the collected data sample, first / second quality indicators, etc.). The data sample may be a data sample generated on the network side. However, it may also be utilized in cases where the data sample is generated on the UE side, such as in case 1 / 2a.
[0106] FIG. 5 is a diagram illustrating a concept in which a data sample, positioned using an AI / ML model of a UE according to an embodiment of the present invention, is provided to a network so that the network can utilize it for AI / ML model state management.
[0107] Specifically, Figure 5 corresponds to Case 2a among the cases for AI / ML method positioning described above, but it can also be seen as a case where data samples are provided to the network for the LCM operation of the network (LMF) in the case of Case 1.
[0108] Case 2a is a case of LMF-based positioning using a UE-side model under UE support, as described above.
[0109] Accordingly, the AI / ML model exists on the UE side, and Part A and Part B information for training (updating) the AI / ML model is also generated on the UE. Specifically, the UE receives RS, specifically PRS, from the base station (gNB) (S510), and can generate Part A information and Part B information based on label information using location information based on channel measurement (S520).
[0110] These Part A information and Part B information may be used for training / updating the UE-side AI / ML model (S530), and in one embodiment of the present invention, the UE may transmit one or more of the Part A or Part B information to the network (S540) for managing the state of the AI / ML model of the network (LMF).
[0111] Meanwhile, Case 1 is a case of UE-based positioning using a UE-side model, and in this case, it is possible for the UE to transmit one or more of Part A or Part B information to the network for AI / ML model state management of the network (S540).
[0112]
[0113] In one embodiment of the present invention, it is proposed that the state of an AI / ML model be determined differently depending on whether a data sample received by a network (LMF) includes Part B information.
[0114] Specifically, if the data sample received by the network (LMF) includes Part B, it can be considered a case where there is room for performance improvement of the AI / ML model even if the UE's AI / ML model is disabled, because the label is included. Therefore, in one embodiment of the present invention, it is proposed to induce performance improvement of the AI / ML model through updating or retraining the AI / ML model even if it is disabled in such cases.
[0115] To this end, the AI / ML model status indicator (S550) transmitted by the network to the UE may include auxiliary information that determines one or more actions of the AI / ML model, such as activation, update, retraining, or fallback, based on the positioning result.
[0116] Specifically, if the second quality indicator in Part B is higher than a threshold set by a specific value or upper signaling, an update is performed, as high label confidence indicates a high potential for model improvement. Conversely, if the second quality indicator in Part B is lower than a threshold set by a specific value or upper signaling, retraining is performed. This can be interpreted as the existing training data being distrusted, making it superior to retrain while the model is inactive rather than improving the previous one. If there is no second quality indicator, a fallback can be performed without an update.
[0117]
[0118] On the other hand, if the data samples received by the network (LMF) contain only Part A, it can be seen as a case where there is little room for performance improvement when disabled. Therefore, it is suggested to retrain the AI / ML model or perform a fallback when disabled. That is, unlike the case where the data samples received by the network (LMF) contain Part B, it can be configured not to perform an update operation.
[0119] Specifically, if the first quality indicator of Part A is higher than a threshold set by a specific value or upper signaling, retraining is performed, which can be viewed as a new learning process. Additionally, if the first quality indicator of Part A is lower than a threshold set by a specific value or upper signaling, a fallback is performed, which means a fallback is performed without updating. Furthermore, a fallback may also be performed if there is no first quality indicator.
[0120]
[0121] Meanwhile, in another embodiment of the present invention, continuous AI / ML model monitoring results are received from a network, and instructions may be given to determine the state of the corresponding AI / ML model based on changes in the numerical values of the continuous results. If the change in the numerical value of the continuous monitoring results is smaller than a specific value or a threshold set as a higher signaling, instructions may be given to perform an update, and if it is larger, instructions may be given to perform retraining.
[0122] In addition, another embodiment of the present invention proposes changing the behavior of a UE according to a number of consecutively indicated deactivations. Specifically, depending on the number of deactivations indicated, the UE may operate to perform update > retraining > fallback on the corresponding AI / ML model. Thus, the number of deactivations indicated by the network may be determined by a specific value or a number set by a higher signaling. For example, an update may be performed when deactivation is indicated once, a retraining when indicated twice consecutively, and a fallback when indicated three times consecutively. However, it is understood that the operation may be configured to include only some of the three states: update, retraining, and fallback.
[0123]
[0124] LMF side
[0125] FIG. 6 is a diagram illustrating a case in which monitoring results of an AI / ML model are derived from the LMF side according to an embodiment of the present invention.
[0126] As illustrated in FIG. 6, the UE can transmit parameters of the AI / ML model or final location information estimated using the AI / ML model to the LMF (S610). The LMF according to the present embodiment can derive monitoring results of the AI / ML model based on the parameters or location information received in this way (S620). Accordingly, the LMF can transmit indicators related to the state of the AI / ML model to the UE (S630), and can manage the UE's AI / ML model as activated, deactivated (updated), deactivated (retrained), deactivated (fallback), etc. (S640) as described in relation to the embodiment of FIG. 3.
[0127] When the monitoring results of the AI / ML model are derived from the network (LMF) as described above, there are the following differences compared to the case where they are derived from the UE side.
[0128] When the network (LMF) derives the model's monitoring results, the target UE may be unaware of these results. Additionally, training samples may not be generated by the LMF. Consequently, the LMF may not know the quality of existing training samples. Instead, the network (LMF) may possess more information regarding changes in other environments or data distributions. Therefore, in this embodiment, it may be advantageous for the network (LMF) to directly indicate the state of the UE's AI / ML model.
[0129]
[0130] That is, in this embodiment as well as in the above-described embodiment, the state-related indicators transmitted by the network to the UE can indicate one or more actions among activation, update, retraining, or fallback of the AI / ML model.
[0131] In addition, in one embodiment of the present invention, the system may be operated to transmit mutually agreed-upon monitoring result values between the UE and the network. In particular, the behavior based on the monitoring result value may vary depending on the surrounding environment, training samples, etc. For example, if the quality of the training samples is excellent and there is high potential for performance improvement in the update method, the threshold value separating update and retraining may be lowered. Accordingly, based on the monitoring result value, the sequence of activation > deactivation (update) > deactivation (retraining) > deactivation (fallback) can be determined according to a specific value or a numerical value set as a higher signaling. However, it goes without saying that only some of these four stages may be included. Furthermore, the above-described embodiments may be performed in combination.
[0132] Meanwhile, even when the monitoring result of the AI / ML model is derived by the LMF, as described above in relation to Figure 5, training data samples may be provided to the network, and the network can determine the state of the AI / ML model differently depending on whether Part B information is included. However, when the monitoring result of the AI / ML model is derived by the LMF, the network can manage the state of each model by considering additional information such as network conditions and the deployment status of the AI / ML model among UEs, in addition to this training data.
[0133]
[0134] The content of the signal transmitted by the target UE to the LMF
[0135] Below, the contents of the signal transmitted by the target UE to the LMF are explained, focusing particularly on Case 1 regarding positioning.
[0136] As described above, the entity generating the monitoring results of the AI / ML model may be the target UE or the LMF. Accordingly, the information that the target UE must transmit to the LMF may vary depending on the entity generating the monitoring results.
[0137]
[0138] When LMF is calculated
[0139] When the LMF produces model monitoring results, as described above in relation to Fig. 6, the content of the signal transmitted by the target UE to the LMF may be parameters of the AI / ML model or estimated final location information.
[0140] Key factors to consider when deciding whether to fallback, retrain, or update include the quality of existing training samples, environmental changes, and changes in data distribution. Accordingly, the signals transmitted by the UE to the LMF may contain the following information.
[0141] - Estimated final location
[0142] - The overall quality value of the training samples used for training the AI / ML model on the current target UE (this may differ from the aforementioned 1st / 2nd quality indicators)
[0143] As discussed in the embodiments described above, the quality of the training samples of the target UE, for which a clear value exists, can be considered an important factor in determining the UE's behavior. It may not be necessary to provide the quality of each training sample for monitoring purposes. However, the calculation method can be one of various methods that can be obtained through common sense. For example, the average value of the overall training sample quality can be considered.
[0144] - Separately transmit the overall quality values of training sample Part A and training sample Part B used for AI / ML model training on the current target UE.
[0145] Since the quality of Part A and Part B is calculated based on different criteria, it may be more accurate to transmit them separately.
[0146] - Transmit environmental change values measurable on the target UE
[0147] - Transmit data distribution change values measurable on the target UE
[0148]
[0149] When the target UE calculates
[0150] The target UE can transmit the monitoring results derived directly to the LMF.
[0151] As discussed in the embodiments described above, the UE may transmit direct instructions desired by the UE to the network (LMF), or the LMF may transmit values of calculated monitoring metrics to make a comprehensive decision.
[0152] Key factors to consider when deciding whether to fallback, retrain, or update include the quality of existing training samples, changes in the environment, and changes in data distribution.
[0153] In this embodiment, the UE can transmit information to the network requesting the deactivation, activation, fallback, retraining, and updating of the corresponding AI / ML model.
[0154] When requesting retraining, the network (LMF) can be informed that the distribution of the existing training data and the actual input data has changed.
[0155] When requesting an update, the UE may indicate to the network (LMF) that there are still insufficient training data samples.
[0156]
[0157] In addition, in one embodiment of the present invention, the UE can transmit the following information to the network (LMF).
[0158] - Monitoring metrics (e.g., accuracy, RMSE, confidence score, etc.)
[0159] - Separately transmit the overall quality values of training sample Part A and training sample Part B used for AI / ML model training on the current target UE.
[0160] This is because it may be more accurate to transmit Part A and Part B separately, as their quality is calculated based on different standards. The information transmitted in this case can be distinguished from the first and second quality indicators.
[0161] - Environmental change values measurable on the target UE
[0162] - Data distribution change values measurable in the target UE
[0163]
[0164] Device configuration
[0165] FIG. 7 illustrates a wireless device that can be applied to the present technology.
[0166] Referring to FIG. 7, the first wireless device (100) and the second wireless device (200) can transmit and receive wireless signals through various wireless access technologies (e.g., LTE, NR). Here, the first wireless device (100) and the second wireless device (200) can correspond to the UE and network (LMF) of FIG. 3, respectively.
[0167] The first wireless device (100) includes one or more processors (102) and one or more memories (104), and may additionally include one or more transceivers (106) and / or one or more antennas (108). The processor (102) controls the memory (104) and / or transceivers (106) and may be configured to implement the descriptions, functions, procedures, proposals, methods and / or flowcharts of operation disclosed in this document. For example, the processor (102) may process information within the memory (104) to generate a first information / signal and then transmit a wireless signal containing the first information / signal through the transceiver (106). Additionally, the processor (102) may receive a wireless signal containing a second information / signal through the transceiver (106) and then store information obtained from the signal processing of the second information / signal in the memory (104). Memory (104) may be connected to the processor (102) and may store various information related to the operation of the processor (102). For example, memory (104) may store software code containing instructions for performing some or all of the processes controlled by the processor (102) or for performing the descriptions, functions, procedures, proposals, methods, and / or operation sequence diagrams disclosed in this document. Here, the processor (102) and memory (104) may be part of a communication modem / circuit / chip designed to implement wireless communication technology (e.g., LTE E-UTRA, 5G NR). A transceiver (106) may be connected to the processor (102) and may transmit and / or receive wireless signals through one or more antennas (108). The transceiver (106) may include a transmitter and / or receiver. The transceiver (106) may be combined with an RF (Radio Frequency) unit. In the present invention, the wireless device may refer to a communication modem / circuit / chip.
[0168] The second wireless device (200) includes one or more processors (202) and one or more memories (204), and may additionally include one or more transceivers (206) and / or one or more antennas (208). The processor (202) controls the memory (204) and / or transceivers (206) and may be configured to implement the descriptions, functions, procedures, proposals, methods and / or operation sequences disclosed in this document. For example, the processor (202) may process information within the memory (204) to generate a third information / signal and then transmit a wireless signal containing the third information / signal through the transceiver (206). Additionally, the processor (202) may receive a wireless signal containing a fourth information / signal through the transceiver (206) and then store information obtained from the signal processing of the fourth information / signal in the memory (204). The memory (204) may be connected to the processor (202) and may store various information related to the operation of the processor (202). For example, the memory (204) may store software code containing instructions for performing some or all of the processes controlled by the processor (202) or for performing the descriptions, functions, procedures, proposals, methods, and / or operation sequence diagrams disclosed in this document. Here, the processor (202) and the memory (204) may be part of a communication modem / circuit / chip designed to implement wireless communication technology (e.g., LTE E-UTRA, 5G NR). The transceiver (206) may be connected to the processor (202) and may transmit and / or receive wireless signals through one or more antennas (208). The transceiver (206) may include a transmitter and / or receiver. The transceiver (206) may be used in combination with an RF unit. In the present invention, the wireless device may refer to a communication modem / circuit / chip.
[0169] Hereinafter, hardware elements of the wireless device (100, 200) will be described in more detail. Although not limited thereto, one or more protocol layers may be implemented by one or more processors (102, 202). For example, one or more processors (102, 202) may implement one or more layers (e.g., functional layers such as PHY, MAC, RLC, PDCP, RRC, SDAP). One or more processors (102, 202) may generate one or more Protocol Data Units (PDUs) and / or Service Data Units (SDUs) according to the descriptions, functions, procedures, proposals, methods, and / or flowcharts of operation disclosed in this document. One or more processors (102, 202) may generate messages, control information, data, or information according to the descriptions, functions, procedures, proposals, methods, and / or flowcharts of operation disclosed in this document. One or more processors (102, 202) may generate a signal (e.g., baseband signal) containing a PDU, SDU, message, control information, data, or information according to the functions, procedures, proposals, and / or methods disclosed in this document and provide it to one or more transceivers (106, 206). One or more processors (102, 202) may receive a signal (e.g., baseband signal) from one or more transceivers (106, 206) and may obtain a PDU, SDU, message, control information, data, or information according to the descriptions, functions, procedures, proposals, methods, and / or flowcharts disclosed in this document.
[0170] One or more processors (102, 202) may be referred to as a controller, microcontroller, microprocessor, or microcomputer. One or more processors (102, 202) may be implemented by hardware, firmware, software, or a combination thereof. For example, one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), one or more Digital Signal Processing Devices (DSPDs), one or more Programmable Logic Devices (PLDs), or one or more Field Programmable Gate Arrays (FPGAs) may be included in one or more processors (102, 202). The descriptions, functions, procedures, proposals, methods, and / or flowcharts disclosed in this document may be implemented using firmware or software, and the firmware or software may be implemented to include modules, procedures, functions, etc. Firmware or software configured to perform the descriptions, functions, procedures, proposals, methods, and / or operation sequences disclosed in this document may be contained in one or more processors (102, 202) or stored in one or more memories (104, 204) and driven by one or more processors (102, 202). The descriptions, functions, procedures, proposals, methods, and / or operation sequences disclosed in this document may be implemented using firmware or software in the form of code, instructions, and / or sets of instructions.
[0171] One or more memories (104, 204) may be connected to one or more processors (102, 202) and may store various forms of data, signals, messages, information, programs, code, instructions, and / or commands. One or more memories (104, 204) may be composed of ROM, RAM, EPROM, flash memory, hard drive, registers, cache memory, computer read storage media, and / or combinations thereof. One or more memories (104, 204) may be located inside and / or outside of one or more processors (102, 202). Additionally, one or more memories (104, 204) may be connected to one or more processors (102, 202) through various technologies such as wired or wireless connections.
[0172] One or more transceivers (106, 206) may transmit user data, control information, wireless signals / channels, etc., as mentioned in the methods and / or operation flowcharts, etc., of this document to one or more other devices. One or more transceivers (106, 206) may receive user data, control information, wireless signals / channels, etc., as mentioned in the descriptions, functions, procedures, proposals, methods and / or operation flowcharts, etc., disclosed in this document from one or more other devices. For example, one or more transceivers (106, 206) may be connected to one or more processors (102, 202) and may transmit and receive wireless signals. For example, one or more processors (102, 202) may control one or more transceivers (106, 206) to transmit user data, control information, or wireless signals to one or more other devices. Additionally, one or more processors (102, 202) may control one or more transceivers (106, 206) to receive user data, control information, or wireless signals from one or more other devices. Additionally, one or more transceivers (106, 206) may be connected to one or more antennas (108, 208), and one or more transceivers (106, 206) may be configured to transmit and receive user data, control information, wireless signals / channels, etc., as described in the descriptions, functions, procedures, proposals, methods, and / or flowcharts of operation disclosed in this document through one or more antennas (108, 208). In this document, one or more antennas may be multiple physical antennas or multiple logical antennas (e.g., antenna ports). One or more transceivers (106, 206) can convert the received wireless signal / channel, etc. from an RF band signal to a baseband signal in order to process the received user data, control information, wireless signal / channel, etc. using one or more processors (102, 202).One or more transceivers (106, 206) can convert user data, control information, wireless signals / channels, etc. processed using one or more processors (102, 202) from baseband signals to RF band signals. To this end, one or more transceivers (106, 206) may include (analog) oscillators and / or filters.
[0173]
[0174] The detailed description of the preferred embodiments of the present invention disclosed above is provided to enable those skilled in the art to implement and practice the present invention. Although the present invention has been described with reference to preferred embodiments, those skilled in the art will understand that various modifications and changes can be made to the present invention without departing from the scope of the invention. For example, those skilled in the art may utilize each configuration described in the above embodiments in a manner that combines with one another.
[0175] Accordingly, the present invention is not intended to be limited to the embodiments shown herein, but to be given the broadest scope consistent with the principles and novel features disclosed herein.
[0176] The AI / ML model monitoring and management method based thereon and the apparatus for the same according to the embodiments of the present invention as described above are suitable for use in a communication environment where AI / ML techniques are supported by 3GPP, but can also be used in the same way in 3GPP's 6G and other heterogeneous next-generation mobile communication systems.
Claims
1. A method for a user device (UE) in a wireless communication system to perform monitoring of an AI (Artificial Intelligence) / ML (Machine Learning) model, Receive configuration information related to the monitoring of the above AI / ML model from the network; Transmits monitoring execution information of the AI / ML model to the network according to the above configuration information; Receive state-related indicators of the AI / ML model from the above network; and Includes managing the state of the AI / ML model based on the above state-related indicators, Managing the state of the above AI / ML model is, including performing one or more operations of activation, update, retrain, or fallback of the above AI / ML model, UE model monitoring method.
2. In Paragraph 1, The update of the above AI / ML model includes training the above AI / ML model on its prior training data using additional training data, and The retraining of the above AI / ML model includes excluding training based on previously learned data of the above AI / ML model and training again by providing new training data. The fallback of the above AI / ML model includes performing an operation without using the above AI / ML model. UE model monitoring method.
3. In Paragraph 1, The above status-related indicators are, Configured to instruct the activation, update, retraining, or fallback of the above AI / ML model according to the quality level of the above monitoring execution information, UE model monitoring method.
4. In Paragraph 1, The above UE performs positioning based on the above AI / ML model, and It additionally includes transmitting information about data for training the AI / ML model to the above network, The above data includes one or more of Part A information including channel measurement information or Part B information including label information, UE model monitoring method.
5. In Paragraph 4, The above state-related indicator is determined differently depending on whether the data includes the Part B information, UE model monitoring method.
6. In Paragraph 4, The above monitoring execution information includes positioning results based on the above AI / ML model, and The above state-related indicator includes auxiliary information that enables the UE to determine one or more actions of activation, update, retraining, or fallback of the AI / ML model based on the positioning result. UE model monitoring method.
7. In Paragraph 4, The above monitoring execution information is information that supports the LMF (Location Management Function) of the above network in producing positioning results, and The above state-related indicator includes explicit instruction information that directs one or more actions of the AI / ML model, such as activation, update, retraining, or fallback, in the UE. UE model monitoring method.
8. A method in which a network in a wireless communication system manages the state of an AI (Artificial Intelligence) / ML (Machine Learning) model of a user device (UE), Transmitting configuration information related to the monitoring of the AI / ML model to the above UE; Receives monitoring execution information of the AI / ML model from the above UE; and Based on the above monitoring execution information, including transmitting state-related indicators of the AI / ML model to the UE, The above status-related indicators are, Including inducing one or more operations among activation, update, retrain, or fallback of the above AI / ML model, Model state management method.
9. In Paragraph 8, The above AI / ML model performs positioning, and The LMF (Location Management Function) of the above network additionally includes receiving information about data for training the AI / ML model from the UE, The above data comprises one or more of Part A information including channel measurement information and a first quality indicator for the channel measurement information, or Part B information including label information and a second quality indicator for the label information. Model state management method.
10. In Paragraph 9, If the above data includes the above Part B information, the state-related indicator induces one or more actions among update, retrain, or fallback of the AI / ML model according to the value of the second quality indicator, Model state management method.
11. In Paragraph 9, If the above data includes only the Part A information, the state-related indicator induces one or more actions, such as retraining or fallback of the AI / ML model, depending on the value of the first quality indicator, Model state management method.
12. In Paragraph 8, The above monitoring execution information is continuously received from the UE, and The above-mentioned status-related indicator is determined based on changes in the monitoring execution information continuously received from the UE, Model state management method.
13. In Paragraph 8, The LMF (Location Management Function) of the above network calculates positioning results based on the monitoring execution information, and The above LMF determines the state-related indicators based on the above monitoring execution information and additional information, Model state management method.
14. In a user device (UE) that performs monitoring of an AI (Artificial Intelligence) / ML (Machine Learning) model in a wireless communication system, At least one processor; and It includes at least one computer memory that can be operably connected to the at least one processor and stores instructions that cause the at least one processor to perform operations when executed, and The above operations are, Receive configuration information related to the monitoring of the above AI / ML model from the network; Transmits monitoring execution information of the AI / ML model to the network according to the above configuration information; Receive state-related indicators of the AI / ML model from the above network; and Includes managing the state of the AI / ML model based on the above state-related indicators, Managing the state of the above AI / ML model is, including performing one or more operations of activation, update, retrain, or fallback of the above AI / ML model, User device.
15. A network device for managing the state of an AI (Artificial Intelligence) / ML (Machine Learning) model of a user device (UE) in a wireless communication system, At least one processor; and It includes at least one computer memory that can be operably connected to the at least one processor and stores instructions that cause the at least one processor to perform operations when executed, and The above operations are, Transmitting configuration information related to the monitoring of the AI / ML model to the above UE; Receives monitoring execution information of the AI / ML model from the above UE; and Based on the above monitoring execution information, including transmitting state-related indicators of the AI / ML model to the UE, The above status-related indicators are, Including inducing one or more operations among activation, update, retrain, or fallback of the above AI / ML model, Network device.
