Model monitoring method and apparatus, communication device, and storage medium

Abstract

The application relates to a model monitoring method and device, a communication device and a storage medium, and relates to the technical field of wireless communication. The method comprises the following steps: receiving a first message sent by a first network element, wherein the first message comprises model monitoring related configuration information; the model monitoring related configuration information comprises model monitoring process configuration information, model monitoring reference signal resource configuration information and / or report configuration information; receiving a second message sent by the first network element, wherein the second message carries a reference signal, and the reference signal is used for resource measurement to obtain label data used for model monitoring; and sending a third message to the first network element, wherein the third message carries the label data and an inference / prediction result, or the third message carries a model monitoring result. The method can define and design a model monitoring process of a wireless intelligent air interface.

Description

Technical Field

[0001] This application relates to the field of wireless communication technology, and in particular to a model monitoring method, apparatus, communication device, storage medium, and computer program product. Background Technology

[0002] With the continuous evolution of wireless communication technology, intelligent wireless interface refers to the technology of optimizing the air interface of wireless communication systems using artificial intelligence (AI) and machine learning (ML) models. Model monitoring, as one of the key technologies supporting the intelligence, adaptability, and self-optimization of future networks, faces unprecedented market opportunities. In the future network environment, model monitoring will not only support more efficient network resource management and optimization but also provide a solid foundation for intelligent applications in various vertical industries.

[0003] However, there is a lack of unified definition and design for the model monitoring process of wireless intelligent air interface in related technologies. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, communication device, storage medium, and computer program product that can define and design the model monitoring process to address the above-mentioned technical problems.

[0005] Firstly, this application provides a model monitoring method. The method includes:

[0006] Receive a first message sent by a first network element, the first message containing configuration information related to model monitoring; the configuration information related to model monitoring includes model monitoring process configuration information, model monitoring reference signal resource configuration information and / or reporting configuration information;

[0007] The system receives a second message sent by the first network element, the second message carrying a reference signal, the reference signal being used to perform resource measurement to obtain tag data for model monitoring;

[0008] A third message is sent to the first network element, the third message carrying the tag data and inference / prediction results, or the third message carrying model monitoring results.

[0009] In one embodiment, the model monitoring process configuration information is used to indicate the model monitoring signaling process and the model monitoring triggering method.

[0010] In one embodiment, the reporting configuration information is used to indicate the model monitoring process, the reporting signaling process, the reporting content, and / or the reporting timing.

[0011] In one embodiment, the model monitoring process includes an inference process and / or a monitoring process:

[0012] The inference process is used to instruct the measurement model input and make predictions, and report the inference / prediction results;

[0013] The monitoring process is used to instruct the measurement model to monitor resources, obtain monitoring results, and report the monitoring results.

[0014] In one embodiment, the method further includes:

[0015] When the measurement resource set / measurement resources are a subset of the model monitoring resource set, the reporting signaling process is the model monitoring reporting signaling process.

[0016] In one embodiment, the method further includes:

[0017] When the measurement resource set / measurement resources are not entirely a subset of the model monitoring resource set, the reporting signaling process includes a single reporting process and / or an independent reporting process.

[0018] The single reporting process is used to instruct the reporting of inference / prediction results of the measurement resource set / measurement resources and monitoring results of the model monitoring resource set through a single reporting process;

[0019] The independent reporting process is used to instruct the reporting of inference / prediction results of the measurement resource set / measurement resources and monitoring results of the model monitoring resource set through different reporting processes.

[0020] In one embodiment, the reported content includes one of the following:

[0021] Reporting indicators;

[0022] Report the incident;

[0023] Report both events and metrics based on events.

[0024] In one embodiment, the configuration of the reporting timing includes at least one of periodic, semi-static, non-periodic, and event-triggered modes.

[0025] In one embodiment, the reported configuration information further includes indication information, which is used to indicate that the reported configuration information allows the UE to decide / select.

[0026] In one embodiment, the model monitoring reference signal resource configuration information is used to indicate the location of reference signal transmission time-frequency resources and the type of reference signal.

[0027] In one embodiment, the reference signal transmission time-frequency resource location includes: information on the relationship between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set.

[0028] In one embodiment, the relationship information between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set includes one of the following:

[0029] The time-frequency resources of the reference signal occupy part or all of the frequency domain resources + all time domain resources of the beam downlink transmission of the predicted resource set;

[0030] The reference signal's time-frequency resource occupancy is part or all of the time-domain resources + all frequency-domain resources of the beam downlink transmission of the predicted resource set;

[0031] The reference signal's time-frequency resource occupancy is a portion of the frequency domain resources and a portion of the time domain resources of the beam downlink transmission of the predicted resource set;

[0032] The temporal resource occupancy of the reference signal in the predicted resource set is either continuous or discontinuous.

[0033] The frequency domain resources of the reference signal are continuous or discontinuous resource blocks RB or resource block groups RBG in the prediction resource set.

[0034] In one embodiment, the model monitoring reference signal resource configuration information is also used to instruct the model to monitor time and frequency resources.

[0035] In one embodiment, the first network element includes at least one of gNB, UE, LMF, and OAM.

[0036] In one embodiment, the first message includes at least one of system message, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling.

[0037] In one embodiment, the second message includes at least one of system message, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling.

[0038] In one embodiment, the third message includes a CSI report message.

[0039] Secondly, this application provides a model monitoring method. The method includes:

[0040] Send a first message to the second network element. The first message contains configuration information related to model monitoring. The configuration information related to model monitoring includes model monitoring process configuration information, model monitoring reference signal resource configuration information, and / or reporting configuration information.

[0041] Send a second message to the second network element, the second message carrying a reference signal, the reference signal being used to perform resource measurement to obtain tag data for model monitoring;

[0042] The system receives a third message sent by the second network element, the third message carrying the tag data and inference / prediction results, or the third message carrying model monitoring results.

[0043] In one embodiment, the method further includes: calculating model monitoring results when the second network element sends tag data and inference / prediction results.

[0044] In one embodiment, the model monitoring process configuration information is used to indicate the model monitoring signaling process and the model monitoring triggering method.

[0045] In one embodiment, the reporting configuration information is used to indicate the model monitoring process, the reporting signaling process, the reporting content, and / or the reporting timing.

[0046] In one embodiment, the model monitoring process includes an inference process and / or a monitoring process:

[0047] The inference process is used to instruct the measurement model input and make predictions, and report the inference / prediction results;

[0048] The monitoring process is used to instruct the measurement model to monitor resources, obtain monitoring results, and report the monitoring results.

[0049] In one embodiment, the method further includes:

[0050] When the measurement resource set / measurement resources are a subset of the model monitoring resource set, the reporting signaling process is the model monitoring reporting signaling process.

[0051] In one embodiment, the method further includes:

[0052] When the measurement resource set / measurement resources are not entirely a subset of the model monitoring resource set, the reporting signaling process includes a single reporting process and / or an independent reporting process.

[0053] The single reporting process is used to instruct the reporting of inference / prediction results of the measurement resource set / measurement resources and monitoring results of the model monitoring resource set through a single reporting process;

[0054] The independent reporting process is used to instruct the reporting of inference / prediction results of the measurement resource set / measurement resources and monitoring results of the model monitoring resource set through different reporting processes.

[0055] In one embodiment, the reported content includes one of the following:

[0056] Reporting indicators;

[0057] Report the incident;

[0058] Report both events and metrics based on events.

[0059] In one embodiment, the configuration of the reporting timing includes at least one of periodic, semi-static, non-periodic, and event-triggered modes.

[0060] In one embodiment, the reported configuration information further includes indication information, which is used to indicate that the reported configuration information allows the UE to decide / select.

[0061] In one embodiment, the model monitoring reference signal resource configuration information is used to indicate the location of reference signal transmission time-frequency resources and the type of reference signal.

[0062] In one embodiment, the reference signal transmission time-frequency resource location includes: information on the relationship between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set.

[0063] In one embodiment, the relationship information between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set includes one of the following:

[0064] The time-frequency resources of the reference signal occupy part or all of the frequency domain resources + all time domain resources of the beam downlink transmission of the predicted resource set;

[0065] The reference signal's time-frequency resource occupancy is part or all of the time-domain resources + all frequency-domain resources of the beam downlink transmission of the predicted resource set;

[0066] The reference signal's time-frequency resource occupancy is a portion of the frequency domain resources and a portion of the time domain resources of the beam downlink transmission of the predicted resource set;

[0067] The temporal resource occupancy of the reference signal in the predicted resource set is either continuous or discontinuous.

[0068] The frequency domain resources of the reference signal are continuous or discontinuous resource blocks RB or resource block groups RBG in the prediction resource set.

[0069] In one embodiment, the model monitoring reference signal resource configuration information is also used to instruct the model to monitor time and frequency resources.

[0070] In one embodiment, the second network element includes at least one of gNB, UE, LMF, and OAM.

[0071] In one embodiment, the first message includes at least one of system message, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling.

[0072] In one embodiment, the second message includes at least one of system message, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling.

[0073] In one embodiment, the third message includes a CSI report message.

[0074] Thirdly, this application also provides a model monitoring device. The device includes:

[0075] The first receiving module is used to receive a first message sent by a first network element. The first message contains configuration information related to model monitoring. The configuration information related to model monitoring includes model monitoring process configuration information, model monitoring reference signal resource configuration information, and / or reporting configuration information.

[0076] The second receiving module is used to receive a second message sent by the first network element. The second message carries a reference signal, which is used to perform resource measurement to obtain tag data for model monitoring.

[0077] The sending module is used to send a third message to the first network element, wherein the third message carries the tag data and inference / prediction results, or the third message carries model monitoring results.

[0078] Fourthly, this application also provides a model monitoring device. The device includes:

[0079] The first sending module is used to send a first message to the second network element. The first message contains configuration information related to model monitoring. The configuration information related to model monitoring includes model monitoring process configuration information, model monitoring reference signal resource configuration information, and / or reporting configuration information.

[0080] The second sending module is used to send a second message to the second network element. The second message carries a reference signal, which is used to perform resource measurement to obtain tag data for model monitoring.

[0081] The receiving module is used to receive a third message sent by the second network element, wherein the third message carries the tag data and inference / prediction results, or the third message carries model monitoring results.

[0082] Fifthly, this application also provides a communication device, including: a transmitter and a receiver;

[0083] The receiver is used to receive a first message sent by the first network element. The first message contains configuration information related to model monitoring. The configuration information related to model monitoring includes model monitoring process configuration information, model monitoring reference signal resource configuration information, and / or reporting configuration information.

[0084] The receiver is configured to receive a second message sent by the first network element, the second message carrying a reference signal, the reference signal being used to perform resource measurement to obtain tag data for model monitoring;

[0085] The transmitter is used to send a third message to the first network element, the third message carrying the tag data and inference / prediction results, or the third message carrying model monitoring results.

[0086] Sixthly, this application also provides a communication device, including: a transmitter and a receiver;

[0087] The transmitter is used to send a first message to the second network element, the first message containing configuration information related to model monitoring; the configuration information related to model monitoring includes model monitoring process configuration information, model monitoring reference signal resource configuration information and / or reporting configuration information;

[0088] The transmitter is used to send a second message to the second network element, the second message carrying a reference signal, the reference signal being used to perform resource measurement to obtain tag data for model monitoring;

[0089] The receiver is used to receive a third message sent by the second network element, the third message carrying the tag data and inference / prediction results, or the third message carrying model monitoring results.

[0090] In a seventh aspect, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the methods described in the first and / or second aspects.

[0091] Eighthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the methods described in the first and / or second aspects.

[0092] The aforementioned model monitoring method, apparatus, communication equipment, storage medium, and computer program product allow the communication equipment (e.g., UE) to obtain model monitoring-related configuration information by receiving a first message sent by a first network element. This configuration information includes at least one of model monitoring process configuration information, model monitoring reference signal resource configuration information, and reporting configuration information. These three types of information respectively determine the model monitoring process, the resource configuration of the model monitoring reference signal, and the configuration information reported to the first network element. The communication equipment receives a second message sent by the first network element, obtains a reference signal from the second message, and measures the reference signal to obtain tag data for model monitoring. The communication equipment sends a third message to the first network element, which may carry tag data and inference / prediction results, or it may carry model monitoring results determined based on the tag data and inference / prediction results. The first network element can obtain the model monitoring results determined by the communication equipment, thereby enabling monitoring of the performance of the AI / ML model on the communication equipment side. This application's solution standardizes the model monitoring process and resources for each communication equipment side, improving the monitoring efficiency of model monitoring on each communication equipment side. Attached Figure Description

[0093] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0094] Figure 1 This is a diagram illustrating the application environment of the model monitoring method in one embodiment;

[0095] Figure 2 This is a flowchart illustrating a model monitoring method in one embodiment;

[0096] Figure 3 This is a flowchart illustrating the model monitoring method in another embodiment;

[0097] Figure 4 This is a flowchart illustrating the model monitoring method in another embodiment;

[0098] Figure 5 This is a flowchart illustrating the model monitoring method in yet another embodiment;

[0099] Figure 6 This is a schematic diagram illustrating the relationship between reference signals and a set of resources in one embodiment.

[0100] Figure 7 This is a flowchart illustrating a single reporting process in one embodiment;

[0101] Figure 8 This is a flowchart illustrating an independent reporting process in one embodiment;

[0102] Figure 9 This is a structural block diagram of the model monitoring device in one embodiment;

[0103] Figure 10 This is a structural block diagram of the model monitoring device in another embodiment;

[0104] Figure 11 This is an internal structural diagram of a communication device in one embodiment. Detailed Implementation

[0105] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0106] With the continuous evolution of wireless communication technology, AI / ML technology is being applied to wireless networks, defining three physical layer use cases: CSI (Channel State Information) feedback enhancement, beam management, and positioning enhancement. These use cases can help network elements optimize scheduling, resource allocation, and beamforming. Currently, related technologies define model lifecycle management frameworks for different use cases, including components such as data collection, model training, model inference, model monitoring, and model switching. A crucial issue for wireless AI is how to ensure good performance across different scenarios, configurations, and sites, thus determining the model monitoring method. Model monitoring can be performed after model inference, monitoring the inference / prediction results on the NW (Network) / UE (User Equipment) side, and making decisions based on different monitoring results. Currently, the time-frequency resource / resource set configuration and signaling flow for model monitoring are not yet determined; therefore, it is necessary to define and design the configuration information and processes related to AI model monitoring.

[0107] The model monitoring method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, the first network element 104 communicates with the second network element 102 via a network. The first network element 102 can be, but is not limited to, an NW, gNB (NextGeneration Node B, 5G base station), LMF (Location Management Function), OAM (Operations Administration and Maintenance), etc. The second network element 104 can be, but is not limited to, a UE, gNB, LMF, OAM, etc., and is not limited here.

[0108] The first and second network elements can transmit configuration information related to model monitoring by transmitting the first message, and can execute the model monitoring process by transmitting the second and third messages, thereby performing model monitoring on the intelligent model of the wireless intelligent air interface.

[0109] In one embodiment, such as Figure 2 As shown, a model monitoring method is provided, which can be applied to... Figure 1 Taking the second network element as an example, for instance, the second network element can be a user-side UE, including the following steps S202 to S206:

[0110] Step S202: Receive a first message sent by the first network element. This first message contains configuration information related to model monitoring. This configuration information is used to instruct the information receiving device to perform the model monitoring process.

[0111] The first network element can be a non-terminal type communication device or a terminal type communication device.

[0112] Model monitoring refers to a stage in model lifecycle management where each AI / ML model is monitored to determine its inference / prediction results, thereby aiding decision-making. Model monitoring is used to monitor performance changes of AI / ML models in a wireless intelligent air interface. The first network element can determine the inference / prediction results of each AI / ML model through model monitoring, monitor these results to obtain model monitoring results, and make decisions based on these results. These decisions may include at least one of the following operations: model selection, activation, deactivation, switching, or rollback. It should be understood that model inference and model prediction can represent the same meaning. In the embodiments of this application, " / " represents "or," for example, inference / prediction results can be represented as inference results or prediction results.

[0113] The configuration information related to model monitoring is used to define and set the model monitoring process, reference signals, and model monitoring reporting involved in model monitoring. This configuration information includes at least one of the following: model monitoring process configuration information, model monitoring reference signal resource configuration information, and reporting configuration information. The model monitoring process configuration information can be used to define the model monitoring signaling process and triggering method. The model monitoring reference signal resource configuration information is used to define the time-frequency resource location of the reference signal used for model monitoring in the corresponding resource / resource set, as well as the reference signal type. The model monitoring reference signal resource configuration information can also include other resource configuration information related to the model monitoring reference signal, such as defining the resource configuration information when issuing reference signals. The reporting configuration information is mainly used to define the configuration information related to reporting monitoring results from the second network element to the first network element during model monitoring.

[0114] Specifically, the first network element can send a first message carrying configuration information related to model monitoring to the UE. The UE can receive the configuration information related to model monitoring in the first message, and obtain the model monitoring process configuration information, model monitoring reference signal resource configuration information, and reporting configuration information contained in the configuration information related to model monitoring. Through the model monitoring process configuration information, the UE can learn about the signaling interaction process between itself and the first network element, as well as the triggering method for triggering model monitoring, thereby providing a basis for triggering model monitoring and signaling interaction for AI / ML model monitoring. Based on the model monitoring reference signal resource configuration information, the UE can selectively receive the reference signals required for model monitoring, and obtain the tag data corresponding to the resource / resource set based on the reference signals. This tag data can be used for model monitoring. By receiving the reporting configuration information, the UE can determine the actions to be performed during model monitoring, the content to be reported, and the reporting method, so as to facilitate the subsequent orderly reporting of model monitoring-related information to the first network element.

[0115] In an exemplary embodiment, the first network element includes at least one of gNB, UE, LMF, and OAM. When the first network element sends a first message and / or a second message, the first message and / or the second message can be sent through multiple first network elements of the same or different types, or through one first network element, or through a subset of first network elements sending the first message and another subset sending the second message. Furthermore, the method of this application embodiment can achieve model monitoring between any air interfaces.

[0116] In an exemplary embodiment, the first message includes at least one of the following: system message, RRC (Radio Resource Control) signaling, MAC CE (Media Access Control Element) signaling, DCI (Downlink Control Information) signaling, and NAS (Non-Access Stratum) signaling. When sending the first message, the first network element can send at least one first message for different model monitoring. Each first message can be a different type of signaling, and each first message can correspond to at least one model. For the same model monitoring, multiple first messages of different types can be sent, so that the first network element can send the first message to different network elements through different types of signaling. In addition, the first message can carry complete configuration information related to model monitoring; the first message can also carry partial configuration information related to model monitoring. Furthermore, the method of this application embodiment can also employ other message types supported by the air interface.

[0117] Step S204: Receive the second message sent by the first network element.

[0118] The second message carries a reference signal for model monitoring, which is used to perform resource measurements to obtain tag data for model monitoring.

[0119] Intelligent wireless interface (IUI) refers to the use of artificial intelligence (AI) and machine learning (ML) technologies to optimize various functions in the air interface of a wireless communication system (i.e., the communication link between the base station and user equipment), such as beamforming, resource allocation, and interference management. These optimization functions require a large amount of real-world network data. AI / ML models can be designed and trained to predict and optimize air interface behavior (such as signal quality and beam direction selection). Model monitoring verifies and evaluates the AI / ML models to ensure their accuracy and reliability in real-world scenarios.

[0120] In AI / ML model training and monitoring, labeled data refers to data with clearly defined results or target values. For example, in beam optimization scenarios, labeled data can reflect the signal transmission quality of different beams, such as the signal strength received by the UE through each beam, SINR (Signal to Interference plus Noise Ratio), or RSRP (Reference Signal Receiving Power). In model monitoring, labeled data serves as comparison data for the inference / prediction results of AI / ML models. After obtaining inference / prediction results from the AI / ML model based on the model input, the labeled data and the inference / prediction results can be used to determine the model evaluation metrics of the AI / ML model; for example, labeled data can be used to determine metrics such as the prediction accuracy of the AI / ML model.

[0121] In one embodiment, the NW (an example of the aforementioned first network element) sends a model monitoring reference signal, such as CSI-RS (Channel State Information Reference Signal) or SSB (Synchronization Signal Block), to the UE (an example of the aforementioned second network element). This reference signal is used by the UE to measure the corresponding air interface resources / resource sets to obtain tag data for model monitoring. The UE can then report the measurement results, i.e., the tag data of the corresponding air interface resources / resource sets, to the NW network side. This tag data is used for AI / ML model performance evaluation.

[0122] The reference signal type can be either CSI-RS or SSB. CSI-RS and SSB are key reference signals used for channel measurement and synchronization in 5G NR. CSI-RS focuses on providing high-precision channel state information to support beam optimization and resource scheduling, while SSB is mainly used for initial access, synchronization, and coarse channel quality measurements.

[0123] When the reference signal is CSI-RS or SSB, the tag data measured by the UE can include signal strength (such as RSRP, SINR), channel quality (such as CQI (Channel Quality Indicator), RI (rank indicator), beam TOPK tag, and other information. The above tag data supports the performance evaluation of different AI / ML models.

[0124] The reference signal can be transmitted based on the time and frequency resources configured for the reference signal in the configuration information related to model monitoring.

[0125] Specifically, the first network element can send reference signals to the UE using the configured time-frequency resources. The UE measures relevant resources based on the reference signals and generates tag data for those resources (such as signal strength, beam direction, channel quality, etc.). The UE can then report the measurement results to the first network element. This tag data can be used for performance testing. High-quality tag data ensures the intelligence level of the wireless intelligent interface, making the communication system more efficient and reliable in beam optimization, resource scheduling, and interference management.

[0126] In an exemplary embodiment, the second message includes at least one of system messages, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling. When sending the second message, the first network element can send at least one second message for different model monitoring. Each second message can be a different type of signaling, and each second message can correspond to at least one model. For the same model monitoring, multiple second messages of different types can be sent, so that the first network element can send second messages to different network elements through different types of signaling. In addition, a second message can carry a complete reference signal; a second message can carry a partial reference signal. Besides, the method of this application embodiment can also use other message types supported by the wireless intelligent air interface.

[0127] Step S206: Send a third message to the first network element. The third message carries the tag data and inference / prediction results, or the third message carries the model monitoring results.

[0128] In this model, inference / prediction results are obtained by predicting the model input based on measurements using an AI / ML model. Model monitoring results are obtained based on labeled data used for model monitoring and inference / prediction results. Labeled data used for model monitoring can be determined by measuring relevant resources / resource sets based on reference signals. Resources / resource sets can be understood as the objects predicted / managed by the AI / ML model; inputting resources / resource sets into the AI / ML model yields corresponding inference / prediction results. Labeled data used for model monitoring can be understood as the actual performance data of these objects themselves, which is not obtained through model inference / prediction. For example, both labeled data and inference / prediction results used for model monitoring can determine whether a beam within a resource / resource set actually belongs to the Top-K beams. Labeled data indicates whether the beam actually belongs to the Top-K beams, while inference / prediction results predict whether the beam actually belongs to the Top-K beams.

[0129] The inference / prediction results correspond to the inference process, which is used by the UE to measure and generate model inputs and predict the model inputs. The model monitoring results correspond to the monitoring process, which is used by the UE to measure the resources / resource sets monitored by the model and obtain the tag data corresponding to the resources / resource sets.

[0130] When the UE sends a third message to the first network element, and the third message carries tag data and inference / prediction results, it can be understood that the UE sends auxiliary information to the first network element. The auxiliary information is used by the first network element to calculate the model monitoring results and make network behavior decisions.

[0131] Specifically, the UE can send a third message carrying tag data and inference / prediction results to the first network element, or the UE can send a third message carrying model monitoring results to the first network element. After acquiring a reference signal and the corresponding tag data, the UE can measure the resource set / resource used as the model input for the AI / ML model, perform model inference, obtain the inference / prediction results for that resource set / resource, and send the tag data and inference / prediction results to the first network element via the third message. Alternatively, the UE can determine the corresponding model monitoring results based on the aforementioned tag data for model monitoring and the model inference / prediction results of the AI / ML model, and send the model monitoring results to the first network element via the third message.

[0132] In one exemplary embodiment, the third message includes a CSI report message. For example, the third message may include two CSI report messages, one of which may contain label data for model monitoring, and the other may contain inference / prediction results. Alternatively, the third message may include one CSI report message, which may contain both label data and inference / prediction results, or the third message may include one CSI report message, which may include model monitoring results.

[0133] In the aforementioned model monitoring method, the communication device (e.g., UE) obtains model monitoring-related configuration information by receiving a first message sent by a first network element. This configuration information includes at least one of model monitoring process configuration information, model monitoring reference signal resource configuration information, and reporting configuration information. These three types of information respectively determine the model monitoring process, the resource configuration of the model monitoring reference signal, and the configuration information reported to the first network element. The UE receives a second message sent by the first network element, obtains the reference signal from the second message, and measures the reference signal to obtain tag data for model monitoring. The UE sends a third message to the first network element. The third message may carry the tag data and the model's inference / prediction results, or it may carry model monitoring results determined based on the model monitoring tag data and the inference / prediction results, thereby enabling performance monitoring of the AI / ML model on the communication device side. This application's solution standardizes the model monitoring process and resources for each communication device side, improving the monitoring efficiency of model monitoring on each communication device side.

[0134] In an exemplary embodiment, the model monitoring process configuration information is used to indicate the model monitoring signaling process and the model monitoring triggering method. Specifically, the model monitoring process configuration information is used to define the model monitoring signaling process and the model monitoring triggering method, wherein the model monitoring signaling process can be a process that controls the transmission of first, second, and third messages between the first network element and the terminal side. For the UE side, when the UE detects model monitoring via a triggering method, it can execute the signaling interaction process required for model monitoring according to the model monitoring signaling process. The model monitoring triggering method can be defined as at least one of periodic, semi-static, non-periodic, and event-based triggering. The UE can autonomously determine / select the above-mentioned model monitoring triggering method.

[0135] In one exemplary embodiment, the reporting configuration information is used to indicate the process of model monitoring, the reporting signaling process, the reporting content, and / or the reporting timing.

[0136] Optionally, the reporting signaling process can be the signaling process executed during model monitoring reporting. This signaling process includes the signaling transmission process between the first network element and the second network element for the third message during model monitoring reporting. The reported content can be the inference / prediction results of the resource, such as the resource's performance indicators, like the Top-K beam information corresponding to the resource. The reporting timing can be periodic, aperiodic, or semi-periodic, such as periodic reporting based on time intervals, like reporting every minute, every hour, or every day.

[0137] In an exemplary embodiment, the model monitoring process includes an inference process and / or a monitoring process: an inference process for instructing the measurement model to input and make predictions, and reporting the inference / prediction results; and a monitoring process for instructing the measurement model to monitor resources and obtain monitoring results, and reporting the monitoring results.

[0138] Specifically, the model monitoring process includes at least one of an inference process and a monitoring process. The UE can choose at least one process to report, thereby reporting inference / prediction results and / or monitoring results. If the model monitoring process includes either an inference process or a monitoring process, the UE can use the corresponding process to report, thereby reporting inference / prediction results or monitoring results. Model monitoring resources can be determined based on corresponding resources / resource sets.

[0139] Furthermore, the UE can determine the model input of the AI / ML model based on the inference flow within the model monitoring process, and then predict resources based on the model input and the AI / ML model to obtain the corresponding inference / prediction results. Alternatively, the UE can determine the model-monitored resources based on the monitoring flow within the model monitoring process and obtain the corresponding tag data for those resources.

[0140] In an exemplary embodiment, when the measurement resource set / measurement resources are a subset of the model monitoring resource set, the reporting signaling process is the model monitoring reporting signaling process.

[0141] Specifically, the measurement resource set / measurement resources can be model inputs, and can be determined based on resources / resource sets. When the measurement resource set / measurement resources are a subset of the model monitoring resource set, the model input can be directly determined through the model monitoring resource set. Prediction is then performed using the model input and the AI / ML model to obtain the inference / prediction results corresponding to the measurement resource set / measurement resources. Monitoring can then be performed directly through the model monitoring resource set to obtain monitoring results. Based on this, the model monitoring reporting signaling process can be used as the reporting signaling process in this case.

[0142] In one example, when the measurement resource set SetB is a subset of the model monitoring resource set SetM, SetB is entirely contained within SetM. In this case, only the model monitoring reporting configuration needs to be defined; no additional processing is required. The model monitoring process can define only one model monitoring reporting signaling flow, such as configuring the CSI report file for SetM. The CSI report file allows for the measurement of model inputs to obtain inference / prediction results, as well as the measurement of the model monitoring resource set to obtain label data. Based on the label data and the inference / prediction results, the monitoring results are calculated.

[0143] In one exemplary embodiment, when the measurement resource set / measurement resources are not entirely a subset of the model monitoring resource set, the reporting signaling process includes a single reporting process and / or independent reporting processes.

[0144] The single reporting process is used to instruct the reporting of the inference / prediction results of the measurement resource set / measurement resources and the monitoring results of the model monitoring resource set through a single reporting process; the independent reporting process is used to instruct the reporting of the inference / prediction results of the measurement resource set / measurement resources and the monitoring results of the model monitoring resource set through different reporting processes.

[0145] Specifically, when the measurement resource set / measurement resources are not entirely a subset of the model monitoring resource set, meaning that some of the measurement resource set / measurement resources may not exist in the model monitoring resource set, the reporting signaling process can be determined to be a single reporting process and / or an independent reporting process.

[0146] In one example, if the measurement resource set SetB is not a subset of the model monitoring resource set SetM, then some resources in the measurement resource set SetB are not in the model monitoring resource set SetM. In this case, it is necessary to further define how to report the measurement content. The reporting signaling process can be defined as a single reporting process or as an independent reporting process.

[0147] A single reporting process can merge the configurations corresponding to the inference process and the monitoring process into one reporting process. Through this reporting process, the inference / prediction results of the measurement resource set SetB and the monitoring results of the model monitoring resource set SetM can be determined together.

[0148] Independent reporting processes can include reporting processes corresponding to the inference process and reporting processes corresponding to the monitoring process. The terminal can determine the inference / prediction results of the measurement set SetB and the monitoring results of the model monitoring resource set SetM based on the reporting processes corresponding to the inference process. In one example, if the reporting process corresponding to the monitoring process is not enabled, the terminal can also process the data using only the reporting process corresponding to the inference process.

[0149] In one example, a configuration scenario where the UE reports signal quality measurements in wireless communication mainly involves two sets: a measurement resource set (SetB), used for inference via AI / ML models, and a model monitoring resource set (SetM), used for model monitoring. SetB and SetM can be determined based on the resources / resource sets corresponding to the wireless intelligent interface.

[0150] SetB and SetM can have various relationships. For example, if SetB is a subset of SetM, and the SetB measurement resource set is entirely contained within the model monitoring resource set SetM, then only the model monitoring reporting configuration needs to be defined, without any additional processing. Conversely, if SetB is not a subset of SetM, and some resources in the SetB measurement set are not in the model monitoring resource set SetM, then further definition is needed on how to report the measurement content.

[0151] In one exemplary embodiment, the reported content includes one of the following:

[0152] Report metrics; report events; based on events, report both events and metrics.

[0153] Specifically, the content reported by the model monitoring can be configured in one of the following ways:

[0154] 1. Report metric: The UE directly reports the measured signal metrics (such as Top-K beam, signal strength, signal-to-noise ratio, etc.).

[0155] 2. Report event: The UE reports specific events (such as signal strength below the threshold, handover trigger, etc.).

[0156] 3. Based on event, report event and metric(s): When a specific event occurs, the UE reports the event and the corresponding measurement metric(s) simultaneously.

[0157] 4. Other possibilities may exist: In addition to the methods mentioned above, there may be other reporting options, depending on the specific needs.

[0158] In one exemplary embodiment, the configuration of the reporting timing includes at least one of periodic, semi-static, non-periodic, and event-triggered modes.

[0159] Specifically, semi-static is used to indicate that part of the reporting timing is periodic and part is non-periodic.

[0160] In one example, the reporting timing can be based on a preset time period, with each interval marking a reporting opportunity, and reporting is performed periodically according to the above process. The reporting timing can also be non-periodic, for example, pre-setting reporting at multiple discrete time points. The reporting timing can also be semi-static, for example, with some reporting opportunities being fixed and others being periodic.

[0161] In one exemplary embodiment, the reported configuration information also includes indication information, which indicates that the UE is allowed to decide / select the reporting method information.

[0162] Specifically, the first network element can send supported configurations to the UE on the terminal side by reporting configuration information. The UE can decide / select from the supported configurations sent by the first network element according to its own situation. In one example, the first network element sends out the supported periodic / semi-static / aperiodic reporting timings, and the UE can decide / select the semi-static mode according to its own situation.

[0163] In one exemplary embodiment, model monitoring reference signal resource configuration information is used to indicate the location of reference signal transmission time-frequency resources and the type of reference signal.

[0164] In an exemplary embodiment, the reference signal transmission time-frequency resource location includes: information regarding the relationship between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set. The reference signal transmission time-frequency resource location can be obtained by configuring the occupied resources / resource set when transmitting the reference signal.

[0165] In one exemplary embodiment, the relationship information between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set includes one of the following:

[0166] The reference signal monitored by the model occupies part or all of the frequency domain resources and all of the time domain resources of the beam downlink transmission of the prediction resource set; the reference signal monitored by the model occupies part or all of the time domain resources and all of the frequency domain resources of the beam downlink transmission of the prediction resource set; the reference signal monitored by the model occupies part of the frequency domain resources and part of the time domain resources of the beam downlink transmission of the prediction resource set; the time domain resources of the reference signal monitored by the model occupying the time domain resources of the prediction resource set are continuous or discontinuous; the frequency domain resources of the reference signal monitored by the model are continuous or discontinuous resource blocks (RBs) or resource block groups (RBGs) in the prediction resource set.

[0167] Specifically, the predicted resource set's beam downlink transmission process involves a channel composed of resources in both the time domain and frequency domain dimensions. For example... Figure 6 The diagram illustrates the relationship between the reference signal and the prediction resource set. In one example, as shown... Figure 6 As shown in the first part, the model-monitored reference signal (such as CFR) can occupy all the time-domain resources of a channel, and in the case of occupying all the time-domain resources, it also occupies a portion of the channel's frequency-domain resources. Additionally, the model-monitored reference signal can occupy all the time-domain resources of a channel, and in the case of occupying all the time-domain resources, it also occupies all the channel's frequency-domain resources; for example, CFR occupies the entire BWP.

[0168] In one example, such as Figure 6 As shown in the second part, the reference signal monitored by the model (such as CFR) can occupy all the frequency domain resources of the channel, and in the case of occupying all the frequency domain resources, it also occupies a portion of the time domain resources of the channel. Furthermore, the reference signal monitored by the model can occupy all the frequency domain resources of the channel, and in the case of occupying all the frequency domain resources, it also occupies all the time domain resources of the channel.

[0169] In one example, such as Figure 6 As shown in the third part, the reference signal (such as CFR) monitored by the model can occupy part of the channel's frequency domain resources, and in the case of occupying part of the frequency domain resources, it can also occupy part of the channel's time domain resources.

[0170] In one example, the temporal resource usage of the reference signal monitored by the model in the set of resources may be continuous or discontinuous. For instance, in the temporal resource interval [0,100] of the set of resources, if the temporal resource interval occupied by the reference signal monitored by the model is [0,10] and [20,30], it is a discontinuous temporal resource. If the temporal resource interval occupied by the reference signal monitored by the model is [0,30], it is a continuous temporal resource.

[0171] In one example, the frequency domain resources of the reference signal monitored by the model are continuous or non-contiguous resource blocks (RBs) or resource block groups (RBGs) within a defined resource set. For instance, within the frequency domain resource interval [0, 10000] of the defined resource set, if the frequency domain resource intervals occupied by the reference signal monitored by the model are [0, 1000] and [2000, 3000], they are non-contiguous RBs or RBGs. If the frequency domain resource interval occupied by the reference signal monitored by the model is [0, 3000], it is a continuous RB or RBG.

[0172] In one example, the predicted resource set can be determined as SetA, which can be determined based on the wireless intelligent air interface (UAI) resources / resource set. For example, SetA can be the complete UAI resources / resource set, or it can be a subset of the UAI resources / resource set.

[0173] In one example, the time-frequency resources used when performing resource measurements based on a reference signal are a subset or the entirety of the predicted resource set. For instance, the time-frequency resources used when performing resource measurements based on a reference signal could be SetM, which could be a subset or the entirety of SetA.

[0174] In one exemplary embodiment, the model monitoring reference signal resource configuration information is also used to instruct the model to monitor time and frequency resources.

[0175] Specifically, the indication of frequency resources for model monitoring can include frequency domain resource indication and time domain resource indication. Frequency domain resource indication can use the lowest carrier bandwidth frequency point (PointA) as a reference point or the starting point of the BWP bandwidth part of resource SetA as a reference point. For example, it can use the RIV (Resource Indicator Value) or Bitmap method to indicate the frequency domain resources occupied by the model monitoring, or define the frequency domain resource occupancy and its period for indication. In one example, without configuring frequency domain resource indication, the default is full frequency domain resource occupancy.

[0176] Time-domain resource indication can be configured by specifying the time-domain period, time slot offset, symbol start position, and number of symbols, or by using the Bitmap method. In one example, without configuring time-domain resource indication, the default is to indicate that all time-domain resources are occupied.

[0177] In one embodiment, such as Figure 3 As shown, a model monitoring method is provided, which can be applied to... Figure 1 The following steps, S302 to S306, are used as an example to illustrate the process of the first network element (network-side NW):

[0178] Step S302: Send a first message to the second network element. The first message contains configuration information related to model monitoring. The configuration information related to model monitoring includes model monitoring process configuration information, model monitoring reference signal resource configuration information, and / or reporting configuration information.

[0179] Specifically, the NW can send a first message carrying configuration information related to model monitoring to the second network element. The second network element can receive the configuration information related to model monitoring in the first message, obtaining the model monitoring process configuration information, model monitoring reference signal resource configuration information, and reporting configuration information contained therein. The second network element can use the model monitoring process configuration information to instruct the signaling interaction process between the terminal and the first network element, as well as the triggering method for model monitoring, thereby providing a basis for triggering model monitoring and signaling interaction for the AI / ML model. Based on the model monitoring reference signal resource configuration information, the second network element can selectively receive the reference signals required for model monitoring and measure the reference signals to obtain the tag data corresponding to the resource / resource set. The second network element can receive and apply the reporting configuration information to determine the actions to be performed during model monitoring, the content to be reported, and the reporting method, so as to facilitate the subsequent orderly execution of signaling related to the model monitoring reporting process between the second and first network elements.

[0180] In an exemplary embodiment, the first network element includes at least one of gNB, UE, LMF, and OAM. When sending the first message and / or the second message, the first network element can send the complete first message and / or the second message through one first network element, or it can send a portion of the first message and / or the second message through multiple first network elements, or each first network element can send the complete first message and / or the second message individually. Furthermore, the method of this application embodiment can realize model monitoring of any wireless intelligent air interface.

[0181] In one embodiment, the first message includes at least one of system messages, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling. When sending the first message, if there is one type of first message, it can carry complete model monitoring-related configuration information; if there are more than one type of first message, each type can carry partial model monitoring-related configuration information; if there are more than one type of first message, each type can also carry complete model monitoring-related configuration information individually. In addition, the method of this application embodiment can also employ other message types supported by the wireless intelligent air interface.

[0182] Step S304: Send a second message to the second network element.

[0183] The second message carries a reference signal for model monitoring, which is used to perform resource measurements to obtain tag data for model monitoring. The reference signal can be sent based on the time-frequency resources configured in the model monitoring-related configuration information.

[0184] Specifically, the NW sends reference signals to the second network element. The second network element measures these reference signals and generates a series of index data (such as signal strength, beam direction, channel quality, etc.). The second network element can then report the measurement results back to the NW.

[0185] In one exemplary embodiment, the second message includes at least one of system messages, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling. When sending the second message, if there is one type of second message, it can carry complete model monitoring-related configuration information; if there are more than one type of second message, each type can carry partial model monitoring-related configuration information; if there are more than one type of second message, each type can also carry complete model monitoring-related configuration information individually. In addition, the method of this application embodiment can also employ other message types supported by the wireless intelligent air interface.

[0186] Step S306: Receive a third message sent by the second network element. The third message carries tag data and inference / prediction results, or the third message carries model monitoring results.

[0187] The inference / prediction results are obtained by performing model inference based on the model input, while the model monitoring results are obtained based on the model monitoring label data and the inference / prediction results. The model monitoring label data is determined based on the model monitoring resource set acquired during model monitoring.

[0188] The NW receiving the third message sent by the second network element can be understood as the second network element sending auxiliary information to the NW. This auxiliary information is used by the NW to calculate the model monitoring results and make network behavior decisions.

[0189] Specifically, the second network element can send a third message carrying label data and inference / prediction results to the NW, or the second network element can send a third message carrying model monitoring results to the NW. After acquiring the reference signal and the corresponding label data, the second network element can measure the input resources of the AI / ML model and perform model inference to obtain the inference / prediction results, and then send the label data and inference / prediction results to the NW via the third message. Alternatively, the terminal can determine the corresponding model monitoring results based on the model monitoring label data and the model inference / prediction results of the AI / ML model, and then send the model monitoring results to the NW via the third message.

[0190] In one exemplary embodiment, the third message includes a CSI report message. For example, the third message may include two CSI report messages, one of which may contain label data for model monitoring, and the other may contain inference / prediction results. Alternatively, the third message may include one CSI report message, which may contain both label data and inference / prediction results, or the third message may include one CSI report message, which may include model monitoring results.

[0191] In the aforementioned model monitoring method, the NW sends a first message to the second network element to provide the second network element with configuration information related to model monitoring. This configuration information includes at least one of the following: model monitoring process configuration information, model monitoring reference signal resource configuration information, and reporting configuration information. These three types of information respectively determine the model monitoring process, the resource configuration of the reference signal during model monitoring, and the configuration information reported to the first network element. The NW sends a second message to the second network element, from which the second network element can obtain the reference signal and measure it to obtain tag data for model monitoring. The NW receives a third message from the second network element. This third message may carry the tag data and the model's inference / prediction results, or it may carry model monitoring results determined based on the model monitoring tag data and the inference / prediction results. The NW can indirectly or directly obtain the model monitoring results determined by the second network element, thereby enabling the monitoring of the performance of the AI / ML model on the communication device side. This application's solution standardizes the model monitoring process and resources for each communication device side, improving the monitoring efficiency of model monitoring on each communication device side.

[0192] In an exemplary embodiment, the method further includes: calculating model monitoring results when the second network element sends tag data and inference / prediction results.

[0193] In an exemplary embodiment, the model monitoring process configuration information is used to indicate the model monitoring signaling process and the model monitoring triggering method. Specifically, the model monitoring process configuration information is used to define the model monitoring signaling process and the model monitoring triggering method. From the UE side, when the UE detects model monitoring via a triggering method, it can execute the signaling interaction process required for model monitoring according to the model monitoring signaling process. The model monitoring triggering method can be defined as at least one of periodic, semi-static, non-periodic, and event-based triggering. The UE can independently determine / select the above-mentioned model monitoring triggering method.

[0194] In one exemplary embodiment, the reporting configuration information is used to indicate the process of model monitoring, the reporting signaling process, the reporting content, and / or the reporting timing.

[0195] In an exemplary embodiment, the model monitoring process includes an inference process and / or a monitoring process: an inference process for instructing the measurement model to input and make predictions, and reporting the inference / prediction results; and a monitoring process for instructing the measurement model to monitor resources and obtain monitoring results, and reporting the monitoring results.

[0196] In an exemplary embodiment, when the measurement resource set / measurement resources are a subset of the model monitoring resource set, the reporting signaling process is the model monitoring reporting signaling process.

[0197] In one exemplary embodiment, when the measurement resource set / measurement resources are not entirely a subset of the model monitoring resource set, the reporting signaling process includes a single reporting process and / or independent reporting processes.

[0198] The single reporting process is used to instruct the reporting of the inference / prediction results of the measurement resource set / measurement resources and the monitoring results of the model monitoring resource set through a single reporting process; the independent reporting process is used to instruct the reporting of the inference / prediction results of the measurement resource set / measurement resources and the monitoring results of the model monitoring resource set through different reporting processes.

[0199] In one exemplary embodiment, the reported content includes one of the following: reporting metrics; reporting events; and reporting both events and metrics based on events.

[0200] In one exemplary embodiment, the configuration of the reporting timing includes at least one of periodic, semi-static, non-periodic, and event-triggered modes.

[0201] In one exemplary embodiment, the reported configuration information also includes indication information, which indicates that the UE is allowed to decide / select the reporting method information.

[0202] In one exemplary embodiment, model monitoring reference signal resource configuration information is used to indicate the location of reference signal transmission time-frequency resources and the type of reference signal.

[0203] In an exemplary embodiment, the reference signal transmission time-frequency resource location includes: information regarding the relationship between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set. The reference signal transmission time-frequency resource location can be obtained by configuring the occupied resources / resource set when transmitting the reference signal.

[0204] In one exemplary embodiment, the relationship information between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set includes one of the following:

[0205] The reference signal monitored by the model occupies part or all of the frequency domain resources and all of the time domain resources of the beam downlink transmission of the prediction resource set; the reference signal monitored by the model occupies part or all of the time domain resources and all of the frequency domain resources of the beam downlink transmission of the prediction resource set; the reference signal monitored by the model occupies part of the frequency domain resources and part of the time domain resources of the beam downlink transmission of the prediction resource set; the time domain resources of the reference signal monitored by the model occupying the time domain resources of the prediction resource set are continuous or discontinuous; the frequency domain resources of the reference signal monitored by the model are continuous or discontinuous resource blocks RB or resource block groups RBG in the prediction resource set.

[0206] In one exemplary embodiment, the model monitoring reference signal resource configuration information is also used to instruct the model to monitor time and frequency resources.

[0207] In an exemplary embodiment, the specific execution process of the above-described model monitoring method is described in detail below with reference to a specific embodiment. This model monitoring method is used for configuring model monitoring resources for a wireless intelligent air interface. The model monitoring method can define and design the model monitoring signaling flow; define and design the time-frequency resource configuration information for model monitoring, including the resource configuration of reference signals and the reporting resource configuration information; define and design the model monitoring reporting content and reporting timing; and define and design the resource occupancy mechanism between model monitoring resources and SetA.

[0208] It should be understood that model monitoring refers to a phase in model lifecycle management where each AI / ML model can be monitored to determine its inference / prediction results, thereby assisting in decision-making. Model monitoring includes UE-side model monitoring and NW-side model monitoring.

[0209] like Figure 4 and Figure 5 As shown in the example, the specific execution process of the model monitoring method includes:

[0210] Step 1: The UE receives signaling sent by the NW. This signaling carries configuration information related to model monitoring, which is applicable to the AI-driven wireless intelligent air interface model performance monitoring process. For example, the signaling includes resource configuration information and / or reporting configuration information.

[0211] The UE or NW can also be other network elements, including at least one of gNB, UE, LMF, and OAM. Signaling includes at least one of system messages, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling.

[0212] Furthermore, the configuration information related to model monitoring includes: model monitoring signaling flow, used to indicate the model monitoring signaling flow and model monitoring triggering method; reference signal time-frequency resource configuration information, used to identify the location of the time-frequency resource of the reference signal for model monitoring and the type of reference signal, such as CSI-RS / SSB, based on which measurement is performed to obtain tag data; model monitoring reporting configuration information, used to identify the model monitoring reporting timing and signaling flow, as well as the relevant reporting content and reporting method; and the resource occupancy mechanism between model monitoring resources and SetA, used to indicate the resource relationship between model monitoring resources and SetA.

[0213] Step 2: The NW sends a model monitoring reference signal to the UE. The model monitoring reference signal is used by the UE to perform beam measurements to obtain model monitoring tag data. This reference signal is sent based on the time-frequency resources configured in the model monitoring related configuration information.

[0214] Step 3: The UE sends auxiliary information to the NW. This auxiliary information is used by the gNB to calculate model monitoring results and make network behavior decisions. In one example, such as... Figure 4 As shown, the UE can report beam measurement results based on network resource configuration, and the network side can calculate model monitoring results and make decisions. In one example, such as... Figure 5 As shown, the UE can report beam measurement results, inference / prediction results and / or monitoring results based on network resource configuration. The network can calculate the model monitoring results and make decisions.

[0215] Network behavior decisions can include model selection / activation / deactivation / switching / rollback operations, etc.

[0216] In one example, the relationship between the model monitoring reference signal resources and the time-frequency resources of the SetA predicted beam set can be defined as one of the following:

[0217] The model monitoring reference signal occupies part / all of the frequency domain BWP resources and all of the time domain resources of the SetA beam downlink transmission; the model monitoring reference signal occupies part / all of the time domain resources and all of the BWP resources of the SetA beam downlink transmission; the model monitoring reference signal occupies part of the frequency domain BWP resources and part of the time domain resources of the SetA beam downlink transmission.

[0218] Additionally, the time-domain resources occupied by the model monitoring reference signal resources can be defined as continuous or discontinuous (periodic / semi-static / aperiodic). For example, it can be a slot / mini-slot / OFDM symbol, where a slot is the basic unit of time resources; a mini-slot is a shorter time slot concept specifically designed for low-latency and high-frequency scheduling scenarios; and an OFDM symbol is the basic information unit in orthogonal frequency division multiplexing.

[0219] The model monitoring frequency domain resources can be continuous or discontinuous (periodic / semi-static / aperiodic) RBs or RBGs in SetA.

[0220] The model monitoring resources can be determined / selected by the UE (e.g., based on activated TCI or inference / prediction results).

[0221] Define the model monitoring time-frequency resource indicator as follows:

[0222] Frequency domain resource indication: Using PointA (the lowest point of the carrier bandwidth frequency) as a reference point, or the BWP (Broadband Wrapper) start point of resource SetA as a reference point, for example, using RIV (Resource Indicator Value) or Bitmap methods to indicate the frequency domain resources occupied by the model monitoring, or defining frequency domain resource occupancy and its period for indication. For example, when no configuration is performed, it indicates that the frequency domain resources are fully occupied.

[0223] Time-domain resource indication: Time-domain resources are indicated by configuring the time-domain period, time slot offset, symbol start position, and number of symbols, or by using the Bitmap method. For example, when no configuration is made, it indicates that the time-domain resources are fully occupied.

[0224] In one example, the reporting configuration information for model monitoring can be defined:

[0225] Among them, the model monitoring process can be divided into (1) the inference process, which is used for UE measurement to generate model input for prediction, and (2) the monitoring process, which is used for UE measurement to monitor model resources to generate tag data.

[0226] Specifically, taking UE-side model monitoring as an example, the following situation is defined:

[0227] (A) When the SetB measurement set is a subset of the monitoring resource set, the reporting configuration only needs to define the model monitoring reporting configuration.

[0228] (B) When the SetB measurement set is not a subset of the model monitoring, the model monitoring report content can be defined as one of the following: 1. report metric; 2. report event; 3. based on event, report event and metric(s); 4. Other cases are not excluded.

[0229] In addition, the reporting timing for model monitoring resource configuration can be defined as one of the following methods: periodic, semi-static, non-periodic, or event-triggered. The model monitoring reporting configuration can be determined / selected by the UE.

[0230] In one embodiment, such as Figure 4 As shown, for the NW model monitoring process, the signaling flow can be defined as follows:

[0231] Step 1: The NW sends a message to the UE, which carries model monitoring-related configurations, including: reference signal measurement / reporting configuration, used for beam management of control beam / data beam.

[0232] Step 2: The NW sends a message to the UE, which carries a reference signal for the UE to perform beam measurements to obtain tag data.

[0233] Step 3: The UE sends a message to the NW, which carries the model monitoring tag data obtained by the UE.

[0234] Step 4: The NW calculates the performance metrics of the model and, in conjunction with the current network environment, cell / UE configuration, etc., provides a decision option.

[0235] In one embodiment, such as Figure 5 As shown, for the NW model monitoring process, the signaling flow can be defined as follows:

[0236] Step 1: The NW sends a message to the UE, which carries model monitoring-related configurations, including: reference signal measurement / reporting configuration, used for beam management of control beam / data beam.

[0237] Step 2: The NW sends a message to the UE, which carries a reference signal for the UE to perform beam measurements to obtain tag data.

[0238] Step 3: The UE sends a message to the NW, which carries the model monitoring label data and inference / prediction results obtained by the UE side, or the monitoring results of the UE side.

[0239] Step 4 (step 3): (1) The UE sends the monitoring results. The NW only needs to combine the current network environment, cell / UE configuration, etc., and make decisions based on the monitoring results; (2) The UE sends the model monitoring label data and inference / prediction results. The NW needs to calculate the monitoring results and then make decisions.

[0240] In one example, such as Figure 6 As shown, the relationship between the model monitoring reference signal resources and the time-frequency resources of the SetA prediction beamset can be defined as one of the following:

[0241] The model monitoring reference signal occupies part / all of the frequency domain BWP resources and all of the time domain resources for downlink transmission of the SetA beam. The model monitoring reference signal occupies part / all of the time domain resources and all of the BWP resources for downlink transmission of the SetA beam. The model monitoring reference signal occupies part of the frequency domain BWP resources and part of the time domain resources for downlink transmission of the SetA beam.

[0242] In one example, such as Figure 7 and Figure 8 As shown, the reporting signaling process in the model monitoring and reporting configuration information can include an inference process and a monitoring process. The inference process is used by the UE to measure and generate model inputs for prediction, while the monitoring process is used by the UE to measure model monitoring resources and generate label data. Taking the UE-side model as an example, the following situation is defined:

[0243] (A) When the SetB measurement set is a subset of the monitoring resource set, the reporting configuration only needs to define the model monitoring reporting configuration.

[0244] (B) When the SetB measurement set is not entirely a subset of the model's monitored set, such as... Figure 7 As shown, a single CSI report configuration file (1) can be used for both inference and monitoring processes through a single reporting workflow. The model input SetB is obtained through measurement, and resource prediction is performed based on SetB to obtain the Top-K beams. The model can also monitor resource SetM and execute the monitoring process to obtain the resource monitoring results corresponding to SetM. Finally, the resource inference / prediction results and resource monitoring results are reported.

[0245] When the SetB measurement set is not exactly a subset of the model's monitoring set, such as Figure 8 As shown, two independent CSI report configuration files can be used to perform the inference and monitoring processes separately. The model input SetB is obtained through measurement, and resource prediction is performed based on SetB to obtain the Top-K beam. The independent monitoring process measures the monitored resource SetM and executes the monitoring process to obtain the resource monitoring results corresponding to SetM. Finally, the resource inference / prediction results and resource monitoring results are reported.

[0246] This embodiment proposes a model monitoring signaling mechanism and a model monitoring resource configuration method suitable for wireless intelligent air interfaces. This model monitoring process can be used for model switching between gNB / UE / OAM / LMF / CN and supports lifecycle management of corresponding network element AI models. Application scenarios for this method can include 6G, 5G-Advanced, wireless AI, and lifecycle management.

[0247] This embodiment enables inter-network element model monitoring capabilities by carrying model monitoring resource configuration information, including reference signal resources and monitoring reporting resources, through signaling, ensuring model performance, and determining a model monitoring signaling process suitable for the AI ​​model lifecycle management process, filling a gap in 3GPP standards.

[0248] This embodiment supports different sizes and continuous / non-continuous model monitoring resource configurations based on the different capabilities of network elements. This allows for flexible model monitoring in specific scenarios or tasks, tailored to different needs and conditions. The monitoring results then inform processing and decision-making, improving model monitoring resource utilization and reducing overhead. Furthermore, different AI models may perform differently in different scenarios / configurations / sites. By monitoring the performance of different models, the most suitable model for the current conditions can be selected, improving performance and accuracy and adapting to diverse needs. While some AI models may require more computing resources and energy to run, model switching / updating / optimization operations can save resources and energy costs when handling smaller-scale or simpler tasks. In the face of environmental changes and unexpected situations, model monitoring enhances the robustness and stability of the system, ensuring reliable performance.

[0249] This embodiment supports monitoring resource configuration based on event, periodic, semi-static, and aperiodic models. This resource configuration can flexibly adapt to different application scenarios and communication protocol frameworks, thereby improving communication efficiency and system reliability.

[0250] It should be understood that, although Figure 2-3 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2-3 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0251] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0252] Based on the same inventive concept, this application also provides a model monitoring device for implementing the model monitoring method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more model monitoring device embodiments provided below can be found in the limitations of the model monitoring method described above, and will not be repeated here.

[0253] In one exemplary embodiment, such as Figure 9 As shown, a model monitoring device 900 is provided, including: a first receiving module 901, a second receiving module 902, and a transmitting module 903, wherein:

[0254] The first receiving module 901 is used to receive a first message sent by the first network element. The first message contains configuration information related to model monitoring. The configuration information related to model monitoring includes model monitoring process configuration information, model monitoring reference signal resource configuration information, and / or reporting configuration information.

[0255] The second receiving module 902 is used to receive a second message sent by the first network element. The second message carries a reference signal, which is used to perform resource measurement to obtain tag data for model monitoring.

[0256] The sending module 903 is used to send a third message to the first network element. The third message carries tag data and inference / prediction results, or the third message carries model monitoring results.

[0257] Furthermore, the model monitoring process configuration information is used to indicate the model monitoring signaling process and the model monitoring triggering method.

[0258] Furthermore, the reported configuration information is used to indicate the process of model monitoring, the reporting signaling process, the reporting content, and / or the reporting timing.

[0259] Furthermore, the model monitoring process includes an inference process and / or a monitoring process: the inference process is used to instruct the measurement model to input and make predictions, and report the inference / prediction results; the monitoring process is used to instruct the measurement model to monitor resources and obtain monitoring results, and report the monitoring results.

[0260] Furthermore, the device also includes a reporting signaling process determination module, specifically used for: when the measurement resource set / measurement resources are a subset of the model monitoring resource set, the reporting signaling process is the model monitoring reporting signaling process.

[0261] Furthermore, the reporting signaling process determination module is specifically used for: when the measurement resource set / measurement resources are not entirely a subset of the model monitoring resource set, the reporting signaling process includes a single reporting process and / or independent reporting processes; a single reporting process is used to instruct the reporting of the inference / prediction results of the measurement resource set / measurement resources and the monitoring results of the model monitoring resource set through a single reporting process; independent reporting processes are used to instruct the reporting of the inference / prediction results of the measurement resource set / measurement resources and the monitoring results of the model monitoring resource set through different reporting processes respectively.

[0262] Furthermore, the reporting content includes one of the following: reporting metrics; reporting events; or reporting both events and metrics based on events.

[0263] Furthermore, the configuration of the reporting timing includes at least one of periodic, semi-static, non-periodic, and event-triggered modes.

[0264] Furthermore, the reported configuration information also includes indication information, which is used to indicate whether the reported configuration information allows the UE to decide / select.

[0265] Furthermore, the model monitoring reference signal resource configuration information is used to indicate the location of the reference signal transmission time-frequency resources and the type of reference signal.

[0266] Furthermore, the location of the reference signal transmission time-frequency resources includes information on the relationship between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set.

[0267] Furthermore, the relationship information between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set includes one of the following: the time-frequency resources of the reference signal occupy part or all of the frequency domain resources + all of the time domain resources of the beam downlink transmission of the prediction resource set; the time-frequency resources of the reference signal occupy part or all of the time domain resources + all of the frequency domain resources of the beam downlink transmission of the prediction resource set; the time-frequency resources of the reference signal occupy part of the frequency domain resources + part of the time domain resources of the beam downlink transmission of the prediction resource set; the time domain resources of the reference signal occupying the time domain resources in the prediction resource set are continuous or discontinuous; the frequency domain resources of the reference signal are continuous or discontinuous resource blocks RB or resource block groups RBG in the prediction resource set.

[0268] Furthermore, the model monitoring reference signal resource configuration information is also used to instruct the model monitoring time and frequency resources.

[0269] Furthermore, the first network element includes at least one of gNB, UE, LMF, and OAM.

[0270] Furthermore, the first message includes at least one of the following: system message, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling.

[0271] Furthermore, the second message includes at least one of system messages, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling.

[0272] Furthermore, the third message includes CSI report messages.

[0273] In one exemplary embodiment, such as Figure 10 As shown, a model monitoring device 1000 is provided, including: a first transmitting module 1001, a second transmitting module 1002, and a receiving module 1003, wherein:

[0274] The first sending module 1001 is used to send a first message to the second network element. The first message contains configuration information related to model monitoring. The configuration information related to model monitoring includes model monitoring process configuration information, model monitoring reference signal resource configuration information, and / or reporting configuration information.

[0275] The second sending module 1002 is used to send a second message to the second network element. The second message carries a reference signal, which is used to perform resource measurement to obtain tag data for model monitoring.

[0276] The receiving module 1003 is used to receive a third message sent by the second network element. The third message carries tag data and inference / prediction results, or the third message carries model monitoring results.

[0277] Furthermore, the device also includes a monitoring result calculation module, specifically used to calculate the model monitoring result when the second network element sends tag data and inference / prediction results.

[0278] Furthermore, the model monitoring process configuration information is used to indicate the model monitoring signaling process and the model monitoring triggering method.

[0279] Furthermore, the reported configuration information is used to indicate the process of model monitoring, the reporting signaling process, the reporting content, and / or the reporting timing.

[0280] Furthermore, the model monitoring process includes an inference process and / or a monitoring process: the inference process is used to instruct the measurement model to input and make predictions, and report the inference / prediction results; the monitoring process is used to instruct the measurement model to monitor resources and obtain monitoring results, and report the monitoring results.

[0281] Furthermore, the device also includes a reporting signaling process determination module, specifically used for: when the measurement resource set / measurement resources are a subset of the model monitoring resource set, the reporting signaling process is the model monitoring reporting signaling process.

[0282] Furthermore, the reporting signaling process determination module is specifically used for: when the measurement resource set / measurement resources are not entirely a subset of the model monitoring resource set, the reporting signaling process includes a single reporting process and / or independent reporting processes; a single reporting process is used to instruct the reporting of the inference / prediction results of the measurement resource set / measurement resources and the monitoring results of the model monitoring resource set through a single reporting process; independent reporting processes are used to instruct the reporting of the inference / prediction results of the measurement resource set / measurement resources and the monitoring results of the model monitoring resource set through different reporting processes respectively.

[0283] Furthermore, the reporting content includes one of the following: reporting metrics; reporting events; or reporting both events and metrics based on events.

[0284] Furthermore, the configuration of the reporting timing includes at least one of periodic, semi-static, non-periodic, and event-triggered modes.

[0285] Furthermore, the reported configuration information also includes indication information, which is used to indicate whether the reported configuration information allows the UE to decide / select.

[0286] Furthermore, the model monitoring reference signal resource configuration information is used to indicate the location of the reference signal transmission time-frequency resources and the type of reference signal.

[0287] Furthermore, the location of the reference signal transmission time-frequency resources includes information on the relationship between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set.

[0288] Furthermore, the relationship information between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set includes one of the following: the time-frequency resources of the reference signal occupy part or all of the frequency domain resources + all of the time domain resources of the beam downlink transmission of the prediction resource set; the time-frequency resources of the reference signal occupy part or all of the time domain resources + all of the frequency domain resources of the beam downlink transmission of the prediction resource set; the time-frequency resources of the reference signal occupy part of the frequency domain resources + part of the time domain resources of the beam downlink transmission of the prediction resource set; the time domain resources of the reference signal occupying the time domain resources in the prediction resource set are continuous or discontinuous; the frequency domain resources of the reference signal are continuous or discontinuous resource blocks RB or resource block groups RBG in the prediction resource set.

[0289] Furthermore, the model monitoring reference signal resource configuration information is also used to instruct the model monitoring time and frequency resources.

[0290] Furthermore, the second network element includes at least one of gNB, UE, LMF, and OAM.

[0291] Furthermore, the first message includes at least one of the following: system message, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling.

[0292] Furthermore, the second message includes at least one of system messages, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling.

[0293] Furthermore, the third message includes CSI report messages.

[0294] Each module in the aforementioned model monitoring device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0295] Figure 11 This is a schematic diagram of the structure of an access network device provided in an embodiment of this application. The access network device may include a receiver, a memory, a processor, at least one communication bus, and a transmitter. The communication bus is used to implement communication connections between components. The memory may include high-speed RAM or non-volatile memory (NVM), such as at least one disk storage device. The memory can store various programs to perform various processing functions and implement the method steps of this embodiment. In this embodiment, the transmitter can send a third message to a first network element, the third message carrying tag data and inference / prediction results, or carrying model monitoring results. The receiver can also receive a first message sent by the first network element, the first message containing configuration information related to model monitoring; the configuration information includes model monitoring process configuration information, model monitoring reference signal resource configuration information, and / or reporting configuration information. The receiver can also receive a second message sent by the first network element, the second message carrying a model monitoring reference signal, which is used for resource measurement to obtain tag data for model monitoring. The transmitter and receiver can be integrated to form a transceiver. Both the transmitter and receiver can be coupled to the processor, and can perform receiving or sending actions under the instruction or control of the processor. The processor can also perform message processing actions.

[0296] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0297] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0298] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0299] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0300] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0301] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0302] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0303] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A model monitoring method, characterized in that, The method includes: Receive a first message sent by the first network element, the first message containing configuration information related to model monitoring; The configuration information related to model monitoring includes model monitoring process configuration information, model monitoring reference signal resource configuration information, and / or reporting configuration information; The system receives a second message sent by the first network element, the second message carrying a reference signal, the reference signal being used to perform resource measurement to obtain tag data for model monitoring; A third message is sent to the first network element, the third message carrying the tag data and inference / prediction results, or the third message carrying model monitoring results.

2. The method according to claim 1, characterized in that, The model monitoring process configuration information is used to indicate the model monitoring signaling process and the model monitoring triggering method.

3. The method according to claim 1, characterized in that, The reporting configuration information is used to indicate the model monitoring process, the reporting signaling process, the reporting content, and / or the reporting timing.

4. The method according to claim 3, characterized in that, The model monitoring process includes an inference process and / or a monitoring process: The inference process is used to instruct the measurement model input and make predictions, and report the inference / prediction results; The monitoring process is used to instruct the measurement model to monitor resources, obtain monitoring results, and report the monitoring results.

5. The method according to claim 3, characterized in that, The method further includes: When the measurement resource set / measurement resources are a subset of the model monitoring resource set, the reporting signaling process is the model monitoring reporting signaling process.

6. The method according to claim 3, characterized in that, The method further includes: When the measurement resource set / measurement resources are not entirely a subset of the model monitoring resource set, the reporting signaling process includes a single reporting process and / or an independent reporting process. The single reporting process is used to instruct the reporting of inference / prediction results of the measurement resource set / measurement resources and monitoring results of the model monitoring resource set through a single reporting process; The independent reporting process is used to instruct the reporting of inference / prediction results of the measurement resource set / measurement resources and monitoring results of the model monitoring resource set through different reporting processes.

7. The method according to claim 3, characterized in that, The reported content includes one of the following: Reporting indicators; Report the incident; Report both events and metrics based on events.

8. The method according to claim 3, characterized in that, The configuration of the reporting timing includes at least one of periodic, semi-static, non-periodic, and event-triggered modes.

9. The method according to claim 3, characterized in that, The reported configuration information also includes indication information, which is used to indicate that the reported configuration information allows the UE to decide / select.

10. The method according to claim 1, characterized in that, The model monitoring reference signal resource configuration information is used to indicate the location of the reference signal transmission time-frequency resources and the type of reference signal.

11. The method according to claim 10, characterized in that, The reference signal transmission time-frequency resource location includes: information on the relationship between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set.

12. The method according to claim 11, characterized in that, The relationship information between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set includes one of the following: The time-frequency resources of the reference signal occupy part or all of the frequency domain resources + all time domain resources of the beam downlink transmission of the predicted resource set; The reference signal's time-frequency resource occupancy is part or all of the time-domain resources + all frequency-domain resources of the beam downlink transmission of the predicted resource set; The reference signal's time-frequency resource occupancy is a portion of the frequency domain resources and a portion of the time domain resources of the beam downlink transmission of the predicted resource set; The temporal resource occupancy of the reference signal in the predicted resource set is either continuous or discontinuous. The frequency domain resources of the reference signal are continuous or discontinuous resource blocks RB or resource block groups RBG in the prediction resource set.

13. The method according to claim 10, characterized in that, The model monitoring reference signal resource configuration information is also used to indicate the model monitoring time and frequency resources.

14. The method according to any one of claims 1-13, characterized in that, The first network element includes at least one of gNB, UE, LMF, and OAM.

15. The method according to any one of claims 1-13, characterized in that, The first message includes at least one of system message, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling.

16. The method according to any one of claims 1-13, characterized in that, The second message includes at least one of system messages, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling.

17. The method according to any one of claims 1-13, characterized in that, The third message includes CSI report messages.

18. A model monitoring method, characterized in that, The method includes: Send a first message to the second network element. The first message contains configuration information related to model monitoring. The configuration information related to model monitoring includes model monitoring process configuration information, model monitoring reference signal resource configuration information, and / or reporting configuration information. Send a second message to the second network element, the second message carrying a reference signal, the reference signal being used to perform resource measurement to obtain tag data for model monitoring; The system receives a third message sent by the second network element, the third message carrying the tag data and inference / prediction results, or the third message carrying model monitoring results.

19. The method according to claim 18, characterized in that, The method further includes: calculating the model monitoring results when the second network element sends tag data and inference / prediction results.

20. The method according to claim 18, characterized in that, The model monitoring process configuration information is used to indicate the model monitoring signaling process and the model monitoring triggering method.

21. The method according to claim 18, characterized in that, The reporting configuration information is used to indicate the model monitoring process, the reporting signaling process, the reporting content, and / or the reporting timing.

22. The method according to claim 21, characterized in that, The model monitoring process includes an inference process and / or a monitoring process: The inference process is used to instruct the measurement model input and make predictions, and report the inference / prediction results; The monitoring process is used to instruct the measurement model to monitor resources, obtain monitoring results, and report the monitoring results.

23. The method according to claim 21, characterized in that, The method further includes: When the measurement resource set / measurement resources are a subset of the model monitoring resource set, the reporting signaling process is the model monitoring reporting signaling process.

24. The method according to claim 21, characterized in that, The method further includes: When the measurement resource set / measurement resources are not entirely a subset of the model monitoring resource set, the reporting signaling process includes a single reporting process and / or an independent reporting process. The single reporting process is used to instruct the reporting of inference / prediction results of the measurement resource set / measurement resources and monitoring results of the model monitoring resource set through a single reporting process; The independent reporting process is used to instruct the reporting of inference / prediction results of the measurement resource set / measurement resources and monitoring results of the model monitoring resource set through different reporting processes.

25. The method according to claim 21, characterized in that, The reported content includes one of the following: Reporting indicators; Report the incident; Report both events and metrics based on events.

26. The method according to claim 21, characterized in that, The configuration of the reporting timing includes at least one of periodic, semi-static, non-periodic, and event-triggered modes.

27. The method according to claim 21, characterized in that, The reported configuration information also includes indication information, which is used to indicate that the reported configuration information allows the UE to decide / select.

28. The method according to claim 21, characterized in that, The model monitoring reference signal resource configuration information is used to indicate the location of the reference signal transmission time-frequency resources and the type of reference signal.

29. The method according to claim 28, characterized in that, The reference signal transmission time-frequency resource location includes: information on the relationship between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set.

30. The method according to claim 29, characterized in that, The relationship information between the time-frequency resources of the reference signal and the time-frequency resources of the prediction resource set includes one of the following: The time-frequency resources of the reference signal occupy part or all of the frequency domain resources + all time domain resources of the beam downlink transmission of the predicted resource set; The reference signal's time-frequency resource occupancy is part or all of the time-domain resources + all frequency-domain resources of the beam downlink transmission of the predicted resource set; The reference signal's time-frequency resource occupancy is a portion of the frequency domain resources and a portion of the time domain resources of the beam downlink transmission of the predicted resource set; The temporal resource occupancy of the reference signal in the predicted resource set is either continuous or discontinuous. The frequency domain resources of the reference signal are continuous or discontinuous resource blocks RB or resource block groups RBG in the prediction resource set.

31. The method according to claim 28, characterized in that, The model monitoring reference signal resource configuration information is also used to indicate the model monitoring time and frequency resources.

32. The method according to any one of claims 18-31, characterized in that, The second network element includes at least one of gNB, UE, LMF, and OAM.

33. The method according to any one of claims 18-31, characterized in that, The first message includes at least one of system message, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling.

34. The method according to any one of claims 18-31, characterized in that, The second message includes at least one of system messages, RRC signaling, MAC CE signaling, DCI signaling, and NAS signaling.

35. The method according to any one of claims 18-31, characterized in that, The third message includes CSI report messages.

36. A model monitoring device, characterized in that, The device includes: The first receiving module is used to receive a first message sent by a first network element. The first message contains configuration information related to model monitoring. The configuration information related to model monitoring includes model monitoring process configuration information, model monitoring reference signal resource configuration information, and / or reporting configuration information. The second receiving module is used to receive a second message sent by the first network element. The second message carries a reference signal, which is used to perform resource measurement to obtain tag data for model monitoring. The sending module is used to send a third message to the first network element, wherein the third message carries the tag data and inference / prediction results, or the third message carries model monitoring results.

37. A model monitoring device, characterized in that, The device includes: The first sending module is used to send a first message to the second network element. The first message contains configuration information related to model monitoring. The configuration information related to model monitoring includes model monitoring process configuration information, model monitoring reference signal resource configuration information, and / or reporting configuration information. The second sending module is used to send a second message to the second network element. The second message carries a reference signal, which is used to perform resource measurement to obtain tag data for model monitoring. The receiving module is used to receive a third message sent by the second network element, wherein the third message carries the tag data and inference / prediction results, or the third message carries model monitoring results.

38. A communication device, characterized in that, include: Transmitter and receiver; The receiver is used to receive a first message sent by the first network element, the first message containing configuration information related to model monitoring; The configuration information related to model monitoring includes model monitoring process configuration information, model monitoring reference signal resource configuration information, and / or reporting configuration information; The receiver is configured to receive a second message sent by the first network element, the second message carrying a reference signal, the reference signal being used to perform resource measurement to obtain tag data for model monitoring; The transmitter is used to send a third message to the first network element, the third message carrying the tag data and inference / prediction results, or the third message carrying model monitoring results.

39. A communication device, characterized in that, include: Transmitter and receiver; The transmitter is used to send a first message to the second network element, the first message containing configuration information related to model monitoring; The configuration information related to model monitoring includes model monitoring process configuration information, model monitoring reference signal resource configuration information, and / or reporting configuration information; The transmitter is used to send a second message to the second network element, the second message carrying a reference signal, the reference signal being used to perform resource measurement to obtain tag data for model monitoring; The receiver is used to receive a third message sent by the second network element, the third message carrying the tag data and inference / prediction results, or the third message carrying model monitoring results.

40. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 35.

Images