Ai / ML function monitoring method and apparatus

Abstract

The embodiments of the present application provide an AI / ML function monitoring method and apparatus. The method comprises: a first device initiating AI / ML function monitoring; and when AI / ML function monitoring is performed, the first device and a second device exchanging AI / ML function monitoring-related information.

Description

Methods and devices for monitoring AI / ML functions Technical Field

[0001] This application relates to the field of communication technology. Background Technology

[0002] With the commercialization of 5G (fifth-generation mobile communication technology), and especially the large-scale deployment of the industrial internet, the application of artificial intelligence (AI) technology in wireless communication has sprung up like mushrooms after rain. Among them, AI / ML model management based on deep learning (ML) or reinforcement learning technologies is particularly important in various wireless AI applications. Due to the overall design of the 3GPP (3rd Generation Partnership Project) protocol, for AI / ML models deployed on the terminal side (UE), model management requires the assistance of network-side (NW) decision-making or auxiliary information, and cannot be solved by the technology itself.

[0003] For one of the AI / ML wireless use cases, there are two ways to monitor the wireless positioning of AI / ML functions on the terminal side: one is performed on the terminal side, and the other is performed on the network side (LMF). The two sides need to exchange information to complete the monitoring process.

[0004] It should be noted that the above introduction to the technical background is only for the purpose of providing a clear and complete explanation of the technical solutions of this application and facilitating understanding by those skilled in the art. It should not be assumed that these technical solutions are known to those skilled in the art simply because they have been described in the background section of this application.

[0005] Summary of the Invention

[0006] The inventors discovered that existing 3GPP protocols lack mechanisms and frameworks related to AI / ML lifecycle management. Similarly, wireless positioning for AI / ML functions requires enhancements to existing standards.

[0007] To address at least one of the above-mentioned problems or other similar issues, embodiments of this application provide a method and apparatus for monitoring AI / ML functions.

[0008] According to one aspect of the embodiments of this application, a method for monitoring AI / ML functions is provided, the method comprising:

[0009] The first device initiates AI / ML function monitoring;

[0010] During AI / ML function monitoring, the first and second devices interact with each other regarding AI / ML function monitoring information.

[0011] According to another aspect of the embodiments of this application, a monitoring device for AI / ML functions is provided, configured in a first device, the device comprising:

[0012] The processing unit initiates AI / ML function monitoring;

[0013] The interaction unit interacts with the second device to exchange information related to AI / ML function monitoring during AI / ML function monitoring.

[0014] One of the beneficial effects of the embodiments of this application is that after the terminal device or network device initiates AI / ML function monitoring, the terminal device and network device interact with AI / ML function monitoring-related information during the AI / ML function monitoring, which solves the problem that the current standard does not standardize the monitoring of AI / ML functions, especially wireless positioning use cases, between the core network device (LMF) and the terminal device (UE).

[0015] Specific embodiments of this application are disclosed in detail with reference to the following description and accompanying drawings, indicating how the principles of this application can be adopted. It should be understood that the embodiments of this application are not limited in scope. Within the spirit and scope of the appended claims, embodiments of this application include many changes, modifications, and equivalents.

[0016] Features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, combined with or replacing features in other embodiments.

[0017] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, whole, step, or component, but does not exclude the presence or addition of one or more other features, wholes, steps, or components. Attached Figure Description

[0018] The elements and features described in one drawing or embodiment of this application may be combined with elements and features shown in one or more other drawings or embodiments. Furthermore, in the drawings, similar reference numerals denote corresponding parts in several drawings and can be used to indicate corresponding parts used in more than one embodiment.

[0019] The accompanying drawings, which form part of the specification, are used to provide a further understanding of the embodiments of this application and illustrate the implementation methods of this application, together with the textual description, to explain the principles of this application. Obviously, the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without creative effort. In the drawings:

[0020] Figure 1 is a schematic diagram of an application scenario of an embodiment of this application;

[0021] Figure 2 is a schematic diagram of a monitoring method for AI / ML functions according to an embodiment of this application;

[0022] Figure 3 is a schematic diagram of information interaction between network devices and terminal devices;

[0023] Figure 4 is another schematic diagram of information interaction between network devices and terminal devices;

[0024] Figure 5 is another schematic diagram of information interaction between network devices and terminal devices;

[0025] Figure 6 is another schematic diagram of information interaction between network devices and terminal devices;

[0026] Figure 7 is another schematic diagram of information interaction between network devices and terminal devices;

[0027] Figure 8 is another schematic diagram of information interaction between network devices and terminal devices;

[0028] Figure 9 is another schematic diagram of information interaction between network devices and terminal devices;

[0029] Figure 10 is a schematic diagram of a monitoring device for AI / ML functions according to an embodiment of this application;

[0030] Figure 11 is another schematic diagram of the monitoring device for AI / ML function according to an embodiment of this application;

[0031] Figure 12 is a schematic block diagram of the system configuration of a terminal device according to an embodiment of this application;

[0032] Figure 13 is a schematic block diagram of the system configuration of a network device according to an embodiment of this application. Detailed Implementation

[0033] Referring to the accompanying drawings, the foregoing and other features of this application will become apparent from the following description. Specific embodiments of this application are specifically disclosed in the description and drawings, illustrating partial implementations in which the principles of this application may be employed. It should be understood that this application is not limited to the described embodiments; rather, it includes all modifications, variations, and equivalents falling within the scope of the appended claims.

[0034] In the embodiments of this application, the terms "first," "second," etc., are used to distinguish different elements by name, but do not indicate the spatial arrangement or chronological order of these elements, and these elements should not be limited by these terms. The term "and / or" includes any one or more of the terms listed in association and all combinations thereof. The terms "comprising," "including," "having," etc., refer to the presence of the stated features, elements, components, or assemblies, but do not exclude the presence or addition of one or more other features, elements, components, or assemblies.

[0035] In the embodiments of this application, the singular forms "a," "the," etc., including the plural forms, should be broadly understood as "a kind" or "a class" rather than limited to the meaning of "an." Furthermore, the term "the" should be understood to include both the singular and plural forms, unless the context explicitly indicates otherwise. Additionally, the term "according to" should be understood as "at least partially based on…," and the term "based on" should be understood as "at least partially based on…," unless the context explicitly indicates otherwise.

[0036] In the embodiments of this application, the term "communication network" or "wireless communication network" may refer to a network that conforms to any of the following communication standards, such as Long Term Evolution (LTE), LTE-Advanced (LTE-A), Wideband Code Division Multiple Access (WCDMA), High-Speed ​​Packet Access (HSPA), etc.

[0037] Communication between devices in a communication system can be carried out according to communication protocols at any stage, including but not limited to the following communication protocols: 1G (generation), 2G, 2.5G, 2.75G, 3G, 4G, 4.5G and future 5G, New Radio (NR), etc., and / or other currently known or future communication protocols.

[0038] In the embodiments of this application, the term "network device" refers, for example, to a device in a communication system that connects a terminal device to a communication network and provides services to that terminal device. Network devices may include, but are not limited to, the following devices: base station (BS), core network (CN), operation administration and maintenance (OAM), over-the-top (OTT) server, access point (AP), transmission reception point (TRP), broadcast transmitter, mobile management entity (MME), gateway, server, radio network controller (RNC), base station controller (BSC), etc.

[0039] Base stations can include, but are not limited to: NodeBs (or NBs), evolved NodeBs (eNodeBs or eNBs), and 5G base stations (gNBs), IAB hosts, etc. They can also include Remote Radio Heads (RRHs), Remote Radio Units (RRUs), relays, or low-power nodes (e.g., femto, pico, etc.). The term "base station" can include some or all of their functions, and each base station can provide communication coverage to a specific geographic area. The term "cell" can refer to a base station and / or its coverage area, depending on the context in which the term is used.

[0040] The core network may include, but is not limited to: MSC (Mobile Service Center), SGSN (Serving GPRS Support Node), GGSN (Gateway GPRS Support Node), MME (Mobile Management Entity), SGW (Serving Gateway), PGW (PDN Gateway), and 5GC (5G Core Network). Furthermore, the term "core network" can include some or all of their functions.

[0041] In the embodiments of this application, the term "user equipment" (UE) refers to a device that accesses a communication network and receives network services through a network device, and can also be called "terminal equipment" (TE). Terminal equipment can be fixed or mobile, and can also be called a mobile station (MS), terminal, user, subscriber station (SS), access terminal (AT), station, etc.

[0042] Terminal devices may include, but are not limited to, the following devices: cellular phones, personal digital assistants (PDAs), wireless modems, wireless communication devices, handheld devices, machine-type communication devices, laptops, cordless phones, smartphones, smartwatches, digital cameras, etc.

[0043] For example, in scenarios such as the Internet of Things (IoT), terminal devices can also be machines or devices for monitoring or measurement, such as including but not limited to: machine-type communication (MTC) terminals, vehicle communication terminals, device-to-device (D2D) terminals, machine-to-machine (M2M) terminals, and so on.

[0044] The following examples illustrate the scenarios of embodiments of this application, but this application is not limited thereto.

[0045] Figure 1 is a schematic diagram of a communication system according to an embodiment of this application, illustrating the case of a terminal device and a network device as examples. As shown in Figure 1, the communication system 100 may include a network device 101 and terminal devices 102 and 103. For simplicity, Figure 1 only illustrates the case of two terminal devices and one network device, but the embodiments of this application are not limited to this.

[0046] In this embodiment of the application, network device 101 and terminal devices 102 and 103 can transmit existing services or services that can be implemented in the future. For example, these services may include, but are not limited to: enhanced mobile broadband (eMBB), massive machine-type communication (mMTC), and ultra-reliable and low-latency communication (URLLC), etc.

[0047] It is worth noting that Figure 1 shows that terminal devices 102 and 103 are within the coverage area of ​​network device 101, but this application is not limited to this. Terminal devices 102 and 103 may not be within the coverage area of ​​network device 101.

[0048] In the embodiments of this application, network device 101 may be, for example, a gNB, or a core network entity (e.g., a Location Management Function (LMF) or an Access and Mobility Management Function (AMF), or a higher-layer network entity (e.g., Operation Administration and Maintenance (OAM), or a network-side OTT server (OTT-server), or may be a part of the functions or entities of any of the above devices.

[0049] In the embodiments of this application, higher-layer signaling may be, for example, Radio Resource Control (RRC) signaling; for example, referred to as an RRC message, including MIB (Master Information Block), system information, and dedicated RRC messages; or referred to as an RRC IE (RRC information element). Higher-layer signaling may also be, for example, MAC (Medium Access Control) signaling; or referred to as MAC CE (MAC control element). Higher-layer signaling may also be LPP (LTE Positioning Protocol) signaling; or referred to as LPP IE (LPP information element). However, this application is not limited to these.

[0050] In the embodiments of this application, the AI / ML function lifecycle management involves, but is not limited to, the following: function and / or model switching, function and / or model selection, function and / or model rollback, function and / or model activation, function and / or model deactivation, and triggering of data collection related to functions and / or models. Furthermore, the embodiments of this application are applicable to scenarios where AI / ML models are deployed on the terminal side, such as wireless positioning (AI / ML Positioning Accuracy Enhancement) terminal side model 1 / 2a, etc. In the following description, unless otherwise specified, "when," "if," and "under certain circumstances" have the same meaning and can be used interchangeably; "supervise" and "monitor" have the same meaning and can be used interchangeably; "trigger" and "initiate" have the same meaning and can be used interchangeably.

[0051] The embodiments of this application will now be described with reference to the accompanying drawings.

[0052] First aspect of the embodiments

[0053] This application provides a method for monitoring AI / ML functions, described from the perspective of a first device. The first device can be a terminal device, such as terminal devices 102 and 103 as shown in FIG1. ​​Correspondingly, the second device is a network device, such as network device 101 as shown in FIG1. ​​However, this application is not limited to this; the first device can also be a network device, such as network device 101 as shown in FIG1, and the second device can be a terminal device, such as terminal devices 102 and 103 as shown in FIG1, depending on the implementation scenario.

[0054] In this embodiment of the application, the network device 101 may be at least one of the following:

[0055] TRP;

[0056] gNB;

[0057] NG-RAN (Next Generation Radio Access Network) network elements;

[0058] Core network elements, such as LMF / AMF / UPF (User Plane Function), etc.

[0059] OAM network element;

[0060] OTT server.

[0061] Figure 2 is a schematic diagram of a monitoring method for AI / ML functions according to an embodiment of this application. As shown in Figure 2, the method includes:

[0062] 210, the first device initiates AI / ML function monitoring;

[0063] 220. During AI / ML function monitoring, the first device and the second device interact with each other regarding AI / ML function monitoring information.

[0064] It is worth noting that Figure 2 above is only an illustrative description of the embodiments of this application, but this application is not limited thereto. For example, the execution order between various operations can be appropriately adjusted, and other operations can be added or some operations can be removed. Those skilled in the art can make appropriate modifications based on the above content, and are not limited to the description in Figure 2 above.

[0065] In the above embodiments, after the first device initiates AI / ML function monitoring, when the first device and the second device interact with each other regarding AI / ML function monitoring information, the terminal device can calculate monitoring metrics and / or make preliminary decisions, and report the aforementioned metrics and / or preliminary decision information to the network device so that the network device can make a final decision.

[0066] According to the above embodiments, after the first and second devices exchange information related to AI / ML function monitoring, the network device can make a final decision. This solves the problem that the current standard does not standardize the monitoring of AI / ML functions, especially for wireless positioning use cases, between core network equipment (LMF) and terminal equipment (UE).

[0067] In this embodiment of the application, in operation 210, AI / ML function monitoring can be initiated by a terminal device, that is, the first device is a terminal device. This application is not limited to this, and AI / ML function monitoring can also be sent by a network device, that is, the first device is a network device. The following description takes the AI / ML function monitoring initiated by a terminal device as an example.

[0068] Figure 3 is a schematic diagram of information interaction between network devices and terminal devices, illustrating a scenario where a terminal device initiates AI / ML function monitoring. In the example in Figure 3, the first device is the terminal device, and the second device is the network device.

[0069] As shown in Figure 3, in step 310, terminal devices 102 and 103 initiate AI / ML function monitoring; in step 320, during AI / ML function monitoring, network device 101 and terminal devices 102 and 103 exchange information related to AI / ML function monitoring. Therefore, terminal devices 102 and 103 can exchange relevant information with network device 101 during AI / ML function monitoring, thus solving the problem that the current standard does not standardize the monitoring of AI / ML functions, especially for wireless positioning use cases, between core network equipment (LMF) and terminal devices (UE).

[0070] It is worth noting that Figure 3 above is only an illustrative description of the embodiments of this application, but this application is not limited thereto. For example, the execution order between various operations can be appropriately adjusted, and other operations can be added or some operations can be removed. Those skilled in the art can make appropriate modifications based on the above content, and are not limited to the description in Figure 3 above.

[0071] In the above embodiments, terminal devices 102 and 103 can initiate AI / ML function monitoring when a first condition is met, and the first condition includes, but is not limited to, at least one of the following:

[0072] When the AI / ML function monitoring is scheduled (the terminal initiates monitoring periodically / cyclically);

[0073] The current state of the terminal device does not meet the requirements of the current AI / ML functions;

[0074] The AI / ML functions of the terminal devices have been updated;

[0075] The models under one or more functions in the AI / ML capabilities of the terminal device have been updated.

[0076] The aforementioned AI / ML function monitoring opportunities include, for example, when the terminal device detects that the AI / ML function performance is poor, such as the positioning accuracy not meeting the current service requirements, or when the terminal device performs periodic function monitoring, etc. In such cases, the terminal device can initiate AI / ML function monitoring periodically or periodically.

[0077] If the current situation does not meet the requirements of the current AI / ML function, such as insufficient power, insufficient storage, excessive speed of the terminal device, or significant changes in the environment or area where the terminal device is located, the terminal device can initiate AI / ML function monitoring.

[0078] If the aforementioned AI / ML functions are updated, such as by acquiring new AI / ML functions for the terminal device or by the terminal device no longer supporting older AI / ML functions, the terminal device can initiate AI / ML function monitoring. The definition of this function can be agreed upon in a predefined manner, and this application does not impose any restrictions on this.

[0079] If the model under one or more of the aforementioned AI / ML functions has been updated, such as by retraining the model for the terminal device and / or changing the structure of the model, the terminal device can initiate AI / ML function monitoring.

[0080] In the above embodiments, after the terminal device initiates AI / ML function monitoring, it can also send information (referred to as first information) to the network device, as shown in operation 315 in Figure 3, to request the network device to send relevant data and / or information.

[0081] For example, the terminal device sends first information to the network device, the first information being used to request the network device to send at least one of the following:

[0082] AI / ML function monitoring related data and / or information related to said data (hereinafter referred to as data-related information);

[0083] Information other than data related to AI / ML function monitoring;

[0084] The first device wants to obtain the configuration required for AI / ML function monitoring.

[0085] The data related to the aforementioned AI / ML function monitoring includes, for example, ground truth data, and related data information such as the timestamp information corresponding to the ground truth data. When the network device receives the aforementioned first information, it can send the aforementioned ground truth data and / or the timestamp corresponding to the ground truth data to the terminal device according to the request of the first information. Thus, the terminal device can perform AI / ML function monitoring based on this.

[0086] Other information besides the data related to AI / ML function monitoring mentioned above, such as metrics information required for AI / ML function and / or model supervision, can be sent to the terminal device by the network device after receiving the first information. Thus, the terminal device can monitor the AI / ML function / model accordingly.

[0087] The configuration required for AI / ML function monitoring mentioned above is, for example, positioning reference signal configuration information. When the network device receives the first information mentioned above, it can send the positioning reference signal configuration information to the terminal device according to the request of the first information. Thus, the terminal device can measure the positioning reference signal and / or report the measurement results.

[0088] This application does not restrict the method of sending the first information. The terminal device may send the first information according to the instructions or requirements of the network device, or it may send the first information on the uplink data channel / signal and / or the uplink control channel / signal.

[0089] The above explanation uses a terminal device as the first device to initiate AI / ML function monitoring. This application is not limited to this; a network device can also initiate AI / ML function monitoring. This application does not limit the method by which a network device initiates AI / ML function monitoring.

[0090] In the above embodiments, after initiating AI / ML function monitoring, the network device can also send relevant information to the terminal device to inform the terminal device that it has initiated AI / ML function monitoring. Furthermore, the network device can also send AI / ML function monitoring-related data and information to the terminal device. In addition, the network device can instruct the terminal device to report AI / ML function monitoring-related data and information, and / or send configuration information to the terminal device to enable the terminal device to perform AI / ML function monitoring-related measurements, etc. For details, please refer to the relevant technologies, which will not be elaborated here.

[0091] In this embodiment of the application, during operation 220, AI / ML function monitoring can be performed by a first device or by a second device. That is, the interaction of AI / ML function monitoring related information between the first device and the second device can be performed when the first device performs AI / ML function monitoring or when the second device performs AI / ML function monitoring.

[0092] In the following description, we will use the example of a first device being a terminal device and a second device being a network device.

[0093] In some embodiments, AI / ML function monitoring is performed by a first device, that is, by a terminal device.

[0094] Figure 4 is another schematic diagram of information interaction between network devices and terminal devices, showing the situation of terminal devices monitoring AI / ML functions.

[0095] In the above embodiments, when the terminal device performs AI / ML function monitoring, it needs the network device to provide the grand truth. Since the grand truth and measurement information are provided by different entities (the grand truth is provided by the network device and the measurement information is provided by the terminal device), signaling interaction is required to ensure that the two can match.

[0096] For example, as shown in Figure 4, in operation 410, the terminal device receives a Position Reference Signal (PRS) and truth information periodically sent by the network device. The transmission period of the PRS and the transmission period of the truth information are synchronized or spaced apart by a predetermined value (referred to as the first value). That is, if the PRS is transmitted periodically, the network device is also configured to transmit the truth information periodically, and the periods of the two are synchronized or spaced apart by an offset (the first value). Thus, the terminal device can obtain the truth information and perform AI / ML function monitoring.

[0097] In the above example, the interval between the PRS transmission period and the truth information transmission period (i.e., the first value mentioned above) is determined by the network device, and this application does not restrict its specific value.

[0098] For example, as shown in Figure 4, in operation 410', the terminal device receives the timestamp information corresponding to the truth information and / or the applicable information corresponding to the truth information sent by the network device, such as the validity area information. Thus, the terminal device can also obtain truth information and perform AI / ML function monitoring.

[0099] In other embodiments, AI / ML function monitoring is performed by a second device, namely, a network device.

[0100] Figure 5 is another schematic diagram of information interaction between network devices and terminal devices, showing the situation of network devices monitoring AI / ML functions.

[0101] In the above embodiments, when the network device (second device) performs AI / ML function monitoring, since it has truth information but not measurement information, the network device and the terminal device still need to interact with AI / ML function monitoring related information in order to perform AI / ML function monitoring.

[0102] For example, as shown in Figure 5, in operation 510, the terminal device sends the measurement results to the network device. Thus, the network device can perform AI / ML function monitoring based on its known truth information and the measurement results provided by the terminal device.

[0103] In the above example, the network device can also be configured to send timestamp information corresponding to the measurement results to the terminal device. For example, as shown in Figure 5, in operation 520, the terminal device sends the timestamp information corresponding to the measurement results to the network device. Thus, the network device can confirm, listen to, or receive the measurement results based on this timestamp information.

[0104] In this embodiment of the application, when the terminal device provides decision information to the network device, the terminal device and the network device need to interact on AI / ML function monitoring related information.

[0105] In some embodiments, the terminal device provides decision information based on the configuration of the network device.

[0106] Figure 6 is another schematic diagram of information interaction between network devices and terminal devices, showing the situation where the terminal device provides decision information based on the configuration of the network device.

[0107] For example, as shown in Figure 6, in operation 610, the terminal device may receive first configuration information sent by the network device, which indicates at least one of the following:

[0108] Instruct terminal devices to report the basis for decision-making;

[0109] Instructing terminal devices to report decision-making information;

[0110] Instructs the terminal device to report the content of the decision information.

[0111] The decision-making criteria mentioned above include, for example, making a decision that the function succeeds when the positioning accuracy is greater than a certain threshold, and making a decision that the function fails when the positioning accuracy is less than a certain threshold, and so on.

[0112] The requirements for the aforementioned decision-making information include, for example, reporting within a specified timeframe, etc.

[0113] The content of the aforementioned decision-making information includes, for example, attaching reason information when reporting, etc.

[0114] The terminal device makes a decision based on the aforementioned first configuration information, and can then feed back the corresponding information to the network device.

[0115] For example, as shown in Figure 6, in operation 620, after receiving the first configuration information, the terminal device sends second information to the network device, requesting the network device to send relevant information through the second information.

[0116] In the example above, after receiving the second information, the network device may, but is not limited to, perform at least one of the following actions (operation 630):

[0117] Send the second configuration information to the terminal device according to the request of the second information;

[0118] Send data and / or information related to the data to the terminal device according to the request of the second information;

[0119] Send capability interaction information and / or function interaction information to the terminal device according to the request of the second information;

[0120] Send instruction information to the terminal device, which is used to instruct the terminal device to continue, modify or terminate the currently initiated AI / ML function monitoring process;

[0121] Stop the action.

[0122] The aforementioned second configuration information is used to instruct the terminal device to perform relevant measurements and / or report measurement results for AI / ML function monitoring decisions, such as PRS measurement and / or reporting configuration, SRS measurement configuration information, etc. The terminal device can perform PRS measurement and / or reporting, SRS signal transmission, and other operations based on this configuration.

[0123] The above data may be, for example, truth information, and the related information may be, for example, the timestamp information corresponding to the truth information, etc.

[0124] The aforementioned capability / functional interaction information may include, for example, at least the terminal device's ability to support traditional and / or AI / ML positioning methods, its ability to support reference signals, its ability to support positioning measurement methods, its ability to provide truth data and related information, and functional information related to AI / ML positioning, etc. Furthermore, the aforementioned capability / functional interaction information may also include the terminal device's computing power, storage capacity, and other capabilities.

[0125] This application does not restrict the method of sending the first configuration information and the second configuration information. For example, the first configuration information and the second configuration information can be sent through higher-level signaling, such as the aforementioned RRC / LPP / MAC CE, or through optional information elements (IE) or information fields in the signaling, or through new signaling or new IE, and so on.

[0126] This application does not restrict the method of sending the second information. For example, the terminal device may send the second information according to the instructions or requirements of the network device, such as through the uplink data channel / signal and / or the uplink control channel / signal, etc.

[0127] In other embodiments, the terminal device provides decision information based on a second condition.

[0128] For example, the terminal device makes a decision when a second condition is met. This second condition includes, but is not limited to, at least one of the following:

[0129] The terminal equipment has highly accurate monitoring data or information;

[0130] The terminal device receives configuration information from the network device and is required to make a decision.

[0131] For example, when a terminal device has highly accurate monitoring data or information, it provides decision information; or, when a terminal device receives configuration information from a network device (such as the aforementioned first or second configuration information), and this configuration information requires the terminal device to make a decision, the terminal device provides decision information.

[0132] In some other embodiments, the terminal device provides decision information based on the first event.

[0133] For example, the terminal device makes a decision when a first event is met. This first event includes, but is not limited to, at least one of the following:

[0134] The first timer expired and the first configuration information was not received;

[0135] The first counter reaches the first threshold and no first configuration information is received;

[0136] The second timer expired and the positioning performance of the AI / ML function was insufficient.

[0137] The second counter reaches the second threshold and the positioning performance of the AI / ML function is insufficient.

[0138] The current conditions of the terminal device cannot meet the requirements of AI / ML functions.

[0139] For example, the terminal device maintains a timer (referred to as the first timer) related to providing decision information. When the first timer expires and no first configuration information is received, the terminal device provides decision information. This application does not restrict the design principles and maintenance methods of the first timer.

[0140] For example, the terminal device maintains a counter (referred to as the first counter) related to providing decision information. When the count of the first counter reaches a certain threshold (referred to as the first threshold) and no first configuration information is received, the terminal device provides decision information. This application does not restrict the design principles and maintenance methods of the first counter.

[0141] For example, the terminal device maintains a timer (referred to as a second timer) related to providing decision information. When the second timer expires and the positioning performance of the AI / ML function is insufficient, the terminal device provides decision information. This application does not restrict the design principles and maintenance methods of the second timer.

[0142] For example, the terminal device maintains a counter (referred to as a second counter) related to providing decision information. When the count of the second counter reaches a certain threshold (referred to as a first threshold) and the positioning performance of the AI / ML function is insufficient, the terminal device provides decision information. This application does not restrict the design principles and maintenance methods of the second counter.

[0143] For example, when the current conditions of the terminal device cannot meet the AI / ML functions, it outputs decision information.

[0144] Figure 7 is another schematic diagram of information interaction between network devices and terminal devices, showing the situation where the terminal device makes a decision based on the second condition or the first event.

[0145] In the above embodiments, as shown in FIG7, after the terminal device makes a decision based on the second condition or the first event, in operation 710, the terminal device may also send third information to the network device. The third information is used to notify the network device of the content of the above decision and / or to request the network device to confirm the content of the above decision.

[0146] This application does not restrict the method of sending third information. For example, the terminal device may send third information according to the instructions or requirements of the network device, such as through the uplink data channel / signal and / or the uplink control channel / signal, etc.

[0147] In this embodiment of the application, when the network device performs AI / ML function monitoring, it may need the terminal device to provide it with some relevant information. Therefore, the terminal device and the network device need to interact with each other regarding AI / ML function monitoring information.

[0148] In some embodiments, when a network device or a terminal device initiates AI / ML function monitoring (e.g., model monitoring), the network device can send information (referred to as fourth information) to the terminal device to instruct the terminal device to report model-related information, such as model input type, etc.

[0149] Figure 8 is a schematic diagram of information interaction between network devices and terminal devices according to the above embodiments.

[0150] In the above embodiment, as shown in FIG8, in operation 810, the terminal device can receive fourth information sent by the network device, which is used to instruct the terminal device to report model-related information.

[0151] In the above embodiments, the terminal device can send model-related information to the network device according to the instruction of the fourth information (operation 820), so that the network device can initiate measurement configuration to the positioning reference unit (PRU) and collect measurement results according to the model-related information, and send the measurement results to the terminal device (operation 830), so that the terminal device can perform model inference based on the measurement results.

[0152] In the above embodiments, the terminal device may also send the model inference result to the network device (operation 840), so that the network device can make a decision based on the model inference result and send decision information to the terminal device (operation 850).

[0153] This application does not restrict the method of sending the fourth information. For example, the fourth information can be sent through higher-level signaling, such as the aforementioned RRC / LPP / MAC CE, or through optional information elements (IE) or information fields in the signaling, or through new signaling or new IE, and so on.

[0154] In some embodiments, when a terminal device initiates AI / ML function monitoring (e.g., model monitoring), it may also request the network device to send PRU information from the network device.

[0155] Figure 9 is a schematic diagram of information interaction between network devices and terminal devices according to the above embodiments.

[0156] For example, as shown in Figure 9, in operation 910, the terminal device sends fifth information to the network device. This fifth information is used by the terminal device to request PRU information from the network device when initiating AI / ML function monitoring. This application does not limit the method of sending the fifth information. For example, the terminal device can send the fifth information according to the instructions or requirements of the network device, such as through the uplink data channel / signal and / or the uplink control channel / signal, etc.

[0157] In the above example, if the network device has a PRU, the network device may, but is not limited to, perform at least one of the following processes (operation 920):

[0158] Send the sixth message to the terminal device, requesting the terminal device to report model-related information;

[0159] Send a list of measurements supported by PRU to the terminal device, such as RSRP (Reference Signal Receiving Power / Reference Signal Received Power), RSTD (Reference Signal Time Difference), RTOA (Relative Time of Arrival), Multi-RTT (Multi-Round-Trip-Time), CIR (Channel Impulse Response), PDP (Power Delay Profile), DP (Delay Profile), RSRPP (Reference Signal Receiving Path Power), etc.

[0160] The seventh message is sent to the terminal device, which is then instructed to perform a measurement and report the result.

[0161] This application does not restrict the method of sending the sixth and seventh information. For example, the sixth and seventh information can be sent through higher-level signaling, such as the aforementioned RRC / LPP / MAC CE, through optional information elements (IE) or information fields in the signaling, or through new signaling or new IE, and so on.

[0162] In the above example, if the network device does not have a PRU or, although it has a PRU, the available PRUs are insufficient, the network device can also send an eighth message (operation 920) to the terminal device to instruct the terminal device to perform AI / ML function monitoring actions and / or to instruct the terminal device to perform additional information exchange for AI / ML function monitoring. The method of sending the eighth message is similar to that of the sixth and seventh messages, and will not be described again here.

[0163] In some possible implementations, the eighth message indicates at least one of the following (not limited to):

[0164] The terminal device lacks sufficient PRU information;

[0165] Instruct the terminal device to terminate the current AI / ML function monitoring process;

[0166] Instruct the terminal device to pause the current AI / ML function monitoring process;

[0167] Instructing terminal devices to interact with AI / ML capabilities and / or AI / ML function-related information (details have been explained above);

[0168] Instruct the terminal device to roll back and / or deactivate the AI / ML function.

[0169] In some possible implementations, the terminal device may perform at least one of the following processes (not limited to) according to the instructions of the eighth information mentioned above:

[0170] Terminate or suspend the current AI / ML function monitoring process;

[0171] Perform AI / ML function rollback and / or AI / ML function deactivation;

[0172] Send the ninth message (Operation 930) to the network device and interact with AI / ML capabilities and / or AI / ML function-related information through the ninth message;

[0173] Send the ninth message (Operation 930) to the network device to request the interaction of information related to the function monitoring method without using PRU.

[0174] This application does not restrict the method of sending the ninth information. For example, the terminal device may send the ninth information according to the instructions or requirements of the network device, such as through the uplink data channel / signal and / or the uplink control channel / signal, etc.

[0175] The above embodiments are merely illustrative examples of embodiments of this application, but this application is not limited thereto, and appropriate modifications can be made based on the above embodiments. For example, the above embodiments can be used alone, or one or more of the above embodiments can be combined.

[0176] According to the embodiments of this application, after the terminal device or network device initiates AI / ML function monitoring, the terminal device and network device exchange information related to AI / ML function monitoring during the AI / ML function monitoring process. This solves the problem that the current standard does not standardize the monitoring of AI / ML functions, especially wireless positioning use cases, between the core network device (LMF) and the terminal device (UE).

[0177] Second aspect of the embodiments

[0178] This application provides an AI / ML function monitoring device. This device can be, for example, a terminal device, a network device, or one or more components or parts configured on a terminal device or network device. For ease of explanation, the device configured with the AI / ML function monitoring device of this application is referred to as a first device, and the device interacting with it is referred to as a second device. Since the function of this device in solving the problem is the same as the method of the first aspect embodiment, its specific implementation can refer to the first aspect embodiment, and the similarities will not be repeated.

[0179] Figure 10 is a schematic diagram of an AI / ML function monitoring device according to an embodiment of this application. The device is configured on a terminal device, which is the first device in the first aspect embodiment. Contents identical to those in the first aspect embodiment will not be repeated. As shown in Figure 10, the AI / ML function monitoring device 1000 includes:

[0180] Processing unit 1010 initiates AI / ML function monitoring;

[0181] The interaction unit 1020 interacts with the second device to exchange information related to AI / ML function monitoring when performing AI / ML function monitoring.

[0182] In some embodiments, the processing unit 1010 initiates AI / ML function monitoring when a first condition is met, the first condition including at least one of the following:

[0183] The time has come to monitor AI / ML capabilities;

[0184] The current state of the terminal device does not meet the requirements of the current AI / ML functions;

[0185] The AI / ML functions of the terminal devices have been updated;

[0186] The models under one or more functions in the AI / ML capabilities of the terminal device have been updated.

[0187] In the above embodiments, the processing unit 1010 may also send first information to the network device, the first information being used to request the network device to send at least one of the following to the terminal device:

[0188] AI / ML function monitoring related data and / or information related to said data;

[0189] Information other than data related to AI / ML function monitoring;

[0190] The terminal device wants to obtain the configuration required for monitoring AI / ML functions.

[0191] In some embodiments, the terminal device performs the aforementioned AI / ML function monitoring.

[0192] In the above embodiments, the interaction unit 1020 can receive positioning reference signals and truth information periodically sent by the network device. The sending period of the positioning reference signal and the sending period of the truth information are synchronized or spaced apart by a first value.

[0193] In the above embodiments, the interaction unit 1020 may also receive at least one of the following information sent by the network device:

[0194] The timestamp information corresponding to the truth value information;

[0195] The applicable information corresponding to the truth value information.

[0196] In other embodiments, the network device performs the aforementioned AI / ML function monitoring.

[0197] In the above embodiment, the interaction unit 1020 sends the measurement results to the network device.

[0198] In the above embodiments, the interaction unit 1020 can also send the timestamp information corresponding to the measurement results to the network device.

[0199] In some embodiments, the interaction unit 1020 receives first configuration information sent by the network device, the first configuration information indicating at least one of the following:

[0200] Instruct terminal devices to report the basis for decision-making;

[0201] Instructing terminal devices to report decision-making information;

[0202] Instructs the terminal device to report the content of the decision information.

[0203] In the above embodiments, the interaction unit 1020 may also send second information to the network device after receiving the first configuration information, and request the network device to send relevant information through the second information;

[0204] After receiving the second information, the network device may perform at least one of the following actions:

[0205] Send the second configuration information to the terminal device according to the request of the second information;

[0206] Send data and / or information related to the data to the terminal device according to the request of the second information;

[0207] Send capability interaction information and / or function interaction information to the terminal device according to the request of the second information;

[0208] Send instruction information to the terminal device, which is used to instruct the terminal device to continue, modify or terminate the currently initiated AI / ML function monitoring process;

[0209] Stop the action.

[0210] In some embodiments, the interaction unit 1020 makes a decision when a second condition is met. The second condition may include at least one of the following:

[0211] The terminal equipment has highly accurate monitoring data or information;

[0212] The terminal device receives configuration information from the network device and is required to make a decision.

[0213] In some embodiments, the interaction unit 1020 makes a decision when a first event is met. The first event may include at least one of the following:

[0214] The first timer expired and the first configuration information was not received;

[0215] The first counter reaches the first threshold and no first configuration information is received;

[0216] The second timer expired and the positioning performance of the AI / ML function was insufficient.

[0217] The second counter reaches the second threshold and the positioning performance of the AI / ML function is insufficient.

[0218] The current conditions of the terminal device cannot meet the requirements of AI / ML functions.

[0219] In the above embodiments, the interaction unit 1020 may also send third information to the network device after making a decision based on the second condition or the first event. The third information is used to notify the network device of the content of the decision and / or to request the network device to confirm the content of the decision.

[0220] In some embodiments, the interaction unit 1020 receives fourth information sent by the network device, which is used to instruct the terminal device to report model-related information.

[0221] In the above embodiments, the interaction unit 1020 can also send model-related information to the network device so that the network device can initiate measurement configuration to the positioning reference unit (PRU) based on the model-related information, collect measurement results, and send the measurement results to the terminal device so that the terminal device can perform model inference based on the measurement results.

[0222] In the above embodiments, the interaction unit 1020 can also send model inference results to the network device so that the network device can make decisions based on the model inference results and send decision information to the terminal device.

[0223] In some embodiments, the interaction unit 1020 sends fifth information to the network device, which is used by the terminal device to request PRU information in the network device when initiating AI / ML function monitoring.

[0224] When a network device has a PRU, the network device performs at least one of the following processes:

[0225] Send the sixth message to the terminal device, requesting the terminal device to report model-related information;

[0226] Send a list of measurements supported by the PRU to the terminal device;

[0227] A seventh message is sent to the terminal device, which instructs the terminal device to perform a measurement and report the measurement result.

[0228] In the above embodiments, if the network device does not have a PRU or has a PRU but the available PRUs are insufficient, the network device may also send an eighth message to the terminal device, which instructs the terminal device to perform AI / ML function monitoring actions and / or instructs the terminal device to perform additional information interaction for AI / ML function monitoring.

[0229] In the above embodiments, the eighth information may indicate at least one of the following:

[0230] The terminal device lacks sufficient PRU information;

[0231] Instruct the terminal device to terminate the current AI / ML function monitoring process;

[0232] Instruct the terminal device to pause the current AI / ML function monitoring process;

[0233] Instructing terminal devices to interact with AI / ML capabilities and / or information related to AI / ML functions;

[0234] Instruct the terminal device to roll back and / or deactivate the AI / ML function.

[0235] In the above embodiments, the interaction unit 1020 may perform at least one of the following processes according to the instruction of the eighth information:

[0236] Terminate or suspend the current AI / ML function monitoring process;

[0237] Perform AI / ML function rollback and / or AI / ML function deactivation;

[0238] Send a ninth message to the network device, and use the ninth message to interact with AI / ML capabilities and / or AI / ML function related information;

[0239] Send a ninth message to the network device, requesting interaction with information related to the function monitoring method that does not use PRU.

[0240] The above embodiments are merely illustrative examples of embodiments of this application, but this application is not limited thereto, and appropriate modifications can be made based on the above embodiments. For example, the above embodiments can be used alone, or one or more of the above embodiments can be combined.

[0241] Figure 11 is another schematic diagram of an AI / ML function monitoring device according to an embodiment of this application. This device is configured on a network device, which is the second device in the first aspect embodiment. It is a network-side processing device corresponding to the first device (terminal device) in the first aspect embodiment. Contents identical to those in the first aspect embodiment will not be repeated. As shown in Figure 11, the AI / ML function monitoring device 1100 includes:

[0242] Processing unit 1110 initiates AI / ML function monitoring;

[0243] The interaction unit 1120 interacts with the second device to exchange information related to AI / ML function monitoring when performing AI / ML function monitoring.

[0244] In some embodiments, the terminal device performs the aforementioned AI / ML function monitoring.

[0245] In the above embodiments, the interaction unit 1120 can periodically send a positioning reference signal and truth information to the terminal device. The sending period of the positioning reference signal and the sending period of the truth information are synchronized or spaced apart by a first value.

[0246] In the above embodiments, the interaction unit 1120 may also send at least one of the following information to the terminal device:

[0247] The timestamp information corresponding to the truth value information;

[0248] The applicable information corresponding to the truth value information.

[0249] In other embodiments, the network device performs the aforementioned AI / ML function monitoring.

[0250] In the above embodiment, the interaction unit 1120 receives the measurement results sent by the terminal device.

[0251] In the above embodiments, the interaction unit 1120 can also receive the timestamp information corresponding to the measurement results sent by the terminal device.

[0252] In some embodiments, the interaction unit 1120 sends first configuration information to the terminal device, the first configuration information indicating at least one of the following:

[0253] Instruct terminal devices to report the basis for decision-making;

[0254] Instructing terminal devices to report decision-making information;

[0255] Instructs the terminal device to report the content of the decision information.

[0256] In the above embodiments, the interaction unit 1120 can also receive second information sent by the terminal device after sending the first configuration information. After receiving the second information, the interaction unit 1120 can perform at least one of the following actions:

[0257] Send the second configuration information to the terminal device according to the request of the second information;

[0258] Send data and / or information related to the data to the terminal device according to the request of the second information;

[0259] Send capability interaction information and / or function interaction information to the terminal device according to the request of the second information;

[0260] Send instruction information to the terminal device, which is used to instruct the terminal device to continue, modify or terminate the currently initiated AI / ML function monitoring process;

[0261] Stop the action.

[0262] In some embodiments, the terminal device makes a decision when a second condition is met. The second condition may include at least one of the following:

[0263] The terminal equipment has highly accurate monitoring data or information;

[0264] The terminal device receives configuration information from the network device and is required to make a decision.

[0265] In some embodiments, the terminal device makes a decision when a first event is met. The first event may include at least one of the following:

[0266] The first timer expired and the first configuration information was not received;

[0267] The first counter reaches the first threshold and no first configuration information is received;

[0268] The second timer expired and the positioning performance of the AI / ML function was insufficient.

[0269] The second counter reaches the second threshold and the positioning performance of the AI / ML function is insufficient.

[0270] The current conditions of the terminal device cannot meet the requirements of AI / ML functions.

[0271] In some embodiments, the interaction unit 1120 may also receive third information sent by the terminal device after the terminal device makes a decision based on the second condition or the first event. The third information is used to notify the network device of the content of the decision and / or to request the network device to confirm the content of the decision.

[0272] In some embodiments, the interaction unit 1120 sends fourth information to the terminal device, which instructs the terminal device to report model-related information.

[0273] In the above embodiments, the interaction unit 1120 can also receive model-related information sent by the terminal device, initiate measurement configuration to the positioning reference unit (PRU) based on the model-related information, collect measurement results, and send the measurement results to the terminal device so that the terminal device can perform model inference based on the measurement results.

[0274] In the above embodiments, the interaction unit 1120 can also receive model inference results sent by the terminal device, make decisions based on the model inference results, and send decision information to the terminal device.

[0275] In some embodiments, the interaction unit 1120 receives fifth information sent by the terminal device, which is used by the terminal device to request PRU information from the network device when initiating AI / ML function monitoring;

[0276] When the network device has a PRU, the interaction unit 1120 performs at least one of the following processes:

[0277] Send the sixth message to the terminal device, requesting the terminal device to report model-related information;

[0278] Send a list of measurements supported by the PRU to the terminal device;

[0279] A seventh message is sent to the terminal device, which instructs the terminal device to perform a measurement and report the measurement result.

[0280] In the above embodiments, if the network device does not have a PRU or has a PRU but the available PRUs are insufficient, the interaction unit 1120 may also send an eighth message to the terminal device, which instructs the terminal device to perform AI / ML function monitoring actions and / or instructs the terminal device to perform additional information interaction for AI / ML function monitoring.

[0281] In the above embodiments, the eighth information may indicate at least one of the following:

[0282] The terminal device lacks sufficient PRU information;

[0283] Instruct the terminal device to terminate the current AI / ML function monitoring process;

[0284] Instruct the terminal device to pause the current AI / ML function monitoring process;

[0285] Instructing terminal devices to interact with AI / ML capabilities and / or information related to AI / ML functions;

[0286] Instruct the terminal device to roll back and / or deactivate the AI / ML function.

[0287] In the above embodiments, the terminal device may perform at least one of the following processes according to the instruction of the eighth information:

[0288] Terminate or suspend the current AI / ML function monitoring process;

[0289] Perform AI / ML function rollback and / or AI / ML function deactivation;

[0290] Send a ninth message to the network device, and use the ninth message to interact with AI / ML capabilities and / or AI / ML function related information;

[0291] Send a ninth message to the network device, requesting interaction with information related to the function monitoring method that does not use PRU.

[0292] In the above embodiments, the interaction unit 1120 can receive the ninth information and interact with the terminal device to exchange information related to AI / ML capabilities and / or AI / ML functions, or interact with the terminal device to exchange information related to non-PRUs.

[0293] The above embodiments are merely illustrative examples of embodiments of this application, but this application is not limited thereto, and appropriate modifications can be made based on the above embodiments. For example, the above embodiments can be used alone, or one or more of the above embodiments can be combined.

[0294] It is worth noting that Figures 10 and 11 above are only illustrative of embodiments of this application, but this application is not limited thereto. For example, other modules or components may be appropriately added or some modules or components may be removed. Those skilled in the art can make appropriate modifications based on the above content, and are not limited to the description in Figures 10 and 11 above.

[0295] Furthermore, for simplicity, Figures 10 and 11 only illustrate the connection relationships or signal flows between the various components or modules, but those skilled in the art should understand that various related technologies such as bus connections can be used. The aforementioned components or modules can be implemented using hardware facilities such as processors, memory, transmitters, and receivers; this application does not limit this implementation.

[0296] According to the embodiments of this application, after the terminal device or network device initiates AI / ML function monitoring, the terminal device and network device exchange information related to AI / ML function monitoring during the AI / ML function monitoring process. This solves the problem that the current standard does not standardize the monitoring of AI / ML functions, especially wireless positioning use cases, between the core network device (LMF) and the terminal device (UE).

[0297] Third aspect of the embodiments

[0298] This application provides a communication system, which can be referred to Figure 1. The contents that are the same as those in the first aspect of the embodiment will not be repeated.

[0299] In some embodiments, the communication system 100 may include at least a network device 101 and terminal devices 102 and 103. In some embodiments, terminal devices 102 and 103 perform the functions of the first device in the first aspect embodiment, and correspondingly, network device 101 performs the functions of the second device in the first aspect embodiment. In other embodiments, network device 101 performs the functions of the first device in the first aspect embodiment, and correspondingly, terminal devices 102 and 103 perform the functions of the second device in the first aspect embodiment. Since the functions of the first and second devices have already been described in the first aspect embodiment, their contents are incorporated herein and will not be repeated here.

[0300] This application also provides a terminal device.

[0301] Figure 12 is a schematic block diagram of the system configuration of a terminal device according to an embodiment of this application. As shown in Figure 12, the terminal device 1200 may include a processor 1210 and a memory 1220; the memory 1220 is coupled to the processor 1210. It is worth noting that this figure is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunications functions or other functions.

[0302] In one embodiment, the terminal device 1200, as the first device in the first aspect embodiment, includes the functions of the device 1000 in the second aspect embodiment. These functions can be integrated into the processor 1210 or configured separately from the processor 1210. For example, the device 1000 can be configured as a chip connected to the processor 1210, and the functions of the device can be realized through the control of the processor 1210.

[0303] As shown in Figure 12, the terminal device 1200 may further include: a communication module 1230, an input unit 1240, a display 1250, and a power supply 1260. It is worth noting that the terminal device 1200 does not necessarily include all the components shown in Figure 12; furthermore, the terminal device 1200 may also include components not shown in Figure 12, which can be found in related technologies.

[0304] As shown in Figure 12, the processor 1210, sometimes also referred to as a controller or operation control, may include a microprocessor or other processor device and / or logic device. The processor 1210 receives input and controls the operation of various components of the terminal device 1200.

[0305] The memory 1220 may be, for example, one or more of a cache, flash memory, hard drive, removable medium, volatile memory, non-volatile memory, or other suitable means. It can store various types of data, and also programs for executing related information. The processor 1210 can execute the program stored in the memory 1220 to perform information storage or processing, etc. The functions of other components are similar to those in existing systems and will not be described further here. The components of the terminal device 1200 can be implemented using dedicated hardware, firmware, software, or a combination thereof without departing from the scope of the invention.

[0306] This application also provides a network device.

[0307] Figure 13 is a schematic block diagram of the system configuration of a network device according to an embodiment of this application. As shown in Figure 13, the network device 1300 may include a processor 1310 and a memory 1320; the memory 1320 is coupled to the processor 1310. The memory 1320 can store various data; in addition, it also stores an information processing program 1330, and executes the program 1330 under the control of the processor 1310.

[0308] In one embodiment, the network device 1300 serves as a second device in the first aspect embodiment and includes the functions of the device 1100 in the second aspect embodiment. These functions can be integrated into the processor 1310 or configured separately from the processor 1310. For example, the device 1100 can be configured as a chip connected to the processor 1310, and the functions of the device can be implemented through the control of the processor 1310.

[0309] In addition, as shown in Figure 13, network device 1300 may also include a transceiver 1340 and an antenna 1350, etc.; the functions of the above components are similar to those in the prior art, and will not be described in detail here. It is worth noting that network device 1300 does not necessarily have to include all the components shown in Figure 13; in addition, network device 1300 may also include components not shown in Figure 13, which can be referred to in the prior art.

[0310] According to the embodiments of this application, after the terminal device or network device initiates AI / ML function monitoring, the terminal device and network device exchange information related to AI / ML function monitoring during the AI / ML function monitoring process. This solves the problem that the current standard does not standardize the monitoring of AI / ML functions, especially wireless positioning use cases, between the core network device (LMF) and the terminal device (UE).

[0311] This application also provides a computer-readable program, wherein when the program is executed in a terminal device or network device, the program causes the computer to perform the method described in the first aspect of the embodiment in the terminal device.

[0312] This application also provides a storage medium storing a computer-readable program, wherein the computer-readable program causes a computer to perform the method described in the embodiments of the first aspect in a terminal device or network device.

[0313] This application also provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, it implements the method described in the embodiments of the first aspect.

[0314] The apparatus and methods described above in this application can be implemented in hardware or in combination with software. This application relates to a computer-readable program that, when executed by a logic component, enables the logic component to implement the apparatus or components described above, or to implement the various methods or steps described above. Logic components include, for example, field-programmable logic devices (FPGAs), microprocessors, and processors used in computers. This application also relates to storage media for storing the above programs, such as hard disks, magnetic disks, optical disks, DVDs, and flash memory.

[0315] The methods / apparatus described in conjunction with the embodiments of this application can be directly embodied in hardware, software modules executed by a processor, or a combination of both. For example, one or more and / or combinations of one or more functional block diagrams shown in the figures can correspond to various software modules in a computer program flow, or to various hardware modules. These software modules can correspond to the various steps shown in the figures, respectively. These hardware modules can be implemented, for example, using a field-programmable gate array (FPGA) to embed these software modules.

[0316] The software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. A storage medium can be coupled to the processor, enabling the processor to read information from and write information to the storage medium; or the storage medium can be an integral part of the processor. The processor and storage medium can reside in an ASIC. The software module can be stored in the memory of a mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if the device (such as a mobile terminal) uses a high-capacity MEGA-SIM card or a high-capacity flash memory device, the software module can be stored in the MEGA-SIM card or the high-capacity flash memory device.

[0317] One or more and / or one or more combinations of functional blocks described in the accompanying drawings can be implemented as a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or any suitable combination thereof for performing the functions described herein. One or more and / or one or more combinations of functional blocks described in the accompanying drawings can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0318] The present application has been described above with reference to specific embodiments. However, those skilled in the art should understand that these descriptions are exemplary and not intended to limit the scope of protection of the present application. Those skilled in the art can make various modifications and variations to the present application based on its spirit and principles, and these modifications and variations are also within the scope of the present application.

[0319] Regarding the above-described embodiments disclosed in this example, the following notes are also disclosed:

[0320] 1. A terminal device, comprising a memory and a processor, the memory storing a computer program, the processor being configured to execute the computer program to implement the following method:

[0321] Initiate AI / ML feature monitoring;

[0322] During AI / ML function monitoring, information related to AI / ML function monitoring is exchanged with network devices.

[0323] 2. A network device, comprising a memory and a processor, the memory storing a computer program, the processor being configured to execute the computer program to implement the following method:

[0324] Initiate AI / ML feature monitoring;

[0325] During AI / ML function monitoring, information related to AI / ML function monitoring is exchanged with the terminal device.

[0326] 3. A communication system comprising the terminal equipment described in Appendix 1 and the network equipment described in Appendix 2.

Claims

1. A monitoring device for artificial intelligence / machine learning (AI / ML) functions, configured in a first device, wherein, The device includes: The processing unit initiates AI / ML function monitoring; The interaction unit interacts with the second device to exchange information related to AI / ML function monitoring when the network device or terminal device is performing AI / ML function monitoring.

2. The apparatus according to claim 1, wherein, The first device is a terminal device, and the processing unit initiates AI / ML function monitoring, including: The processing unit initiates AI / ML function monitoring when a first condition is met, wherein the first condition includes at least one of the following: The time has come to monitor AI / ML capabilities; The current condition of the first device does not meet the requirements of the current AI / ML functions; The AI / ML functions of the first device have been updated; The model under one or more functions in the AI / ML functionality of the first device has been updated.

3. The apparatus according to claim 2, wherein, The processing unit sends first information to the second device, the first information being used to request the second device to send at least one of the following: AI / ML function monitoring related data and / or information related to said data; Information other than data related to AI / ML function monitoring; The first device wants to obtain the configuration required for AI / ML function monitoring.

4. The apparatus according to claim 1, wherein, The terminal device performs AI / ML function monitoring. The first device is a terminal device, and the second device is a network device. The interaction unit interacts with the second device to exchange information related to AI / ML function monitoring, including: The interaction unit receives positioning reference signals and truth information periodically sent by the second device. The sending period of the positioning reference signals and the sending period of the truth information are synchronized or spaced apart by a first value.

5. The apparatus according to claim 1, wherein, The terminal device performs AI / ML function monitoring. The first device is a terminal device, and the second device is a network device. The interaction unit interacts with the second device to exchange information related to AI / ML function monitoring, including: The interaction unit receives at least one of the following information sent by the second device: The timestamp information corresponding to the truth value information; The applicable information corresponding to the truth value information.

6. The apparatus according to claim 1, wherein, The network device performs AI / ML function monitoring. The first device is a terminal device, and the second device is a network device. The interaction unit interacts with the second device to exchange information related to AI / ML function monitoring, including: The interaction unit sends the measurement results to the second device.

7. The apparatus according to claim 6, wherein, The interaction unit interacts with the second device to exchange information related to AI / ML function monitoring, and also includes: The interaction unit sends the timestamp information corresponding to the measurement result to the second device.

8. The apparatus according to claim 1, wherein, The first device is a terminal device, and the second device is a network device. The interaction unit interacts with the second device to exchange information related to AI / ML function monitoring, including: The interaction unit receives first configuration information sent by the second device, wherein the first configuration information indicates at least one of the following: Instruct the first device to report the basis for its decision; Instructs the first device to report decision information; Instructions are given on the content of the decision information to be reported by the first device.

9. The apparatus according to claim 8, wherein, The interaction unit interacts with the second device to exchange information related to AI / ML function monitoring, and also includes: After receiving the first configuration information, the interaction unit sends the second information to the second device, and requests the second device to send relevant information through the second information; Upon receiving the second information, the second device performs at least one of the following actions: Send second configuration information to the first device according to the request of the second information; Send data and / or information related to the data to the first device according to the request of the second information; Send capability interaction information and / or function interaction information to the first device according to the request of the second information; Send indication information to the first device, the indication information being used to instruct the first device to continue, modify, or terminate the currently initiated AI / ML function monitoring process; Stop the action.

10. The apparatus according to claim 1, wherein, The first device is a terminal device, and the second device is a network device. The interaction unit interacts with the second device to exchange information related to AI / ML function monitoring, including: The interaction unit makes a decision when the second condition is met.

11. The apparatus according to claim 1, wherein, The first device is a terminal device, and the second device is a network device. The interaction unit interacts with the second device to exchange information related to AI / ML function monitoring, including: The interaction unit makes a decision when the first event is met.

12. The apparatus according to claim 10, wherein, The interaction unit interacts with the second device to exchange information related to AI / ML function monitoring, and also includes: After making a decision, the interaction unit sends a third message to the second device. The third message is used to notify the second device of the content of the decision and / or to request the second device to confirm the content of the decision.

13. The apparatus according to claim 1, wherein, The first device is a terminal device, and the second device is a network device. The interaction unit interacts with the second device to exchange information related to AI / ML function monitoring, including: The interaction unit receives fourth information sent by the second device, which is used to instruct the first device to report model-related information.

14. The apparatus according to claim 13, wherein, The interaction unit interacts with the second device to exchange information related to AI / ML function monitoring, and also includes: The interaction unit sends model-related information to the second device, so that the second device can initiate measurement configuration with the positioning reference unit (PRU) based on the model-related information, collect measurement results, and send the measurement results to the first device, so that the first device can perform model inference based on the measurement results.

15. The apparatus according to claim 14, wherein, The interaction unit interacts with the second device to exchange information related to AI / ML function monitoring, and also includes: The interaction unit sends the model inference results to the second device so that the second device can make a decision based on the model inference results and send decision information to the first device.

16. The apparatus according to claim 1, wherein, The first device is a terminal device, and the second device is a network device. The interaction unit interacts with the second device to exchange information related to AI / ML function monitoring, including: The interaction unit sends a fifth message to the second device, the fifth message being used by the first device to request PRU information from the second device when initiating AI / ML function monitoring; When the second device has a PRU, the second device performs at least one of the following processes: Send a sixth message to the first device, requesting the first device to report model-related information; Send a list of measurements supported by the PRU to the first device; A seventh message is sent to the first device, instructing the first device to perform a measurement and report the measurement result.

17. The apparatus according to claim 16, wherein, If the second device does not have a PRU or has a PRU but insufficient available PRUs, the second device also sends an eighth message to the first device. The interaction unit receives the eighth message, which indicates the action of the first device in monitoring AI / ML functions and / or instructs the first device to perform additional information interaction for monitoring AI / ML functions.

18. The apparatus according to claim 17, wherein, The eighth information indicates at least one of the following: The first device is indicated to have insufficient PRU information; Instruct the first device to terminate the current AI / ML function monitoring process; Instruct the first device to pause the current AI / ML function monitoring process; Instruct the first device to interact with AI / ML capabilities and / or AI / ML function-related information; Instruct the first device to roll back the AI / ML function and / or deactivate the AI / ML function.

19. The apparatus according to claim 17, wherein, The interactive unit performs at least one of the following processes according to the instruction of the eighth information: Terminate or suspend the current AI / ML function monitoring process; Perform AI / ML function rollback and / or AI / ML function deactivation; Send a ninth message to the second device, and use the ninth message to interact with AI / ML capabilities and / or AI / ML function related information; Send a ninth message to the second device, requesting interaction with information related to the function monitoring method that does not use PRU through the ninth message.

20. A communication system, comprising terminal equipment and network equipment, wherein, The terminal device or the network device initiates and performs AI / ML function monitoring. The terminal device and the network device interact to monitor AI / ML function-related information.

Images