Wireless communication method and device

Abstract

Provided are a wireless communication method and device, the method comprising: a first network element receiving first information reported by a second network element, the first information being used for indicating a performance monitoring result of a first model used by a terminal device; according to the embodiment of the invention, the network element (namely the first network element) of the core network receives the model performance monitoring result (namely the performance monitoring result of the first model used by the terminal equipment) reported by the terminal equipment, so that the core network or a third-party server can monitor the performance of the model provided by the core network or the third-party server; and thus, the communication quality of the communication system can be ensured.

Description

Wireless communication method and device Technical Field

[0001] The present application relates to the field of communication technology, and more particularly, to a method and device for wireless communication. Background Art

[0002] To ensure the performance of a model, such as an artificial intelligence (AI) model, performance monitoring is often necessary. For models deployed on terminal devices that are generated by the core network or third-party servers, monitoring the performance of these models on the terminal device to ensure their effective operation is a pressing issue.

[0003] Summary of the Invention

[0004] The present application provides a method and device for wireless communication. The following introduces various aspects involved in the present application.

[0005] In a first aspect, a method for wireless communication is provided, the method comprising: a first network element receiving first information reported by a second network element, the first information being used to indicate a performance monitoring result of a first model used by the terminal device.

[0006] In a second aspect, a method for wireless communication is provided, the method comprising: a third network element sends a request message to the first network element, the request message being used to request performance monitoring of a first model used by a terminal device.

[0007] According to a third aspect, a method for wireless communication is provided, the method comprising: a user plane function network element of a core network receives first information reported by a terminal device, wherein the first information is used to indicate a performance monitoring result of a first model used by the terminal device.

[0008] In a fourth aspect, a method for wireless communication is provided, the method comprising: a terminal device reporting first information to a network element of a core network, wherein the first information is used to indicate a performance monitoring result of a first model used by the terminal device.

[0009] In the fifth aspect, a core network network element is provided, which is a first network element. The core network network element includes: a first receiving unit, used to receive first information reported by a second network element, and the first information is used to indicate the performance monitoring results of the first model used by the terminal device.

[0010] In the sixth aspect, a core network network element is provided, which is a third network element. The core network network element includes: a sending unit, used to send a request message to the first network element, and the request message is used to request performance monitoring of the first model used by the terminal device.

[0011] In the seventh aspect, a core network network element is provided, which is a user plane function network element of the core network, and the core network network element includes: a first receiving unit, used to receive first information reported by a terminal device, and the first information is used to indicate the performance monitoring result of the first model used by the terminal device.

[0012] In an eighth aspect, a terminal device is provided, comprising: a reporting unit for reporting first information to a network element of a core network, wherein the first information is used to indicate a performance monitoring result of a first model used by the terminal device.

[0013] In the ninth aspect, a core network element is provided, comprising a processor and a memory, wherein the memory is used to store one or more computer programs, and the processor is used to call the computer program in the memory so that the core network element executes part or all of the steps in the method of the first aspect, the second aspect or the third aspect.

[0014] In the eighth aspect, a terminal device is provided, comprising a processor and a memory, wherein the memory is used to store one or more computer programs, and the processor is used to call the computer program in the memory so that the core network element executes part or all of the steps in the method of the fourth aspect.

[0015] Ninthly, an embodiment of the present application provides a communication system, which includes the above-mentioned core network element and / or terminal device. In another possible design, the system may also include other devices that interact with the core network element and / or terminal device in the solution provided in the embodiment of the present application.

[0016] In the tenth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program enables the core network network element to execute part or all of the steps in the methods of the above aspects.

[0017] In an eleventh aspect, embodiments of the present application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a core network element to perform some or all of the steps of the methods described in each of the above aspects. In some implementations, the computer program product may be a software installation package.

[0018] In the twelfth aspect, an embodiment of the present application provides a chip, which includes a memory and a processor. The processor can call and run a computer program from the memory to implement some or all of the steps described in the methods of the above aspects.

[0019] The embodiment of the present application receives the model performance monitoring results (i.e., the performance monitoring results of the first model used by the terminal device) reported by the terminal device through the network element of the core network (i.e., the first network element), which helps the core network or a third-party server to monitor the performance of the model provided by it, thereby helping to ensure the communication quality of the communication system. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] FIG1 is a wireless communication system used in an embodiment of the present application.

[0021] FIG2 is a schematic diagram of a neural network applicable to an embodiment of the present application.

[0022] FIG3 is a schematic diagram of a convolutional neural network applicable to an embodiment of the present application.

[0023] FIG4 is a flow chart of a communication method according to an embodiment of the present application.

[0024] FIG5 is a schematic diagram of a method for model monitoring using a control plane provided in an embodiment of the present application.

[0025] FIG6 is a schematic diagram of a method for model monitoring using user plane data provided in an embodiment of the present application.

[0026] FIG7 is a schematic structural diagram of a core network element provided in an embodiment of the present application.

[0027] FIG8 is a schematic structural diagram of another core network element provided in an embodiment of the present application.

[0028] FIG9 is a schematic structural diagram of another core network element provided in an embodiment of the present application.

[0029] FIG10 is a schematic structural diagram of a terminal device provided in an embodiment of the present application.

[0030] FIG11 is a schematic structural diagram of a device for communication according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] The technical solution in this application will be described below with reference to the accompanying drawings.

[0032] The technical solutions of the embodiments of the present application can be applied to various communication systems, such as: fifth generation (5G) system or new radio (NR), long term evolution (LTE) system, LTE frequency division duplex (FDD) system, LTE time division duplex (TDD) system, etc. The technical solutions provided by the present application can also be applied to future communication systems, such as the sixth generation mobile communication system, satellite communication system, etc.

[0033] First, a brief introduction to the network architecture applicable to this application is given.

[0034] As an example, Figure 1 shows an example diagram of a network architecture. The network architecture shown in Figure 1 takes the 5G system as an example. Among them, one of the most important features of the 5G network architecture is the service-oriented architecture, that is, the core network element (service provider) can provide specific services and make them available to other network elements (consumers) through a defined application programming interface (API). The network architecture can include three parts: the terminal device part, the data network (DN) part, and the operator network part. Among them, the operator network may include one or more of the following functional entities: access network (AN) equipment, user plane function (UPF) entity, access and mobility management function (AMF) entity, session management function (SMF) entity, policy control function (PCF) entity, application function (AF) entity, network slice selection function (NSSF) entity, authentication and authorization service function (AUSF) entity, unified data management (UDM) entity, network exposure function (NEF) entity, network repository function (NRF) entity, network slice-specific authentication and authorization function (NSSAAF) entity, etc. In the above-mentioned operator network, the part other than the access network equipment part can be called the core network part, and the functional entities of the core network part can be called core network equipment.

[0035] The following is an illustrative description of the functions of each part or functional entity involved in the network architecture in the 5G network.

[0036] Terminal device: A terminal device may also be referred to as user equipment (UE), access terminal, user unit, user station, mobile station, mobile station (MS), mobile terminal (MT), remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent, or user apparatus. In the embodiments of the present application, a terminal device may refer to a device that provides voice and / or data connectivity to a user and can be used to connect people, objects, and machines, such as a handheld device or vehicle-mounted device with wireless connectivity. The terminal device in the embodiments of the present application can be a mobile phone, a tablet computer, a laptop computer, a PDA, a mobile internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, etc. Optionally, the UE can be used to act as a base station. For example, the UE can act as a scheduling entity that provides sidelink signals between UEs in V2X or D2D, etc. For example, a cellular phone and a car communicate with each other using sidelink signals. The cellular phone and smart home devices communicate without relaying the communication signal through the base station.

[0037] Access network equipment: Access network equipment provides network access for authorized terminal devices in a specific area and can utilize transmission channels of varying quality based on the terminal device's level and service requirements. Access network equipment manages wireless resources, provides access services to terminal devices, and forwards control signals and data between terminal devices and the core network.

[0038] An access network device may be a device in a wireless network. An access network device may also be referred to as a radio access network (RAN) device or a network device. For example, an access network device may be a base station. The access network device in the embodiments of the present application may refer to a radio access network (RAN) node (or device) that connects a terminal device to a wireless network. A base station can broadly cover various names as follows, or be replaced with the following names, such as: NodeB, evolved NodeB (eNB), next generation NodeB (gNB), relay station, access point, transmission point (TRP), transmission point (TP), master station MeNB, secondary station SeNB, multi-standard radio (MSR) node, home base station, network controller, access node, wireless node, access point (AP), transmission node, transceiver node, baseband unit (BBU), remote radio unit (RRU), active antenna unit (AAU), remote radio head (RRH), central unit (CU), distributed unit (DU), positioning node, etc. A base station can be a macro base station, a micro base station, a relay node, a donor node or the like, or a combination thereof. A base station can also refer to a communication module, a modem or a chip used to be set in the aforementioned device or apparatus. The base station can also be a mobile switching center and a device that performs base station functions in device-to-device D2D, vehicle-to-everything (V2X), and machine-to-machine (M2M) communications, a network-side device in a 6G network, or a device that performs base station functions in future communication systems. The base station can support networks with the same or different access technologies. The embodiments of this application do not limit the specific technology and specific device form used by the access network device.

[0039] Base stations can be fixed or mobile. For example, a helicopter or drone can be configured to act as a mobile base station, and one or more cells can move based on the location of the mobile base station. In other examples, a helicopter or drone can be configured to act as a device that communicates with another base station.

[0040] In some deployments, the access network device in the embodiments of the present application may refer to a CU or a DU, or the access network device may include a CU and a DU. The gNB may also include an AAU.

[0041] The access network equipment and terminal equipment can be deployed on land, including indoors or outdoors, handheld or vehicle-mounted; they can also be deployed on water; they can also be deployed in the air on aircraft, balloons, and satellites. The embodiments of this application do not limit the scenarios in which the access network equipment and terminal equipment are located.

[0042] UPF entity: UPF is the user plane function in the core network, which can be responsible for forwarding and receiving user data (such as business data flow) in the terminal device. For example, UPF can receive user data from DN and transmit it to the terminal device through the access network device; or, UPF can also receive user data from the terminal device through the access network device and then forward it to DN. The transmission resources and scheduling functions that provide services to terminal devices in UPF are managed and controlled by SMF. The bearer between the terminal and the UPF network element may include: the user plane connection between the UPF network element and the access network device, and the establishment of a channel between the access network device and the terminal. Among them, the user plane connection is a QoS flow that can be established between the UPF network element and the access network device to transmit data.

[0043] AMF entity: AMF is the mobility management function in the core network. It can be used to implement other functions of the mobility management entity (MME) in addition to session management, such as lawful interception or access authorization (or authentication). In some embodiments, in addition to performing mobility management on terminal devices, the AMF can also be responsible for forwarding session management related messages between terminal devices and SMF.

[0044] SMF entity: SMF is the session management function in the core network, which is mainly responsible for session management, Internet protocol (IP) address allocation and management of terminal devices, selection of endpoints for manageable user plane functions, policy control, or charging function interfaces, downlink data notification, and configuration of routing information for user plane functions.

[0045] PCF entity: PCF is the policy management function in the core network, which is responsible for formulating policies related to mobility management, session management, and billing for terminal devices. Specifically, PCF can provide policy rule information to functional entities of the control plane (such as AMF and SMF entities) to manage and control mobility management and session management of terminal devices.

[0046] AF entity: AF mainly supports interaction with the 3rd generation partnership project (3GPP) core network to provide services, for example, influencing data routing decisions, policy control functions, or providing some third-party services to the network side. In other words, AF can be mainly used to convey the requirements of the application side to the network side. In some embodiments, AF can be understood as a third-party server, such as an application server in the Internet, which provides relevant business information, including providing service quality requirement information corresponding to the business to PCF, and sending user plane data information of the business to A-UPF. In some embodiments, AF can also be a service provider (content provider, CP).

[0047] DN: A DN is a network used to transmit data. It can be a private network, such as a local area network (LAN), an external network not controlled by a carrier, such as the internet, or a proprietary network deployed jointly with carriers, such as the network that provides IP multimedia core network subsystem (IMS) services.

[0048] NSSF entity: NSSF is the network slice selection function in the core network. The functions it supports include: selecting a set of network slice instances to serve the UE; determining the allowed network slice selection assistance information (NSSAI) and, when necessary, determining the mapping to the contracted single-network slice selection assistance information (S-NSSAI); determining the configured NSSAI and, when necessary, determining the mapping to the contracted S-NSSAI; determining the set of AMFs that may be used to query the UE, or determining a list of candidate AMFs based on the configuration.

[0049] AUSF can be used to receive AMF's request for terminal authentication, request a key from UDM, and then forward the issued key to AMF for authentication processing.

[0050] UDM can include functions such as the generation and storage of user contract data, management of authentication data, and support interaction with external third-party servers.

[0051] NEF can be used for capability exposure. That is, based on NEF, network capabilities can be exported to external networks. External, untrusted applications can access core network data through NEF to ensure network security. NEF can also provide external application QoS capability exposure, event subscription, and AF request distribution.

[0052] NRF is used to register, manage, and monitor the status of core network elements, enabling automated management of core network elements. Upon startup, a core network element must register with the NRF before it can provide services. Registration information may include the core network element's type, address, and service list.

[0053] In addition, 5G networks have added a network data analytics function (NWDAF) to the core network. Based on NWDAF, data can be collected from various core network elements and network management systems, and big data statistics and analysis or intelligent data analysis can be performed to obtain network-side analysis or prediction data, thereby assisting each network element to more effectively control terminal device access based on the data analysis results.

[0054] In some embodiments, the network architecture shown in FIG1 can be divided into a user plane and a control plane, wherein the user plane can be used to transmit data, such as data transmission through a user plane functional network element, and the control plane can be used to transmit signaling. For example, the portion above the dotted line in FIG1 can be referred to as the control plane, and the portion below the dotted line can be referred to as the user plane.

[0055] It should be understood that the above functional entities in the core network can also be referred to as network elements, and this application is not limited to this. For example, the UPF entity can also be referred to as a UPF network element, and the AMF entity can also be referred to as an AMF network element, etc.

[0056] It should also be understood that in some embodiments, the xx functional entity or xx network element may be directly referred to as xx. For example, the UPF entity (or UPF network element) may be referred to as UPF, and the AMF entity (or AMF network element) may be referred to as AMF. For the sake of convenience, xx (such as UPF, AMF, etc.) mentioned in the embodiments of this application may refer to the xx entity or xx network element, which will not be repeated hereafter.

[0057] In the network architecture shown in Figure 1, various components or functional entities can communicate with each other through interfaces. For example, a terminal device can establish an access stratum (AS) connection with the AN via the Uu interface, exchanging AS messages and wireless data transmission. A terminal device can establish a non-access stratum (NAS) connection with the AMF via the N1 interface, exchanging NAS messages. The AN can connect to the AMF via the N2 interface to transmit radio bearer control information from the core network to the AN. The UPF can transmit data with the AN via the N3 interface and with the DN via the N6 interface. The interfaces connecting other components or functional entities can be found in Figure 1 and will not be detailed here.

[0058] It should be understood that the network architecture shown above is only an exemplary illustration, and the network architecture applicable to the embodiments of the present application is not limited to this. Any network architecture that can realize the functions of the above-mentioned functional entities is applicable to the embodiments of the present application.

[0059] It should be understood that the access network equipment, AMF, SMF, UPF, and PCF shown in Figure 1 are just names, and the names do not limit the equipment themselves. In 5G networks and other future networks, the entities corresponding to the access network equipment, AMF, SMF, UPF, and PCF may also have other names, and this embodiment of the application does not specifically limit this.

[0060] It should be understood that the interface names between the functional entities shown in Figure 1 are only an example. In the specific implementation, the interface names between the functional entities can also be other names, such as the interface names between the functional entities in the 6G network. The embodiments of the present application do not make specific limitations on this.

[0061] It should be understood that all or part of the functions of the communication device in this application can also be implemented through software functions running on hardware, or through virtualization functions instantiated on a platform (such as a cloud platform).

[0062] It should be understood that the network architecture described in the embodiments of the present application is intended to more clearly illustrate the technical solutions of the embodiments of the present application, and does not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art will know that with the evolution of the network architecture, the embodiments of the present application may also be applicable to similar technical problems.

[0063] AI

[0064] In recent years, artificial intelligence research represented by neural networks has achieved great results in many fields, and it will play an important role in people's production and life for a long time to come.

[0065] FIG2 is a schematic diagram of a neural network applicable to an embodiment of the present application. The neural network shown in FIG2 can be divided into three categories according to the positions of different layers: an input layer 210, a hidden layer 220, and an output layer 230. Generally speaking, the first layer is the input layer 210, the last layer is the output layer 230, and the intermediate layers between the first and last layers are all hidden layers 220. Samples can be input from the input layer 210, processed by the hidden layer 220, and the final result is generated in the output layer 230. Among them, each node represents a processing unit, which can be considered to simulate a neuron. Multiple neurons form a layer of a neural network, and multiple layers of information transmission and processing construct an overall neural network.

[0066] With the continuous development of neural network research, deep learning algorithms have been proposed in recent years, introducing a large number of hidden layers. This layer-by-layer training of multi-hidden neural networks for feature learning has greatly improved the learning and processing capabilities of neural networks and has been widely applied in pattern recognition, signal processing, optimization and combination, anomaly detection, and other fields.

[0067] Similarly, with the development of deep learning, convolutional neural networks have also been further studied. Figure 3 is a schematic diagram of a convolutional neural network to which an embodiment of the present application is applicable. The basic structure of a convolutional neural network may include: an input layer 310, multiple convolutional layers 320, multiple pooling layers 330, a fully connected layer 340, and an output layer 350. The introduction of the convolutional layer 320 and the pooling layer 330 effectively controls the sharp increase in network parameters, limits the number of parameters, and exploits the characteristics of the local structure, thereby improving the robustness of the algorithm.

[0068] AI and wireless communications

[0069] R18 proposes multiple use cases for applying AI to wireless communications. These include using AI / machine learning (ML) technologies to compress and decompress channel state information (CSI), reducing air interface transmission overhead and improving the accuracy of CSI feedback information. AI / ML technologies are used to predict beam information in the time and spatial domains, minimizing measurement overhead and latency and improving beam selection accuracy. AI / ML technologies are also used to predict UE location information, improving the accuracy of terminal device location information in non-line-of-sight (NLOS) scenarios.

[0070] Current wireless communication systems offer greater flexibility than ever before, emphasizing broad applicability across diverse scenarios and full utilization of limited resources. However, the fundamental principles underlying much of this work are still based on theoretical modeling of actual communication environments or simple parameter selection. The gains achieved by this basic approach are gradually diminishing in diverse and complex communication environments. To address this situation, it is necessary to adopt new methods and approaches, integrating them with traditional wireless communication theories and systems, to explore new approaches, break through performance bottlenecks, and further enhance the performance of wireless communication systems.

[0071] Therefore, based on the various use cases of AI in wireless communications, Release 18 proposes research on the lifecycle management of AI models. This includes, for example, model generation, model deployment, model transmission, model monitoring, and model updates.

[0072] Model monitoring, i.e., model performance monitoring, can include multiple aspects. In some embodiments, model performance monitoring can include one or more of the following aspects: model reasoning accuracy, model reasoning time, model impact on communication system performance, model size, model robustness, and model generalization.

[0073] The accuracy of model inference is the most important performance indicator of the model. If the error rate of the results of the UE's inference using the model increases, it means that the model performance has degraded and the model is no longer suitable for the terminal device's needs.

[0074] The model's inference time is the time it takes for a terminal device to perform inference using a model. Generally, the model's inference time is inversely proportional to its performance. For example, a longer model's inference time is considered to have poor performance.

[0075] The impact of the model on the performance of the communication system can be reflected through some performance parameters of the communication system, such as throughput, bit error rate, etc.

[0076] The size of the model is limited by the storage capacity of the terminal device. If the terminal device does not have enough space to store the model, the model will affect the communication performance of the terminal device.

[0077] The robustness of a model can be used to measure whether the model can still maintain the accuracy of judgment when facing slight changes in the input data, that is, whether it has the ability to tolerate slight changes in the input samples.

[0078] The generalization of a model can measure its ability to process new samples. If a model can only be used accurately in very few cases, it is considered that the model has poor generalization.

[0079] In actual use, AI models have a scope of application and usage scenarios. An AI model can only be applied to specific scenarios and perform inference on specific samples. Therefore, when the model used in a specific scenario changes, the performance of the model will decrease when the scenario or environment changes. For example, when the model used on the terminal device side changes, when the environment in which the terminal device is located changes, when the characteristics of the sample changes, and when the needs of the UE change, the performance of the model will decrease. To avoid this situation, it is necessary to monitor the operating effect of the model, or the performance of the model. Furthermore, the model can be managed based on its operating effect, such as timely updating of the model, to ensure the efficient operation of the communication system.

[0080] In some embodiments, the model may be generated in the core network or a third-party server. In this case, how to monitor the performance of the model on the terminal device side and ensure the effective operation of the model on the terminal device side is an urgent problem to be solved.

[0081] In order to solve the above problems, an embodiment of the present application provides a wireless communication method, which receives the model performance monitoring results reported by the terminal device (i.e., the performance monitoring results of the first model used by the terminal device) through the network element of the core network (i.e., the first network element), which helps the core network or a third-party server to monitor the performance of the model provided by it, thereby helping to ensure the communication quality of the communication system.

[0082] The communication method provided in the embodiment of the present application is described in detail below with reference to Figures 4 to 6.

[0083] Figure 4 is a flow chart of a communication method according to an embodiment of the present application. The method shown in Figure 4 may include step S410, which is described in detail below.

[0084] In step S410, the first network element receives first information reported by the second network element, wherein the first information may be used to indicate a performance monitoring result of the first model.

[0085] In some embodiments, the first network element may be one of the core network elements mentioned above. For example, the first network element may be a PCF responsible for formulating policies related to mobility management, session management, and billing for terminal devices. In another example, the first network element may be a NEF responsible for capability exposure. In another example, the first network element may be a NWDAF responsible for data analysis.

[0086] The first model may be a model deployed on the terminal device side, or in other words, the first model may be a model used by the terminal device. The first model mentioned here may be, for example, an AI / ML model. In some embodiments, the first model may be used for compression and decompression of channel state information. In other embodiments, the first model may be used for beam prediction, such as predicting beam information in the time domain or spatial domain. In still other embodiments, the first model may be used to predict the location information of the terminal device.

[0087] As mentioned above, the performance monitoring of the model may include one or more of the following aspects: the accuracy of model reasoning, the reasoning time of the model, the impact of the model on the performance of the communication system, the size of the model, the robustness of the model, and the generalization of the model.

[0088] In some embodiments, the performance monitoring results of the first model may include monitoring results of one or more of the above-mentioned aspects. For example, the performance monitoring results of the first model may include monitoring results of the accuracy of model reasoning of the first model, and may also include monitoring results of the time-consuming model reasoning of the first model. For another example, the performance monitoring results of the first model may include monitoring results of the accuracy of model reasoning, and monitoring results of the impact of the first model on the performance of the communication system. The performance monitoring results of the first model mentioned here are only given as examples, and this application does not limit this.

[0089] In some embodiments, the first information may indicate model information of the first model, such as a model identifier (ID), so as to associate the performance monitoring result with the corresponding model.

[0090] The performance of the first model may vary in different scenarios and environments. For example, the performance of the first model may vary when the first model is used in different time periods or at different locations. Therefore, the first information may indicate the performance of the first model during the monitoring period and / or the location where the first model performance was generated.

[0091] In some embodiments, the first information may indicate all monitoring results or monitoring data of the performance of the first model. In other embodiments, the first information may indicate a threshold reached by the performance of the first model, or a preset condition.

[0092] The thresholds and preset conditions mentioned herein can be used to determine changes in the operational performance of the first model. For example, the threshold reached by the first model's performance can be a threshold corresponding to a single performance of the first model. In another example, the threshold reached by the first model's performance can be an aggregate threshold corresponding to multiple model performances of the first model. As an example, the aggregate threshold can correspond to the computational results of multiple model performances of the first model, that is, the aggregate threshold can be a threshold of the computational results of multiple model performances of the first model.

[0093] Based on the threshold reached by the first model's performance, the first model can be promptly updated or otherwise processed when its performance degrades to ensure the communication quality of the communication system. Furthermore, the first information indicating the threshold reached by the first model's performance can conserve resources for transmitting the first information.

[0094] The first information may indicate one or more of the above-mentioned information of the first model. For example, the first information may indicate the identifier of the first model and the threshold reached by the performance of the first model during the monitoring period, or may indicate the threshold reached by the performance of the first model and the location where the performance occurred. For another example, the first information may indicate the identifier of the first model, the performance of the first model during the monitoring period, and the location where the performance occurred.

[0095] Since the performance monitoring results of the first model can generally be obtained by monitoring a terminal device using the first model, in some embodiments, the second network element may be a terminal device, that is, the first information may be directly reported by the terminal device. In other embodiments, the first information may also be indirectly reported by the terminal device via another device. For example, the first information may be indirectly reported by a terminal device in the core network via a session management network element, that is, the second network element may be a session management network element.

[0096] Typically, the model provider, or the model generator, monitors the performance of the model. If the first model is generated by a core network element or a third-party server, that is, if the first model is provided by the core network element or the third-party server, the core network element or the third-party server needs to monitor the performance of the first model. Therefore, in some embodiments, the core network element or the third-party server can receive the performance monitoring results of the first model through the first network element in the core network.

[0097] In an embodiment of the present application, using the first network element in the core network as a bridge, the network elements of the core network and the third-party server can obtain the performance monitoring results of the first model, thereby enabling the network elements of the core network and the third-party server to monitor the performance of the first model on the terminal device side, which helps to ensure the communication quality of the communication system.

[0098] Based on the first network element, the core network network element and the third-party server can monitor the performance of the first model in various ways. In other words, the first information can be carried in various ways. For example, the first information can be carried in control plane signaling or user plane data of the core network.

[0099] In some embodiments, the control plane may be used to perform performance monitoring of the first model, or in other words, the information (i.e., the first information) for indicating the performance monitoring result of the first model used by the terminal device is carried in the control plane signaling. For example, the first information may be carried in the control signaling for communication between the first network element and the terminal device. In this way, the terminal device can communicate directly with the first network element, such as reporting the first information to the first network element, which is simple to implement and easy to use. In other embodiments, the user plane data may be used to perform performance monitoring of the first model, or in other words, the first information is carried in the user plane data. For example, the first information may be carried in the data for communication between the user plane function and the terminal device. In the case where the data volume of the performance monitoring result of the model is large, carrying the first information in the user plane data is conducive to saving control signaling.

[0100] In some embodiments, before the first network element receives the first information reported by the second network element, the method shown in FIG4 may further include step S420.

[0101] In step S420, the first network element sends a first policy to the second network element, wherein the first policy may be related to performance monitoring of the first model.

[0102] As an implementation, the first policy may be used to indicate the content of the first model performance monitoring. For example, the first policy may be used to indicate model information of the first model, such as the model ID of the first model, or may be used to indicate parameter information associated with the performance monitoring of the first model, such as one or more of the following parameter information of the first model: model performance to be monitored; threshold of the model performance to be monitored; monitoring period; and monitoring location.

[0103] The performance of the first model may include multiple aspects, or in other words, multiple performance parameters. Therefore, the first strategy may include model performance to be monitored to indicate which performance of the first model is to be monitored.

[0104] The performance of the first model may vary during different time periods during the model's operation. Based on different needs, the performance of the first model may be monitored at different time periods. Therefore, the first policy may include a monitoring period to indicate when the performance of the first model is to be monitored. For example, if the user or model provider is more concerned about the operational performance of the first model during a first time period, the first policy may include the first time period. For another example, if the performance of the first model during a second time period may affect the performance of the communication system, the first policy may include the second time period.

[0105] If the operating location of the first model changes, or the location of the device using the first model changes, the performance of the first model may change. The performance of the first model can be monitored at different locations according to different needs. Therefore, the first strategy can include monitoring locations. For example, when the device using the first model is located at the edge of a cell, the performance of the first model is poor, and the first strategy can include the cell edge location. For another example, if the user or model provider is more concerned about the performance of the first model when the device using the first model is located at the center of the cell, the first strategy can include the cell center location.

[0106] In some embodiments, the first strategy may further include a threshold of the model performance to be monitored of the first model. Based on the threshold of the model performance to be monitored, it can be determined whether the model performance to be monitored meets the usage requirements. For example, when the model performance to be monitored is higher or lower than the threshold of the model performance to be monitored, it is considered that the model performance to be monitored meets or does not meet the requirements. If the accuracy of the model reasoning of the first model is lower than the first threshold, it is considered that the model reasoning accuracy of the first model does not meet the usage requirements. In this case, the first strategy may include a first threshold of the accuracy of the model reasoning of the first model.

[0107] The aforementioned threshold may be a threshold corresponding to a single model parameter of the first model, or may be an aggregate threshold corresponding to multiple model parameters as mentioned above. In some cases, the first strategy may further include multiple thresholds for the model performance to be monitored of the first model. For example, the performance of the first model reaching different thresholds may indicate various operating states of the first model.

[0108] As another implementation manner, the first policy may be used to indicate a manner in which the terminal device reports the first information.

[0109] For example, the first policy may be used to instruct the reporting of all monitoring results, or monitoring data, of the performance of the first model to be monitored. This allows the model provider to obtain detailed operating data of the first model, helping the model provider fully understand the operating status of the first model, optimizing the model, and improving the rationality of using the first model.

[0110] For another example, the first strategy can be used to indicate the threshold reached by reporting the performance of the first model. In other words, the first strategy can be used to indicate the monitoring results that reach the threshold among all the monitoring results of the performance of the first model to be monitored. As an example, when the threshold of the first model includes multiple thresholds, the first strategy can be used to indicate the performance of the first model that reaches the threshold and the threshold reached by the performance. In this way, the model provider can manage the first model in a timely manner according to the threshold reached by the performance of the first model, such as model update, so as to ensure the communication quality of the system. At the same time, reporting the performance of the first model that reaches the threshold and / or the threshold reached by the performance of the first model helps to save transmission resources. For the implementation method of using the control plane to monitor model performance, this reporting method can save control plane signaling.

[0111] The first policy instructs the terminal device to report the first information in multiple ways.

[0112] As an implementation, the first policy may explicitly indicate the manner in which the terminal device reports the first information. For example, the first policy may include the manner in which the terminal device reports the first information, which is simple to implement. For example, the first policy may include one bit, and different values ​​of the bit may represent different manners in which the terminal device reports the first information.

[0113] As another implementation, the first policy may implicitly indicate how the terminal device reports the first information. For example, the parameter information related to the performance monitoring of the first model may be implicitly indicated by the terminal device in terms of how the first information is reported. If the parameter information related to the performance monitoring of the first model indicated in the first policy includes information associated with the threshold of the model performance to be monitored, the first policy may indicate the threshold reached by reporting the performance of the first model. As an example, the first policy may include the threshold of the model performance to be monitored of the first model, and one or more of the following information of the first model: model identification; model performance to be monitored; monitoring period; and monitoring location. If the parameter information related to the performance monitoring of the first model indicated in the first policy does not include information associated with the threshold of the model performance to be monitored, the first policy may indicate reporting all monitoring results of the performance of the first model to be monitored. As an example, the first policy may include one or more of the following information of the first model: model identification; model performance to be monitored; monitoring period; and monitoring location. The first policy helps save resources by implicitly indicating how the terminal device reports the first information.

[0114] If the second network element is a session management network element, the first network element may include the first policy in a policy and charging control (PCC) rule and send the rule to the session management network element.

[0115] In some embodiments, before the first network element receives the first information reported by the second network element, the method shown in FIG4 may further include step S430.

[0116] In step S430, the first network element receives a request message (also referred to as a first request message) sent by the third network element, wherein the request message may be used to request performance monitoring of the first model.

[0117] The third network element may be a provider of the first model, or a device that generates the first model. The third network element may also be a device that can communicate with the provider of the first model. For example, the third network element may be a network element in the core network used to generate the model, such as a network function (NF). In other words, the first model may be generated by the third network element, or the first model may be provided by the third network element. For another example, the third network element may be an application function in the core network. As an example, if the first model is provided or generated by a third-party server, then the third network element may be an application function in the core network, wherein the third-party server can access the core network through the application function in the core network.

[0118] In some embodiments, the first request message may include model information of the first model, such as a model ID. In other embodiments, the first request message may also include model monitoring content, such as the performance of the model to be monitored.

[0119] The performance of the first model may vary in different scenarios and environments. For example, the performance of the first model may vary when used in different time periods or at different locations. To account for different usage requirements, the first request message may also include the monitoring period and monitoring location corresponding to the performance to be monitored of the first model.

[0120] In some embodiments, the first request message may further include a method for reporting the performance monitoring results of the first model. For example, the first request message may request reporting of all performance monitoring results of the first model, which helps to fully understand the operating status of the first model and optimize the model. For another example, the first request message may request reporting of monitoring results that meet a threshold among the performance monitoring results of the first model, which helps to save transmission resources.

[0121] The first request message may directly include the reporting method of the performance monitoring results of the first model, or may implicitly indicate the reporting method of the performance monitoring results of the first model through the parameters related to the performance of the first model included therein. For example, if the first request message includes the threshold of the model performance to be monitored of the first model, the first request message may request the reporting of the monitoring results that reach the threshold in the performance monitoring results of the first model, such as the threshold reached by the model performance to be monitored and the model performance that reaches the threshold. For another example, if the first request message does not include the threshold of the model performance to be detected of the first model, the first request message may request the reporting of all monitoring results of the performance of the first model.

[0122] The threshold mentioned here may be a threshold corresponding to a performance of the first model mentioned above, one threshold among multiple thresholds corresponding to a performance, and one or more of aggregated thresholds corresponding to multiple performances.

[0123] That is, the first request message may include one or more of the following information of the first model: model identification; model performance to be monitored; threshold of model performance to be monitored; monitoring period; and monitoring location.

[0124] In some embodiments, the method shown in FIG. 4 may further include step S440 .

[0125] In step S440, the first network element sends a response message to the request message (ie, the first request message mentioned above) to the third network element.

[0126] The response message may include monitoring results of the first model, such as the first information. For example, the response message may include one or more of the following information about the first model: a model identifier; performance of the first model during the monitoring period; a threshold reached by the first model performance; and a location where the first model performance was generated.

[0127] The response message may also include indication information of whether the request is completed. If the performance monitoring of the first model fails, or the performance monitoring result is invalid, etc., the response message may also include indication information that the request is not completed. For example, based on the indication information that the request is not completed, it can be determined whether to perform model monitoring of the first model again, or adjust the model monitoring parameters of the first model. If the performance monitoring of the first model is successful, or the performance monitoring result is valid, the response message may include indication information that the request is completed. For example, based on the indication information that the request has been completed, the first model can be managed based on the performance monitoring result of the first model, such as continuing to use, updating, deactivating, etc.

[0128] The number of response messages to the first request message can be one or more. If the number of response messages to the first request message is one, then when the model monitoring requested in the request message has been completed, or the monitoring result of the model performance requested in the request has been obtained, the response message is sent. If the number of response messages to the first request message is multiple, then, regardless of whether all the monitoring contents requested by the first request message have been completed, when the model monitoring result reaches the model performance threshold to be monitored in the request, a response message to the first request message is sent. In this way, in the case that some key parameters of the first model reach the threshold, the model can be managed in a timely manner, such as updating, deactivating, etc., which helps to improve the communication quality of the system.

[0129] As mentioned above, if the first information is carried in user-plane data, the first network element can receive the first information reported by the session management network element. As an implementation method, the core network session management network element can receive the first information reported by the terminal device through the user-plane function network element. For example, the user-plane function network element can receive the first information reported by the terminal device and report the received first information to the session management network element. The first information can be used to indicate the performance monitoring result of the first model used by the terminal device.

[0130] The first information may be used to indicate one or more of the following information about the first model: a model identifier; performance of the first model during a monitoring period; a threshold reached by the first model performance; and a location where the first model performance was generated. This information may be the same as the information described in step S410 and is not further described here for the sake of brevity.

[0131] In some embodiments, the first information may be carried in a header of a first data packet received by a user plane functional network element from a terminal device to implement model performance monitoring based on user plane data.

[0132] In some embodiments, before the user plane function network element receives the first information reported by the terminal device, the user plane function network element may also receive a request message (which may be referred to as a second request message) sent by the session management network element. For example, the second request message may be determined based on the first policy (the first policy described in step S420 above).

[0133] The second request message mentioned here may be related to performance monitoring of the first model. For example, the second request message may include one or more of the following information about the first model: model identification; model performance to be monitored; model performance threshold to be monitored; monitoring period; and monitoring location. The above information about the first model can be found in the previous description and will not be repeated here for simplicity.

[0134] For another example, the second request message may include a method for the terminal device to report the first information. The method for the terminal device to report the first information may be one or more of the methods for reporting the first information indicated in the first policy (the first policy described in step S420 above). As an example, the method for reporting the first information may include reporting all monitoring results of the performance of the first model to be monitored, and may also include reporting the threshold reached by the performance of the first model. The method for reporting the first information may be implicitly indicated through parameter information associated with the performance of the first model in the second request message, or may be directly indicated through the second request message, or may be called an explicit indication. For the sake of brevity, it will not be repeated here.

[0135] The following will provide a detailed introduction to this embodiment of the present application from the perspective of the terminal device.

[0136] In some embodiments, the terminal device may report the first information to a core network network element. The core network network element mentioned here may be the first network element mentioned above, or may be a user plane function network element. For example, the terminal device may report the first information to the session management function network element via the user plane function network element. The terminal device may also report the first information to the first network element via the user plane function network element and the session management function network element.

[0137] The first information may be used to indicate the performance monitoring results of the first model used by the terminal device. For example, the first information may be used to indicate one or more of the following information about the first model: a model identifier; the performance of the first model during a monitoring period; a threshold reached by the first model performance; and the location where the first model performance was generated. The above information about the first model can be found in the previous description and will not be repeated here for the sake of brevity.

[0138] The first information may be carried in a variety of ways. For example, the first information may be carried in control plane signaling of the core network. In another example, the first information may be carried in user plane data of the core network.

[0139] If the first information is carried in control plane signaling of the core network, then before the terminal device reports the first information to the network element of the core network, the terminal device may also receive a first policy sent by the first network element, where the first policy is related to performance monitoring of the first model. The first policy mentioned here is the same as the first policy mentioned in step S420 above and is not repeated here for the sake of brevity.

[0140] If the first information is carried in user plane data of the core network, then before the terminal device reports the first information to the network element of the core network, the terminal device may also accept a request message (which may be referred to as a third request message) sent by the session management network element. The third request message may be used to request the terminal device to perform performance monitoring on the first model. For example, the third request message may be determined based on the first policy (the first policy described in step S420 above).

[0141] In some embodiments, the third request message may include one or more of the following information about the first model: a model identifier; model performance to be monitored; a threshold for the model performance to be monitored; a monitoring period; and a monitoring location. The above information about the first model can be found in the previous description and is not further described here for brevity.

[0142] The third request message may include a manner in which the terminal device reports the first information. The manner in which the terminal device reports the first information may be one or more of the manners for reporting the first information indicated in the first policy. As an example, the manner in which the first information is reported may include reporting all monitoring results of the performance of the first model to be monitored, or may include reporting the threshold reached by the performance of the first model. The manner in which the first information is reported may be implicitly indicated through parameter information associated with the performance of the first model in the third request message, or may be directly indicated through the third request message, or may be referred to as an explicit indication. For the sake of brevity, it will not be elaborated here.

[0143] It should be noted that the third request message from the session management network element received by the terminal device may contain the same or different content as the second request message from the session management network element received by the user plane function network element. For example, the third request message may not include the threshold of the model performance to be monitored, while the second request message may include the threshold of the model performance to be monitored. For another example, both the second request message and the third request message may include the threshold of the model performance to be monitored, or both may not include the threshold of the model performance to be monitored.

[0144] It should be noted that if the first request message sent by the third network element and received by the first network element includes the threshold of the model performance to be monitored, the first strategy may include the threshold of the model performance to be monitored, or may not include the threshold of the model performance to be monitored.

[0145] If the terminal device reports all monitoring results of the first model's performance to be monitored, but the third network element requests the first network element to determine the threshold reached by the first model's performance, then all monitoring results of the first model's performance must be processed, such as to determine whether the first model's performance has reached the threshold. As one implementation, the first network element can determine whether the first model's performance has reached the threshold. As another implementation, the user plane function network element can determine whether the first model's performance has reached the threshold.

[0146] In addition, in the above judgment process, the first network element and / or the user plane function network element can determine whether the monitoring result reported by the terminal device is valid based on the monitoring period.

[0147] The following describes in detail the method of using the control plane to perform model performance monitoring, taking the first network element as the policy management function of the core network and the second network element as the terminal device as an example, in conjunction with FIG5 .

[0148] Figure 5 is a schematic diagram of a method for model monitoring using a control plane according to an embodiment of the present application. The method shown in Figure 5 may include steps S510a to S570b.

[0149] In step S510a, the application function of the core network sends a model performance monitoring request to the first network element.

[0150] When a model provider, such as a third-party server, wants to monitor the performance of a model used on a terminal device, it can send a model performance monitoring request to the first network element through an application function. The request may include the model ID to be monitored, the type of model performance to be monitored, the time period to start model performance monitoring, the monitoring location, and the threshold for reporting model performance.

[0151] In step S510b, the third network element sends a model performance monitoring request to the first network element.

[0152] When a model provider, such as a network element within the core network (a third network element), wants to monitor the performance of a model used on a terminal device, it can send a model performance monitoring request to the first network element. This request may include the model ID to be monitored, the type of model performance to be monitored, the time period to initiate model performance monitoring, the monitoring location, and the threshold for reporting model performance.

[0153] In step S520, the first network element generates a model performance monitoring strategy (ie, the first strategy mentioned above).

[0154] The first network element may generate a model performance monitoring strategy based on the model performance monitoring request, wherein the strategy may include the parameters of the model to be monitored included in the request of step S510a or step S510b.

[0155] In step S530, the first network element sends the model performance monitoring policy to the terminal device.

[0156] In step S540, the terminal device starts model performance monitoring.

[0157] The terminal device may start model performance monitoring based on the received model performance monitoring policy, within the time period and / or monitoring location indicated in the policy.

[0158] If the model performance monitoring strategy in step S530 does not include the model performance threshold parameter, S550a may be executed; if the model performance monitoring strategy in step S530 includes the model performance threshold parameter, S550b may be executed.

[0159] In step S550a, the terminal device reports the model performance monitoring result to the first network element.

[0160] The terminal device can report the model performance monitoring results to the first network element. The model performance results can include, for example, the model ID, the performance of the current model, etc.

[0161] After reporting the model performance monitoring result to the first network element, the terminal device may execute step S560a.

[0162] In step S550b, the terminal device reports the model performance monitoring result to the first network element.

[0163] The terminal device may determine whether the model performance monitoring result reaches the model performance threshold. When the model performance reaches the threshold, the terminal device may report the model performance monitoring result reaching the threshold to the first network element. The model performance result may include, for example, the model ID, the performance of the current model, and the threshold reached by the model performance.

[0164] In addition, the terminal device can determine whether the model performance monitoring result is valid based on the time period information of the model performance monitoring. That is, the model performance monitoring result is considered valid only if it reaches a threshold during the monitoring time period. The terminal device can report the valid monitoring result to the first network element. Otherwise, the terminal device can reply that the model performance monitoring request failed.

[0165] After reporting the model performance monitoring result to the first network element, the terminal device may execute step S560b.

[0166] In step S560a, the first network element determines whether the model performance reaches a threshold.

[0167] The first network element may compare the model performance reported by the terminal device with the threshold received in step S510a or step S510b, thereby determining whether the model performance reaches the threshold.

[0168] In addition, the first network element can determine whether the received response is valid based on the time period information of the model performance monitoring, that is, only the model performance monitoring results received within the time period are considered valid. Otherwise, the first network element will reply that the model performance monitoring request failed.

[0169] In step S560b, the first network element forwards the model performance monitoring result reported by the terminal device to the requester.

[0170] If the model monitoring request is initiated by an application function of the core network, step S570a may be executed next; if the model monitoring request is initiated by a third network element of the core network, step S570b may be executed next.

[0171] In step S570a, the first network element sends a reply message (ie, the response message mentioned above) to the model performance monitoring request to the application function.

[0172] The message may include information such as the model ID, the performance of the model during the monitoring period, the threshold reached by the model performance, and the location where the model performance was generated.

[0173] In step S570b, the first network element sends a reply message of the model performance monitoring request to the third network element.

[0174] The message may include information such as the model ID, the performance of the model during the monitoring period, the threshold reached by the model performance, and the location where the model performance was generated.

[0175] It should be noted that, in some cases, step S560b and step S570b have the same meaning, and only step S570b may be executed.

[0176] The following takes the first network element as the policy management function of the core network and the second network element as the SMF as an example, and combines Figure 6 to describe in detail the method of using user plane data to perform model performance monitoring.

[0177] Figure 6 is a schematic diagram of a method for model monitoring using user plane data according to an embodiment of the present application. The method shown in Figure 6 may include steps S610a to S690b.

[0178] In step S610a, the application function of the core network sends a model performance monitoring request to the first network element.

[0179] When a model provider, such as a third-party server, wants to monitor the performance of a model used on a terminal device, it can send a model performance monitoring request to the first network element through an application function. The request may include the model ID to be monitored, the type of model performance to be monitored, the time period to start model performance monitoring, the monitoring location, and the threshold for reporting model performance.

[0180] In step S610b, the third network element of the core network sends a model performance monitoring request to the first network element.

[0181] When a model provider, such as a network element within the core network (a third network element), wants to monitor the performance of a model used on a terminal device, it can send a model performance monitoring request to the first network element. This request may include the model ID to be monitored, the type of model performance to be monitored, the time period to start model performance monitoring, the monitoring location, and the threshold for reporting model performance.

[0182] In step S620, the first network element generates a model performance monitoring strategy (ie, the first strategy mentioned above).

[0183] The first network element may generate a model performance monitoring strategy based on the model performance monitoring request, wherein the strategy may include the parameters of the model to be monitored included in the request in step S610a or step S610b.

[0184] In step S630, the first network element sends the model monitoring policy to the session management network element.

[0185] For example, the first network element may include the model performance monitoring policy in a PCC rule and send it to the session management network element.

[0186] In step S640a, the session management network element sends a model performance monitoring request to the user plane function.

[0187] For example, the session management network element may send a model performance monitoring request to the user plane function via an N4 message. The request message may be determined based on the model performance monitoring policy in step S620.

[0188] In step S640b, the session management network element sends a model performance monitoring request to the terminal device.

[0189] For example, the session management network element may send a model performance monitoring request to the terminal device via a NAS message. The request message may be determined based on the model performance monitoring policy in step S620.

[0190] In some embodiments, the content of the model performance monitoring request sent by the session management network element to the terminal device and the model performance monitoring request sent to the user plane function may be the same or different.

[0191] In step S650, the terminal device starts model performance monitoring.

[0192] The terminal device can start its own model performance monitoring based on the parameters in the request received from the session management network element.

[0193] In step S660, the terminal device sends the model performance monitoring result to the user plane function.

[0194] For example, the terminal device can add the model performance monitoring result to the header of the uplink data packet and send it to the user plane function. As an example, the header of the data packet can also include information such as the model ID, the time and / or location of the monitoring result.

[0195] In step S670, the user plane function reports the model performance monitoring result to the session management network element.

[0196] First, the user plane function can obtain the model performance monitoring results reported by the terminal device by reading the parameters in the packet header.

[0197] If the terminal device reports a model performance monitoring result that meets the threshold, the user plane function can directly report the result to the session management network element. If the terminal device reports a model performance monitoring result that does not meet the threshold, the user plane function can determine whether the model performance meets the threshold based on the model performance monitoring request received from the session management network element and send the performance that meets the threshold to the session management network element.

[0198] In step S680, the session management network element sends the model performance monitoring result to the first network element.

[0199] In step S690a, the first network element sends a model performance monitoring request reply message (ie, the response message mentioned above) to the application function.

[0200] The message may include the model ID, the performance of the model in the time period, the threshold reached by the model performance, the location where the model performance was generated, etc.

[0201] In step S690b, the first network element sends a model performance monitoring request reply message (ie, the response message mentioned above) to the third network element.

[0202] The message may include the model ID, the performance of the model in the time period, the threshold reached by the model performance, the location where the model performance was generated, etc.

[0203] In an embodiment of the present application, core network network elements are used to collect performance parameters of the models used by terminal devices, thereby helping model providers monitor the performance of terminal device models. By sending the collected model performance parameters to the model providers via the control plane or user plane, the model providers can update the models in a timely manner, ensuring the quality of the communication system. Furthermore, by using the core network network elements as a centralized collection node and identifying them with model IDs, it is not necessary for each model provider to maintain a constant connection with the terminal device for each model, thus saving signaling resources and improving efficiency.

[0204] The above describes the method embodiment of the present application in detail, and the following describes the device embodiment of the present application in detail. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so for parts not described in detail, reference can be made to the previous method embodiment.

[0205] FIG7 is a schematic structural diagram of a core network element provided in an embodiment of the present application. The core network element 700 may be a first network element. The core network element 700 may include a first receiving unit 710.

[0206] The first receiving unit 710 is configured to receive first information reported by a second network element, where the first information is used to indicate a performance monitoring result of a first model used by the terminal device.

[0207] In some embodiments, the first information is carried in control plane signaling or user plane data of the core network.

[0208] In some embodiments, the core network element further includes: a first sending unit, configured to send a first policy to the second network element before receiving the first information reported by the second network element, where the first policy is related to performance monitoring of the first model.

[0209] In some embodiments, the first policy is used to indicate parameter information related to performance monitoring of the first model, and / or the manner in which the terminal device reports the first information.

[0210] In some embodiments, the first strategy includes one or more of the following information of the first model: a model identifier; a model performance to be monitored; a threshold of the model performance to be monitored; a monitoring period; and a monitoring location.

[0211] In some embodiments, the core network element further includes: a second receiving unit, configured to receive a request message sent by the third network element before receiving the first information reported by the second network element, wherein the request message is used to request performance monitoring of the first model.

[0212] In some embodiments, the core network element further includes: a second sending unit, configured to send a response message to the request message to the third network element, wherein the response message includes the first information.

[0213] In some embodiments, the request message includes one or more of the following information of the first model: a model identifier; a model performance to be monitored; a threshold of the model performance to be monitored; a monitoring period; and a monitoring location.

[0214] In some embodiments, the third network element is a network element in the core network used to generate a model; or, the third network element is an application function in the core network.

[0215] In some embodiments, the first information is used to indicate one or more of the following information of the first model: model identification; first model performance within a monitoring period; a threshold reached by the first model performance; and a location where the first model performance is generated.

[0216] In some embodiments, the core network element is a policy control network element in the core network.

[0217] FIG8 is a schematic structural diagram of another core network element provided in an embodiment of the present application. Core network element 800 may be a third network element. Core network element 800 may include a sending unit 810.

[0218] The sending unit 810 is used to send a request message to the first network element, where the request message is used to request performance monitoring of the first model used by the terminal device.

[0219] In some embodiments, the core network element further includes: a receiving unit for receiving a response message to the request message sent by the first network element, the response message including first information, and the first information is used to indicate a performance monitoring result of the first model.

[0220] In some embodiments, the first information includes one or more of the following information of the first model: model identification; first model performance during a monitoring period; a threshold reached by the first model performance; and a location where the first model performance is generated.

[0221] In some embodiments, the request message includes one or more of the following information of the first model: a model identifier; a model performance to be monitored; a threshold of the model performance to be monitored; a monitoring period; and a monitoring location.

[0222] In some embodiments, the core network element is a network element in the core network used to generate a model; or, the core network element is an application function in the core network.

[0223] FIG9 is a schematic structural diagram of another core network element provided in an embodiment of the present application. The core network element 900 may be a user plane function element of the core network. The core network element 900 may include a first receiving unit 910.

[0224] The first receiving unit 910 is configured to receive first information reported by a terminal device, where the first information is used to indicate a performance monitoring result of a first model used by the terminal device.

[0225] In some embodiments, the first information is carried in a header of a first data packet received by the core network element from the terminal device.

[0226] In some embodiments, the core network element further includes: a second receiving unit, which is used to receive a request message sent by the session management network element before receiving the first information reported by the terminal device, and the request message is related to the performance monitoring of the first model.

[0227] In some embodiments, the request message includes one or more of the following information of the first model: a model identifier; a model performance to be monitored; a threshold of the model performance to be monitored; a monitoring period; and a monitoring location.

[0228] FIG10 is a schematic structural diagram of a terminal device provided in an embodiment of the present application. The terminal device 1000 may include a reporting unit 1010 .

[0229] The reporting unit 1010 is used to report first information to a network element of the core network, where the first information is used to indicate a performance monitoring result of a first model used by the terminal device.

[0230] In some embodiments, the first information is carried in control plane signaling or user plane data of the core network.

[0231] In some embodiments, the terminal device further includes: a first receiving unit, the first receiving unit being used to receive a first policy sent by the first network element before reporting the first information to the network element of the core network, wherein the first policy is related to performance monitoring of the first model.

[0232] In some embodiments, the first policy is used to indicate parameter information related to performance monitoring of the first model, and / or the manner in which the terminal device reports the first information.

[0233] In some embodiments, the first strategy includes one or more of the following information of the first model: a model identifier; a model performance to be monitored; a threshold of the model performance to be monitored; a monitoring period; and a monitoring location.

[0234] In some embodiments, if the first information is carried in the user plane data of the core network, the terminal device also includes: a second receiving unit, used to receive a request message sent by the session management network element before reporting the first information to the network element of the core network, and the request message is used to request the terminal device to perform performance monitoring on the first model.

[0235] In some embodiments, the request message includes one or more of the following information of the first model: a model identifier; a model performance to be monitored; a threshold of the model performance to be monitored; a monitoring period; and a monitoring location.

[0236] In some embodiments, the network element of the core network is a user plane function network element.

[0237] In some embodiments, the first information is carried in a header of a first data packet sent by the terminal device to the user plane function network element.

[0238] In some embodiments, the first information is used to indicate one or more of the following information of the first model: model identification; first model performance within a monitoring period; a threshold reached by the first model performance; and a location where the first model performance is generated.

[0239] Figure 11 is a schematic block diagram of a communication device according to an embodiment of the present application. Dashed lines in Figure 11 indicate that the unit or module is optional. Device 1100 may be used to implement the method described in the above method embodiment. Device 1100 may be a chip, a terminal device, or a core network device.

[0240] The device 1100 may include one or more processors 1110. The processor 1110 may support the device 1100 to implement the method described in the method embodiment above. The processor 1110 may be a general-purpose processor or a special-purpose processor. For example, the processor may be a central processing unit (CPU). Alternatively, the processor may be another general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0241] The apparatus 1100 may further include one or more memories 1120. The memories 1120 store programs that can be executed by the processor 1110, causing the processor 1110 to perform the methods described in the above method embodiments. The memories 1120 may be independent of the processor 1110 or integrated into the processor 1110.

[0242] The apparatus 1100 may further include a transceiver 1130. The processor 1110 may communicate with other devices or chips via the transceiver 1130. For example, the processor 1110 may transmit and receive data with other devices or chips via the transceiver 1130.

[0243] The present application also provides a computer-readable storage medium for storing a program. The computer-readable storage medium can be applied to the core network element and / or terminal device provided in the embodiments of the present application, and the program causes a computer to execute the methods performed by the core network element and / or terminal device in various embodiments of the present application.

[0244] The present application also provides a computer program product. This computer program product includes a program. This computer program product can be applied to the core network element and / or terminal device provided in the embodiments of the present application, and the program causes a computer to execute the methods performed by the core network element and / or terminal device in various embodiments of the present application.

[0245] The present application also provides a computer program that can be applied to the core network element and / or terminal device provided in the present application, and enables a computer to execute the methods performed by the core network element and / or terminal device in various embodiments of the present application.

[0246] It should be understood that the terms "system" and "network" in this application can be used interchangeably. In addition, the terms used in this application are only used to explain the specific embodiments of this application and are not intended to limit this application. The terms "first", "second", "third", and "fourth" in the specification and claims of this application and the accompanying drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0247] In the embodiments of this application, the term "indication" may refer to a direct indication, an indirect indication, or an indication of an association. For example, "A indicates B" may refer to a direct indication of B, e.g., B can obtain information through A; it may refer to an indirect indication of B, e.g., A indicates C, e.g., B can obtain information through C; or it may refer to an association between A and B.

[0248] In the embodiment of the present application, "B corresponding to A" means that B is associated with A and B can be determined based on A. However, it should be understood that determining B based on A does not mean determining B based solely on A, but B can also be determined based on A and / or other information.

[0249] In the embodiments of the present application, the term "corresponding" may indicate a direct or indirect correspondence between the two, or an association relationship between the two, or a relationship between indication and indication, configuration and configuration, etc.

[0250] In the embodiments of the present application, "pre-definition" or "pre-configuration" may be implemented by pre-storing corresponding codes, tables, or other methods that can be used to indicate relevant information in a device (e.g., a terminal device and a network device). The present application does not limit the specific implementation method. For example, pre-definition may refer to information defined in a protocol.

[0251] In the embodiments of the present application, the “protocol” may refer to a standard protocol in the communications field, for example, it may include an LTE protocol, an NR protocol, and related protocols used in future communication systems, and the present application does not limit this.

[0252] In the embodiments of this application, the term "and / or" is simply a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0253] In various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0254] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0255] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0256] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0257] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be read by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0258] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A wireless communication method, characterized in that: include: The first network element receives first information reported by the second network element, where the first information is used to indicate a performance monitoring result of a first model used by a terminal device.

2. The method according to claim 1, characterized in that The first information is carried in control plane signaling or user plane data of the core network.

3. The method according to claim 1 or 2, characterized in that: Before the first network element receives the first information reported by the second network element, the method further includes: The first network element sends a first policy to the second network element, where the first policy is related to performance monitoring of the first model.

4. The method according to claim 3, characterized in that The first strategy is used to indicate parameter information related to performance monitoring of the first model and / or the manner in which the second network element reports the first information.

5. The method according to claim 3 or 4, wherein the first strategy includes one or more of the following information of the first model: Model identification; The model performance to be monitored; The threshold of model performance to be monitored; Monitoring period; and Monitoring location.

6. The method according to any one of claims 1 to 5, characterized in that Before the first network element receives the first information reported by the second network element, the method further includes: The first network element receives a request message sent by a third network element, where the request message is used to request performance monitoring of the first model.

7. The method according to claim 6, characterized in that The method further comprises: The first network element sends a response message to the request message to the third network element, where the response message includes the first information.

8. The method according to claim 6 or 7, characterized in that: The request message includes one or more of the following information of the first model: Model identification; The model performance to be monitored; The threshold of model performance to be monitored; Monitoring period; and Monitoring location.

9. The method according to any one of claims 6 to 8, characterized in that: The third network element is a network element in the core network used to generate a model; or, the third network element is an application function in the core network.

10. The method according to any one of claims 1 to 9, characterized in that The first information is used to indicate one or more of the following information of the first model: Model identification; first model performance during the monitoring period; a threshold reached by the performance of the first model; and The location of the first model performance is generated.

11. The method according to any one of claims 1 to 10, characterized in that The second network element is the terminal device or a session management network element.

12. The method according to any one of claims 1 to 11, characterized in that The first network element is a policy control network element in the core network.

13. A wireless communication method, characterized in that: include: The third network element sends a request message to the first network element, where the request message is used to request performance monitoring of a first model used by the terminal device.

14. The method according to claim 13, characterized in that The method further comprises: The third network element receives a response message to the request message sent by the first network element, where the response message includes first information, and the first information is used to indicate a performance monitoring result of the first model.

15. The method according to claim 14, characterized in that The first information includes one or more of the following information of the first model: Model identification; first model performance during the monitoring period; a threshold reached by the performance of the first model; and The location of the first model performance is generated.

16. The method according to any one of claims 13 to 15, characterized in that The request message includes one or more of the following information of the first model: Model identification; The model performance to be monitored; The threshold of model performance to be monitored; Monitoring period; and Monitoring location.

17. The method according to any one of claims 13 to 16, characterized in that The third network element is a network element in the core network used to generate a model; or, the third network element is an application function in the core network.

18. A wireless communication method, characterized in that: include: The user plane functional network element receives first information reported by a terminal device, where the first information is used to indicate a performance monitoring result of a first model used by the terminal device.

19. The method according to claim 18, characterized in that The first information is carried in the header of a first data packet received by the user plane functional network element from the terminal device.

20. The method according to claim 18 or 19, characterized in that Before the user plane function network element receives the first information reported by the terminal device, the method further includes: The user plane function network element receives a request message sent by a session management network element, where the request message is related to performance monitoring of the first model.

21. The method according to claim 20, characterized in that The request message includes one or more of the following information of the first model: Model identification; The model performance to be monitored; The threshold of model performance to be monitored; Monitoring period; and Monitoring location.

22. A wireless communication method, characterized in that: include: The terminal device reports first information to a network element of the core network, where the first information is used to indicate a performance monitoring result of a first model used by the terminal device.

23. The method according to claim 22, characterized in that The first information is carried in control plane signaling or user plane data of the core network.

24. The method according to claim 23, characterized in that If the first information is carried in control plane signaling of the core network, then before the terminal device reports the first information to a network element of the core network, the method further includes: The terminal device receives a first policy sent by a first network element, where the first policy is related to performance monitoring of the first model.

25. The method according to claim 24, characterized in that The first strategy is used to indicate parameter information related to performance monitoring of the first model and / or the manner in which the terminal device reports the first information.

26. The method according to claim 24 or 25, characterized in that The first strategy includes one or more of the following information of the first model: Model identification; The model performance to be monitored; The threshold of model performance to be monitored; Monitoring period; and Monitoring location.

27. The method according to claim 23, characterized in that If the first information is carried in user plane data of the core network, before the terminal device reports the first information to a network element of the core network, the method further includes: The terminal device receives a request message sent by a session management network element, where the request message is used to request the terminal device to perform performance monitoring on the first model.

28. The method according to claim 27, characterized in that The request message includes one or more of the following information of the first model: Model identification; The model performance to be monitored; The threshold of model performance to be monitored; Monitoring period; and Monitoring location.

29. The method according to claim 23, characterized in that The network element of the core network is a user plane function network element.

30. The method according to claim 29, characterized in that The first information is carried in the header of the first data packet sent by the terminal device to the user plane functional network element.

31. The method according to any one of claims 22 to 30, characterized in that The first information is used to indicate one or more of the following information of the first model: Model identification; first model performance during the monitoring period; a threshold reached by the performance of the first model; and The location of the first model performance is generated.

32. A core network element, characterized in that: The core network element is a first network element, and the core network element includes: The first receiving unit is used to receive first information reported by the second network element, where the first information is used to indicate a performance monitoring result of a first model used by a terminal device.

33. The core network element according to claim 32, characterized in that: The first information is carried in control plane signaling or user plane data of the core network.

34. The core network element according to claim 32 or 33, characterized in that: The core network element also includes: The first sending unit is used to send a first policy to the second network element before receiving the first information reported by the second network element, where the first policy is related to performance monitoring of the first model.

35. The core network element according to claim 34, characterized in that: The first strategy is used to indicate parameter information related to performance monitoring of the first model and / or the manner in which the second network element reports the first information.

36. According to the core network element of claim 34 or 35, the first strategy includes one or more of the following information of the first model: Model identification; The model performance to be monitored; The threshold of model performance to be monitored; Monitoring period; and Monitoring location.

37. The core network element according to any one of claims 32 to 36, characterized in that: The core network element also includes: The second receiving unit is used to receive a request message sent by a third network element before receiving the first information reported by the second network element, where the request message is used to request performance monitoring of the first model.

38. The core network element according to claim 37, characterized in that: The core network element also includes: The second sending unit is used to send a response message of the request message to the third network element, and the response message includes the first information.

39. The core network element according to claim 37 or 38, characterized in that: The request message includes one or more of the following information of the first model: Model identification; The model performance to be monitored; The threshold of model performance to be monitored; Monitoring period; and Monitoring location.

40. The core network element according to any one of claims 37 to 39, characterized in that: The third network element is a network element in the core network used to generate a model; or, the third network element is an application function in the core network.

41. The core network element according to any one of claims 32 to 40, characterized in that: The first information is used to indicate one or more of the following information of the first model: Model identification; first model performance during the monitoring period; a threshold reached by the performance of the first model; and The location of the first model performance is generated.

42. The core network element according to any one of claims 32 to 41, characterized in that: The second network element is the terminal device or a session management network element.

43. The core network element according to any one of claims 32 to 42, characterized in that: The first network element is a policy control network element in the core network.

44. A core network element, characterized in that: The core network element is a third network element, and the core network element includes: A sending unit is used to send a request message to a first network element, where the request message is used to request performance monitoring of a first model used by a terminal device.

45. The core network element according to claim 44, characterized in that: The core network element also includes: A receiving unit is used to receive a response message to the request message sent by the first network element, wherein the response message includes first information, and the first information is used to indicate a performance monitoring result of the first model.

46. ​​The core network element according to claim 45, characterized in that: The first information includes one or more of the following information of the first model: Model identification; first model performance during the monitoring period; a threshold reached by the performance of the first model; and The location of the first model performance is generated.

47. The core network element according to any one of claims 44 to 46, characterized in that: The request message includes one or more of the following information of the first model: Model identification; The model performance to be monitored; The threshold of model performance to be monitored; Monitoring period; and Monitoring location.

48. The core network element according to any one of claims 44 to 47, characterized in that: The core network element is a network element in the core network used to generate a model; or, the core network element is an application function in the core network.

49. A core network element, characterized in that: The core network element is a user plane function network element, and the core network element includes: The first receiving unit is used to receive first information reported by a terminal device, where the first information is used to indicate a performance monitoring result of a first model used by the terminal device.

50. The core network element according to claim 49, characterized in that: The first information is carried in the header of a first data packet received by the user plane functional network element from the terminal device.

51. The core network element according to claim 49 or 50, characterized in that: The core network element also includes: The second receiving unit is used to receive a request message sent by a session management network element before receiving the first information reported by the terminal device, where the request message is related to the performance monitoring of the first model.

52. The core network element according to claim 51, characterized in that: The request message includes one or more of the following information of the first model: Model identification; The model performance to be monitored; The threshold of model performance to be monitored; Monitoring period; and Monitoring location.

53. A terminal device, characterized in that: include: A reporting unit is used to report first information to a network element of a core network, wherein the first information is used to indicate a performance monitoring result of a first model used by a terminal device.

54. The terminal device according to claim 53, characterized in that: The first information is carried in control plane signaling or user plane data of the core network.

55. The terminal device according to claim 54, characterized in that: If the first information is carried in control plane signaling of the core network, the terminal device further includes: The first receiving unit is used to receive a first policy sent by a first network element before reporting the first information to the network element of the core network, where the first policy is related to performance monitoring of the first model.

56. The terminal device according to claim 55, characterized in that: The first strategy is used to indicate parameter information related to performance monitoring of the first model and / or the manner in which the terminal device reports the first information.

57. The terminal device according to claim 55 or 56, characterized in that: The first strategy includes one or more of the following information of the first model: Model identification; The model performance to be monitored; The threshold of model performance to be monitored; Monitoring period; and Monitoring location.

58. The terminal device according to claim 54, characterized in that: If the first information is carried in user plane data of the core network, the terminal device further includes: The second receiving unit is used to receive a request message sent by a session management network element before reporting the first information to the network element of the core network, wherein the request message is used to request the terminal device to perform performance monitoring on the first model.

59. The terminal device according to claim 58, characterized in that: The request message includes one or more of the following information of the first model: Model identification; The model performance to be monitored; The threshold of model performance to be monitored; Monitoring period; and Monitoring location.

60. The terminal device according to claim 54, characterized in that: The network element of the core network is a user plane function network element.

61. The terminal device according to claim 60, characterized in that: The first information is carried in the header of the first data packet sent by the terminal device to the user plane functional network element.

62. The terminal device according to any one of claims 53 to 61, characterized in that: The first information is used to indicate one or more of the following information of the first model: Model identification; first model performance during the monitoring period; a threshold reached by the performance of the first model; and The location of the first model performance is generated.

63. A core network element, characterized in that: The invention comprises a memory and a processor, wherein the memory is used to store a program, and the processor is used to call the program in the memory to execute the method according to any one of claims 1 to 21.

64. A terminal device, characterized in that: It comprises a memory and a processor, wherein the memory is used to store a program, and the processor is used to call the program in the memory to execute the method as claimed in any one of claims 22 to 31.

65. A chip, characterized in that: It comprises a processor, which is used to call a program from a memory, so that a device equipped with the chip executes a method as claimed in any one of claims 1 to 21.

66. A chip, characterized in that: It comprises a processor, which is used to call a program from a memory, so that a device equipped with the chip executes a method as claimed in any one of claims 22 to 31.

67. A computer-readable storage medium, characterized in that A program is stored thereon, the program causing a computer to execute the method according to any one of claims 1 to 21.

68. A computer-readable storage medium, characterized in that A program is stored thereon, the program causing a computer to execute the method according to any one of claims 22 to 31.

69. A computer program product, characterized in that The method comprises a program which causes a computer to execute the method according to any one of claims 1 to 21.

70. A computer program product, characterized in that A program is included, which causes a computer to execute the method as claimed in any one of claims 22 to 31.

71. A computer program, characterized in that The computer program enables a computer to execute the method according to any one of claims 1 to 21.

72. A computer program, characterized in that The computer program causes a computer to execute the method according to any one of claims 22 to 31.

Images