Information transmission method and apparatus, and related device

Abstract

The present application belongs to the field of communication computation. Disclosed are an information transmission method and apparatus, and a related device. The information transmission method in the embodiments of the present application comprises: a first device sending first information to a second device, wherein the first information is used for requesting performance monitoring performed on an artificial intelligence (AI) service; and the first device acquiring second information sent by the second device, wherein the second information comprises information related to a performance monitoring result of the AI service or information related to a performance monitoring truth value of the AI service, the first device comprises a terminal, an application function (AF) or an OTT server, and the second device comprises a network device.

Description

Information transmission methods, devices and related equipment

[0001] Cross-references

[0002] This disclosure claims priority to Chinese Patent Application No. 202411619836.X, filed on November 13, 2024, entitled "Information Transmission Method, Apparatus and Related Equipment", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application belongs to the field of communication technology, specifically relating to an information transmission method, apparatus and related equipment. Background Technology

[0004] Currently, the application scenarios of Artificial Intelligence (AI) at the terminal are generally divided into local terminal inference and cloud-based inference. When the terminal performs AI unit inference, if the terminal can collect the truth value, the inference accuracy of the AI ​​unit can be ensured. However, in some scenarios, due to limitations such as perspective, the terminal cannot obtain the complete truth value, thus making it impossible to monitor the performance of the AI ​​unit's model inference and, consequently, to know whether the inference accuracy of the AI ​​unit has deteriorated. Summary of the Invention

[0005] This application provides an information transmission method, apparatus, and related equipment, which can solve the problem that in some scenarios, the terminal cannot monitor the performance of the AI ​​unit's model inference, and thus cannot know whether the inference accuracy of the AI ​​unit has deteriorated.

[0006] Firstly, an information transmission method is provided, the method comprising:

[0007] The first device sends a first message to the second device, the first message being used to request performance monitoring of the artificial intelligence (AI) service.

[0008] The first device acquires the second information sent by the second device, the second information including information related to the performance monitoring results of the AI ​​service or information related to the true value of the performance monitoring of the AI ​​service;

[0009] The first device includes a terminal, an application function (AF) or an OTT server; the second device includes network equipment.

[0010] Secondly, an information transmission method is provided, the method comprising:

[0011] The second device obtains the first information sent by the first device, which is used to request performance monitoring of the artificial intelligence (AI) service.

[0012] The second device sends second information to the first device, the second information including information related to the performance monitoring results of the AI ​​service or information related to the true value of the performance monitoring of the AI ​​service;

[0013] The first device includes a terminal, an application function (AF) or an OTT server; the second device includes network equipment.

[0014] Thirdly, an information transmission device is provided, comprising:

[0015] The first sending module is used to send first information to the second device, the first information being used to request performance monitoring of artificial intelligence (AI) services;

[0016] The first receiving module is used to obtain the second information sent by the second device, the second information including information related to the performance monitoring result of the AI ​​service or information related to the performance monitoring truth value of the AI ​​service.

[0017] The second device includes network devices.

[0018] Fourthly, an information transmission device is provided, comprising:

[0019] The second receiving module is used to obtain first information sent by the first device, the first information being used to request performance monitoring of artificial intelligence (AI) services.

[0020] The fourth sending module is used to send second information to the first device, the second information including information related to the performance monitoring results of the AI ​​service or information related to the performance monitoring truth value of the AI ​​service;

[0021] The first device includes a terminal, an application function (AF) or an OTT server.

[0022] Fifthly, an information transmission apparatus is provided, the apparatus being configured to perform the steps of the method described in the first aspect, or to implement the steps of the method described in the second aspect.

[0023] In a sixth aspect, a terminal is provided, the terminal including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method as described in the first aspect.

[0024] In a seventh aspect, a terminal is provided, including a processor and a communication interface, wherein the communication interface is used to send first information to a second device, the first information being used to request performance monitoring of an artificial intelligence (AI) service; and to obtain second information sent by the second device, the second information including information related to the performance monitoring result of the AI ​​service or information related to the performance monitoring truth value of the AI ​​service.

[0025] Eighthly, a network-side device is provided, the network-side device including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method as described in the second aspect.

[0026] In a ninth aspect, a network-side device is provided, including a processor and a communication interface, wherein the communication interface is used to acquire first information sent by a first device, the first information being used to request performance monitoring of an artificial intelligence (AI) service; and to send second information to the first device, the second information including information related to the performance monitoring result of the AI ​​service or information related to the performance monitoring truth value of the AI ​​service.

[0027] In a tenth aspect, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect, or implement the steps of the method described in the second aspect.

[0028] Eleventhly, a wireless communication system is provided, comprising: a terminal and a network-side device, wherein the terminal can be used to perform the steps of the method as described in the first aspect, and the network-side device can be used to perform the steps of the method as described in the second aspect.

[0029] In a twelfth aspect, a chip is provided, the chip including a processor and a communication interface coupled to the processor, the processor being configured to run programs or instructions to implement the method as described in the first aspect, or to implement the method as described in the second aspect.

[0030] In a thirteenth aspect, a computer program / program product is provided, the computer program / program product being stored in a storage medium, the computer program / program product being executed by at least one processor to implement the steps of the method as described in the first or second aspect.

[0031] In this embodiment, a first device sends first information to a second device, the first information being a request for performance monitoring of an artificial intelligence (AI) service. The first device then receives second information sent by the second device, the second information including information related to the performance monitoring results of the AI ​​service or information related to the performance monitoring truth value of the AI ​​service. The first device includes a terminal, an application function (AF) server, or an OTT server; the second device includes a network device. In this scheme, the network provides the first device with information related to the performance monitoring results of the AI ​​service or information related to the performance monitoring truth value. This allows the first device to monitor the model inference of the AI ​​unit based on the information provided by the network, thereby determining whether the model inference accuracy of the AI ​​unit has deteriorated, and thus ensuring the inference performance of the AI ​​unit related to the AI ​​service requested by the first device. Attached Figure Description

[0032] Figure 1 shows a structural diagram of a communication system applicable to an embodiment of this application;

[0033] Figure 2 illustrates the interaction diagram of the network providing authentication services;

[0034] Figure 3 shows one of the flowcharts of the information transmission method according to an embodiment of this application;

[0035] Figure 4 shows one of the flowcharts of the information transmission method according to an embodiment of this application;

[0036] Figure 5a shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0037] Figure 5b shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0038] Figure 5c shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0039] Figure 6a shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0040] Figure 6b shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0041] Figure 6c shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0042] Figure 6d shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0043] Figure 7a shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0044] Figure 7b shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0045] Figure 7c shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0046] Figure 7d shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0047] Figure 7e shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0048] Figure 7f shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0049] Figure 7g illustrates the interaction of information transmission methods in some embodiments of this application;

[0050] Figure 8a shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0051] Figure 8b shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0052] Figure 8c shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0053] Figure 8d shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0054] Figure 9a shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0055] Figure 9b shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0056] Figure 9c shows an interactive schematic diagram of the information transmission method in some embodiments of this application;

[0057] Figure 10 shows a schematic diagram of one of the modules of the information transmission device according to an embodiment of this application;

[0058] Figure 11 shows a second schematic diagram of the information transmission device according to an embodiment of this application;

[0059] Figure 12 shows a structural block diagram of a communication device according to an embodiment of this application;

[0060] Figure 13 shows a structural block diagram of the terminal according to an embodiment of this application;

[0061] Figure 14 shows a structural block diagram of one of the network-side devices according to an embodiment of this application;

[0062] Figure 15 shows a second structural block diagram of the network-side device according to an embodiment of this application. Detailed Implementation

[0063] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0064] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, the first object can be one or more. Furthermore, "or" in this application indicates at least one of the connected objects. For example, the scope of protection for "A or B" covers at least three scenarios: Scenario 1: including A but not B; Scenario 2: including B but not A; Scenario 3: including both A and B. In addition, the terms "A and / or B," "at least one of A and B," and "at least one of A or B" also cover at least the above three scenarios. The character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0065] The term "instruction" in this application can be either a direct instruction (or explicit instruction) or an indirect instruction (or implicit instruction). A direct instruction can be understood as one in which the sender explicitly informs the receiver of specific information, the operation to be performed, or the requested result, etc., in the instruction sent. An indirect instruction can be understood as one in which the receiver determines the corresponding information based on the instruction sent by the sender, or makes a judgment and determines the operation to be performed or the requested result, etc., based on the judgment result.

[0066] It is worth noting that the technologies described in this application are not limited to Long Term Evolution (LTE) / LTE-Advanced (LTE-A) systems, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA), or other systems. The terms "system" and "network" in this application are often used interchangeably, and the described technologies can be used with the systems and radio technologies mentioned above, as well as with other systems and radio technologies. The following description describes New Radio (NR) systems for illustrative purposes, and the term NR is used in most of the following description; however, these technologies can also be applied to systems other than NR systems, such as 6th generation (6G) radio systems. th Generation 6G communication system.

[0067] Figure 1 shows a block diagram of a wireless communication system applicable to an embodiment of this application. The wireless communication system includes a terminal 11 and a network-side device 12. The terminal 11 can be a mobile phone, tablet computer, laptop computer, notebook computer, personal digital assistant (PDA), handheld computer, netbook, ultra-mobile personal computer (UMPC), mobile internet device (MID), augmented reality (AR), virtual reality (VR) device, robot, wearable device, flight vehicle, vehicle user equipment (VUE), shipboard equipment, pedestrian user equipment (PUE), smart home (home devices with wireless communication capabilities, such as refrigerators, televisions, washing machines, or furniture), game console, personal computer (PC), ATM, or self-service machine, etc. Wearable devices include: smartwatches, smart bracelets, smart headphones, smart glasses, smart jewelry (smart bracelets, smart chains, smart rings, smart necklaces, smart anklets, smart anklets, etc.), smart wristbands, smart clothing, etc. Among these, in-vehicle devices can also be referred to as in-vehicle terminals, in-vehicle controllers, in-vehicle modules, in-vehicle components, in-vehicle chips, or in-vehicle units, etc. It should be noted that the specific type of terminal 11 is not limited in this application embodiment. Network-side equipment 12 may include access network equipment or core network equipment, wherein access network equipment may also be referred to as Radio Access Network (RAN) equipment, radio access network function, or radio access network unit. Access network equipment may include base stations, Wireless Local Area Network (WLAN) access points (AS), or Wireless Fidelity (WiFi) nodes, etc.The term "base station" can be referred to as Node B (NB), Evolved Node B (eNB), Next Generation Node B (gNB), New Radio Node B (NR Node B), Access Point, Relay Base Station (RBS), Serving Base Station (SBS), Base Transceiver Station (BTS), Radio Base Station, Radio Transceiver, Basic Service Set (BSS), Extended Service Set (ESS), Home Node B (HNB), Home Evolved Node B, Transmit / Receive Point (TRP), or any other suitable term in the relevant field, as long as the same technical effect is achieved. The term "base station" is not limited to any specific technical terminology. It should be noted that this application embodiment only uses a base station in an NR system as an example for description and does not limit the specific type of base station.

[0068] Core network equipment, also known as core network nodes, core network functions, or core network elements, includes, but is not limited to, at least one of the following: Mobility Management Entity (MME), Access and Mobility Management Function (AMF), Session Management Function (SMF), User Plane Function (UPF), Policy Control Function (PCF), Policy and Charging Rules Function (PCRF), Edge Application Server Discovery Function (EASDF), Unified Data Management (UDM), Unified Data Repository (UDR), Home Subscriber Server (HSS), Centralized network configuration (CNC), Network Repository Function (NRF), Network Exposure Function (NEF), Local NEF (or L-NEF), and Binding Support. The core network functions include: BSF (Block Network Function), Application Function (AF), Location Management Function (LMF), Gateway Mobile Location Centre (GMLC), and Network Data Analytics Function (NWDAF). It should be noted that this application embodiment only uses core network equipment in the NR system as an example and does not limit the specific type of core network equipment. If the name of the core network equipment mentioned in this application embodiment changes in subsequent protocol versions (e.g., 6G), it will still be within the scope of protection of this application.

[0069] Optionally, the core network equipment can be implemented by one or more functional modules in a single device, or by multiple devices working together; this application does not specifically limit this. It is understood that the aforementioned functional modules can be network elements in hardware devices, software functional modules running on dedicated hardware, or virtualized functional modules instantiated on a platform (e.g., a cloud platform).

[0070] To enable those skilled in the art to better understand the embodiments of this application, the following description will be provided first.

[0071] The network provides verification services:

[0072] As shown in Figure 2, the requesting party sends a verification service request to the network and sends the input data of the AI ​​unit through the user plane. The network performs model inference in the AI ​​unit and feeds back the inference result of the AI ​​unit to the requesting party. The requesting party calculates and obtains the verification result.

[0073] The above process allows for the verification of the inference performance of the network-side AI unit based on the requester's dataset. However, this process is applicable only to scenarios where the requester can obtain the truth value. If the requester cannot obtain the truth value, but the network side can, then the above process is not suitable.

[0074] The information transmission method provided in this application will be described in detail below with reference to the accompanying drawings and through some embodiments and application scenarios.

[0075] As shown in Figure 3, this application embodiment provides an information transmission method, including:

[0076] Step 301: The first device sends a first message to the second device, the first message being used to request performance monitoring of the artificial intelligence (AI) service;

[0077] Step 302: The first device obtains the second information sent by the second device, the second information including information related to the performance monitoring results of the AI ​​service or information related to the performance monitoring truth value of the AI ​​service;

[0078] The first device includes a terminal, an application function (AF) or an OTT server; the second device includes a network device.

[0079] Optionally, the performance monitoring result related information of the AI ​​service includes the performance monitoring result of the first AI unit, or the performance monitoring truth value related information of the AI ​​service includes at least one of the performance monitoring truth value associated with the first AI unit and the performance monitoring truth value associated with the first AI function; wherein, the first AI unit is the AI ​​unit of the first device that performs model inference, and the first AI function is the AI ​​function corresponding to the first AI unit.

[0080] Alternatively, the performance monitoring results related to the AI ​​service may include the performance monitoring results of the third AI unit and the performance monitoring results associated with the second AI function; or, the performance monitoring truth value related to the AI ​​service may include the performance monitoring truth value associated with the third AI unit; the second AI function is the AI ​​function corresponding to the third AI unit.

[0081] The above performance monitoring truth value can also be described as monitoring truth value or truth value.

[0082] Optionally, the performance monitoring results include at least one of the following:

[0083] Performance monitoring of key performance indicators (KPIs);

[0084] The monitoring event, the second information mentioned above is sent when the monitoring event is met. By setting the monitoring event, the continuous transmission of the second information mentioned above can be avoided, effectively reducing signaling transmission overhead.

[0085] The timestamp is used to indicate the time corresponding to the information in the performance monitoring results.

[0086] Optionally, the first information includes at least one of the following:

[0087] The monitoring types include monitoring based on the inference results of the first AI unit, monitoring based on the third AI unit, and monitoring based on the truth value; the third AI unit is an AI unit related to the first AI unit.

[0088] The monitoring feedback methods include providing monitoring results, providing monitoring truth values, periodic feedback, and event-triggered feedback. For example, event-triggered feedback helps reduce interaction overhead, while periodic feedback helps obtain more complete monitoring information. In this embodiment, the above monitoring feedback methods can be dynamically requested based on the requester's different states (such as signal quality or battery level).

[0089] The monitoring period can be long or short. A long period helps reduce transmission overhead, while a short period helps obtain more complete monitoring information. In this embodiment, the above-mentioned monitoring period can be dynamically requested based on the different states of the requester (such as signal quality level, battery level).

[0090] Monitoring indicators;

[0091] Monitoring events;

[0092] Monitor parameters related to the event, such as the threshold for the event to be triggered.

[0093] In this embodiment, a first device sends first information to a second device, the first information being a request for performance monitoring of an artificial intelligence (AI) service. The first device then receives second information sent by the second device, the second information including information related to the performance monitoring results of the AI ​​service or information related to the performance monitoring truth value of the AI ​​service. The first device includes a terminal, an application function (AF) server, or an OTT server; the second device includes a network device. In this scheme, the network provides the first device with information related to the performance monitoring results of the AI ​​service or information related to the performance monitoring truth value. This allows the first device to monitor the model inference of the AI ​​unit based on the information provided by the network, thereby determining whether the model inference accuracy of the AI ​​unit has deteriorated, and thus ensuring the inference performance of the AI ​​unit related to the AI ​​service requested by the first device.

[0094] Optionally, after the first device sends the first information to the second device, the method further includes:

[0095] The first device sends first AI service data to the second device, the first AI service data including the model inference results of the first AI unit;

[0096] The second information is obtained based on the model reasoning results of the first AI unit.

[0097] In this embodiment, the network side obtains the above-mentioned performance monitoring result information or the performance monitoring truth value information of the AI ​​service based on the model inference result of the first AI unit. In this way, there is no need for the second device to send the truth value, avoiding the network side directly exposing the truth value. Moreover, the performance monitoring result is calculated by the second device, which can save the computing resources of the first device.

[0098] Optionally, the first information is further used to request the network side to perform model inference for the second AI unit; or, the method further includes:

[0099] The first device sends a third message to the second device, the third message being used to request the network side to perform model inference for the second AI unit;

[0100] Wherein, the second AI unit is an AI unit paired with the first AI unit, and the first AI unit is the AI ​​unit that performs model inference on the first device;

[0101] The second information is obtained based on the model reasoning results of the first AI unit and the model reasoning results of the second AI unit.

[0102] In this embodiment, the first information can be used to simultaneously request performance monitoring of the AI ​​service and request the network side to perform model inference for the second AI unit. Alternatively, the third information can be used to request the network side to perform model inference for the second AI unit, while the first information can be used to request the network to perform performance monitoring of the AI ​​service.

[0103] In this embodiment of the application, the second AI unit is an AI unit paired with the first AI unit, or it can be described as the second AI unit being aligned with the first AI unit, or it can be described as the second AI unit being matched with the first AI unit.

[0104] Optionally, the alignment of the first AI unit and the second AI unit may mean that the first AI unit and the second AI unit are sub-units of the target unit, or it may mean that when the first AI unit and the second AI unit perform joint reasoning, the reasoning accuracy meets the preset requirements. For example, the joint reasoning accuracy of the first AI unit and the second AI unit is greater than or equal to the first preset threshold, or the error of the joint reasoning is less than or equal to the second preset threshold.

[0105] Optionally, the input data of the second AI unit includes the model inference results of the first AI unit;

[0106] After the first device sends the first information or the third information to the second device, the method further includes:

[0107] The first device sends first AI service data to the second device, the first AI service data including the model inference results of the first AI unit.

[0108] In some embodiments of this application, a first device first performs model inference for a first AI unit to obtain the model inference result of the first AI unit, and a second device uses the model inference result of the first AI unit as input data for model inference of a second AI unit. Based on the model inference result of the first AI unit, the second device obtains the model inference result of the second AI unit.

[0109] Optionally, the input data of the first AI unit includes the model inference results of the second AI unit;

[0110] After the first device sends the first information or the third information to the second device, the method further includes:

[0111] The first device acquires the model inference results of the second AI unit sent by the second device;

[0112] The first device performs model reasoning of the first AI unit based on the model reasoning result of the second AI unit, and obtains the model reasoning result of the first AI unit;

[0113] The first device sends second AI service data to the second device. The second AI service data includes at least one of the following: the model inference result of the first AI unit; and a third AI unit, which is an AI unit related to the first AI unit.

[0114] As one implementation, the third AI unit serves as a proxy model for the first AI unit, or a reference model.

[0115] In some embodiments of this application, the second device first performs model inference on the second AI unit, and the first device then performs model inference on the first AI unit based on the model inference result of the second AI unit to obtain the model inference result of the first AI unit. It can be understood that the model inference result of the second AI unit has no physical meaning and is merely an intermediate parameter in the model inference process. This prevents the first AI unit from being used by other unauthorized requesters.

[0116] Optionally, the reasoning result of the second AI unit satisfies any one of the following conditions:

[0117] The reasoning result of the second AI unit does not have corresponding physical parameters;

[0118] The reasoning results of the second AI unit have corresponding physical parameters.

[0119] In one embodiment, when the output of the second AI unit has corresponding physical parameters, the first device can parse the physical parameters corresponding to the output of the second AI unit, while the second device cannot parse the physical parameters corresponding to the output of the second AI unit; or,

[0120] When the output of the second AI unit has corresponding physical parameters, both the first device and the second device can parse the physical parameters corresponding to the output of the second AI unit.

[0121] The reasoning result of the second AI unit may include any one of the following:

[0122] The output of the second AI unit has no physical meaning.

[0123] The physical meaning of the output of the second AI unit cannot be interpreted by the second device, but can be understood by the first device.

[0124] The physical meaning of the output of the second AI unit can be interpreted by both the second and first devices.

[0125] In one implementation, the reasoning result of the second AI unit is the output result of the second AI unit. This output result has no physical meaning, which helps to protect the second AI unit from being used by devices other than the first device and can be used to restrict the users of the second AI unit.

[0126] In one implementation, the reasoning result of the second AI unit is the output result of the second AI unit. The second device cannot interpret the physical meaning of the output result, but the first device can know the physical meaning of the output result. This is beneficial to protecting the second AI unit from being used by devices other than the first device. It can be used to restrict the users of the second AI unit. Compared with an output result without physical meaning, it is beneficial for the first device to perform verification of the transmission result.

[0127] In one implementation, the reasoning result of the second AI unit is the output result of the second AI unit. Both the second device and the first device can parse the physical meaning of the output result, which is beneficial for sharing the output result of the second AI unit with other requesters.

[0128] In this embodiment, the reasoning result of the second AI unit does not have corresponding physical parameters, so even if a device other than the first device obtains the reasoning result of the second AI unit, it cannot parse the reasoning result. This is beneficial to protect the second AI unit from being used by devices other than the first device, thereby limiting the users of the second AI unit.

[0129] In this embodiment, the reasoning result of the second AI unit has corresponding physical parameters, which is beneficial for the first device to verify whether the reasoning result of the second AI unit is accurate, and can better support sharing the second AI unit with other devices.

[0130] Optionally, when the first device sends the third information to the second device, the first information includes:

[0131] The first identifier of the model inference task associated with the performance monitoring task of the first information request.

[0132] Based on this first identifier, the second device is able to associate the performance monitoring task of the first information request with the corresponding inference task.

[0133] Optionally, the first information is further used to request the second device to perform model inference for the second AI unit;

[0134] Before the first device sends the first information to the second device, the method further includes:

[0135] The first device sends a fourth message to the second device, the fourth message being used to request the network side to perform model inference for the second AI unit;

[0136] The first information includes task update information, which is used to replace the model inference task requested by the fourth information with the model inference and performance monitoring task requested by the first information.

[0137] It should be noted that the third and fourth information in this application embodiment have the following differences: after sending the first information, the second device stops executing the task requested by the fourth information, while after sending the first information, the second device continues to execute the task requested by the third information.

[0138] In some embodiments of this application, the first device may first send a fourth message requesting the network side to perform model inference for the second AI unit, and then send the first message, modifying the model inference task requested by the fourth message into a model inference and performance monitoring task, thereby achieving the purpose of flexibly changing the monitoring task.

[0139] Optionally, the first information includes at least one of the following:

[0140] The second AI unit;

[0141] Description information related to the second AI unit;

[0142] The data characteristic information of the second AI unit, for example, the data characteristic ID of the second AI unit.

[0143] As one implementation, the second AI unit includes at least one of the following:

[0144] The executable file corresponding to the second AI unit;

[0145] The reference model structure indication for the second AI unit;

[0146] The order of model parameters for the second AI unit;

[0147] The parameter file for the second AI unit;

[0148] Instructions from the second AI unit;

[0149] Modification layer instructions for the second AI unit;

[0150] The layer indicator corresponding to the second AI unit, for example, the second AI unit corresponds to the first 1 to K layers.

[0151] As one implementation, the descriptive information related to the second AI unit includes at least one of the following:

[0152] The effective duration of the second AI unit;

[0153] The configuration environment of the second AI unit includes: deep learning frameworks (TensorFlow, PyTorch, etc.), supported library versions, etc.

[0154] The second AI unit's model input information indicator is used to indicate data items, data format, data precision, etc.

[0155] The second AI unit's model output information indicator is used to indicate the accuracy, dimension, and storage format of the output data.

[0156] In this embodiment of the application, when the first information is used to request the network side to perform model inference of the second AI unit, the relevant information of the second AI unit is sent so that the network side can perform model inference of the second AI unit based on the relevant information of the second AI unit.

[0157] Optionally, before the first device receives the second information sent by the second device, the method further includes:

[0158] The first device sends a third AI unit to the second device. The third AI unit is an AI unit related to the first AI unit; for example, the third AI unit is a proxy model of the first AI unit, or a reference model.

[0159] The performance monitoring results related to the AI ​​service include the performance monitoring results of the third AI unit and the performance monitoring results associated with the second AI function; or, the performance monitoring truth value related to the AI ​​service includes the performance monitoring truth value associated with the third AI unit.

[0160] In this embodiment, the first device sends the aforementioned third AI unit to the second device, enabling the second device to obtain the performance monitoring result information of the aforementioned AI service based on the performance monitoring result of the third AI unit. The second device does not need to obtain the performance monitoring result information of the aforementioned AI service based on the inference result of the first AI unit, thereby avoiding the first device sending the inference result of the first AI unit to the second device and reducing signaling transmission overhead.

[0161] Optionally, the method in this application embodiment further includes:

[0162] The first device obtains the performance monitoring result of the first AI unit based on the performance monitoring result of the third AI unit or the performance monitoring true value associated with the third AI unit.

[0163] In this embodiment of the application, there is a fixed difference in inference accuracy between the third AI unit and the first AI unit, and this difference can be known on the first device side.

[0164] In this embodiment, the first device can obtain the difference in inference accuracy between the first AI unit and the third AI unit, and based on the difference and the performance monitoring result of the third AI unit, obtain the performance monitoring result of the first AI unit, and perform performance monitoring based on the performance monitoring result of the first AI unit to obtain the performance monitoring result of the AI ​​service.

[0165] Optionally, the first information is information transmitted between a first network function and / or a second network function in the first device and the second device; and / or,

[0166] The second information is information transmitted between the second network function in the first device and the second device; and / or,

[0167] The third information is the information transmitted between the first device and the first network function; and / or,

[0168] The fourth information is the information transmitted between the first device and the first network function;

[0169] The first network function is used to convert external network requirements into internal requirements.

[0170] The second network function is used for at least one of business processing and the implementation of business policy rules;

[0171] The third and fourth information are used to request the network side to perform model inference for the second AI unit.

[0172] The aforementioned first network function can also be described as a first entity, and the aforementioned second network function can also be described as an eighth network element. The network functions involved in the embodiments of this application are described below with reference to Table 1. The second device in the embodiments of this application can implement the network functions of at least one network element in Table 1 below. For example, the second device may include at least one network element other than the twelfth network element in Table 1.

[0173] Table 1

[0174] Optionally, the first device sends first information to the second device, including:

[0175] The first device sends a first message to the first network function in the second device. The first message is used to request resource allocation for AI services. The first message includes the first information, and the first information also includes at least one of the following: data characteristic information of the first AI unit and AI function corresponding to the first AI unit.

[0176] The method further includes:

[0177] The second message sent by the first network function in the second device is obtained. The second message is a response message to the first message. The second message is carried by control plane signaling and includes at least one of the following: communication resource node information, AI resource node information, data characteristic information of the first AI unit, and a first indication; wherein, the first indication is used to indicate whether the data characteristic information of the first AI unit matches the data characteristic information that the network can collect.

[0178] In this embodiment of the application, the first device interacts with the first network function in the second device to obtain the resources configured for the AI ​​services of the first device by the network side.

[0179] In this embodiment, a first device sends first information to a second device, the first information being a request for performance monitoring of an artificial intelligence (AI) service. The first device then receives second information sent by the second device, the second information including information related to the performance monitoring results of the AI ​​service or information related to the performance monitoring truth value of the AI ​​service. The first device includes a terminal, an application function (AF) server, or an OTT server; the second device includes a network device. This allows the first device to monitor the model inference of the AI ​​unit based on information provided by the network, thereby determining whether the model inference accuracy of the AI ​​unit has deteriorated, and thus ensuring the inference performance of the AI ​​unit related to the AI ​​service requested by the first device.

[0180] As shown in Figure 4, this application embodiment also provides an information transmission method, including:

[0181] Step 401: The second device obtains the first information sent by the first device, which is used to request performance monitoring of the artificial intelligence (AI) service.

[0182] Step 402: The second device sends second information to the first device, the second information including information related to the performance monitoring results of the AI ​​service or information related to the true value of the performance monitoring of the AI ​​service;

[0183] The first device includes a terminal, an application function (AF) or an OTT server; the second device includes network equipment, and optionally, the second device includes core network equipment.

[0184] Optionally, the performance monitoring result related information of the AI ​​service includes the performance monitoring result of the first AI unit, or the performance monitoring truth value related information of the AI ​​service includes at least one of the performance monitoring truth value associated with the first AI unit and the performance monitoring truth value associated with the first AI function; wherein, the first AI unit is the AI ​​unit of the first device that performs model inference, and the first AI function is the AI ​​function corresponding to the first AI unit.

[0185] Alternatively, the performance monitoring results related to the AI ​​service may include the performance monitoring results of the third AI unit and the performance monitoring results associated with the second AI function; or, the performance monitoring truth value related to the AI ​​service may include the performance monitoring truth value associated with the third AI unit; the second AI function is the AI ​​function corresponding to the third AI unit.

[0186] The above performance monitoring truth values ​​can also be described as monitoring truth values ​​or truth values, or labels.

[0187] Optionally, the performance monitoring results include at least one of the following:

[0188] Performance monitoring of key performance indicators (KPIs);

[0189] The monitoring event, the second information mentioned above is sent when the monitoring event is met. By setting the monitoring event, the continuous transmission of the second information mentioned above can be avoided, effectively reducing signaling transmission overhead.

[0190] The timestamp is used to indicate the time corresponding to the information in the performance monitoring results.

[0191] Optionally, the first information includes at least one of the following:

[0192] The monitoring types include monitoring based on the inference results of AI units, monitoring based on AI units related to the first AI unit, and monitoring based on truth values.

[0193] The monitoring feedback methods include feedback of monitoring results, feedback of monitoring truth values, periodic feedback, and event-triggered feedback. For example, event-triggered feedback helps reduce interaction overhead, while periodic feedback helps obtain more complete monitoring information. In this embodiment, the above monitoring feedback methods can be dynamically configured according to the different states of the requester (such as signal quality high or low, battery level high or low).

[0194] The monitoring period can be long or short. A long period helps reduce transmission overhead, while a short period helps obtain more complete monitoring information. In this embodiment, the monitoring period can be dynamically configured according to the different states of the requester (such as signal quality level, battery level).

[0195] Monitoring indicators;

[0196] Monitoring events;

[0197] Monitor parameters related to the event; for example, the reference includes the threshold for the event to be triggered.

[0198] In this embodiment, the second device receives first information sent by the first device, which requests performance monitoring of an artificial intelligence (AI) service. The second device then sends second information to the first device, which includes information related to the performance monitoring results of the AI ​​service or information related to the performance monitoring truth value of the AI ​​service. This allows the first device to monitor the model inference of the AI ​​unit based on information provided by the network, thereby determining whether the model inference accuracy of the AI ​​unit has deteriorated, and thus ensuring the inference performance of the AI ​​unit related to the AI ​​service requested by the first device.

[0199] Optionally, before the second device sends the second information to the first device, the method further includes:

[0200] The second device acquires the first AI service data sent by the first device, the first AI service data including the model inference results of the first AI unit;

[0201] The second information is obtained based on the model reasoning results of the first AI unit.

[0202] In this embodiment, the network side obtains the above-mentioned performance monitoring result information or the performance monitoring truth value information of the AI ​​service based on the model inference result of the first AI unit. In this way, there is no need for the second device to send the truth value, avoiding the network side directly exposing the truth value. Moreover, the performance monitoring result is calculated by the second device, which can save the computing resources of the first device.

[0203] Optionally, the first information is also used to request the network side to perform model inference for the second AI unit;

[0204] Alternatively, the method may further include:

[0205] The second device obtains the third information sent by the first device, which is used to request the network side to perform model inference for the second AI unit;

[0206] Wherein, the second AI unit is an AI unit paired with the first AI unit, and the first AI unit is the AI ​​unit that performs model inference on the first device;

[0207] The second information is obtained based on the model reasoning results of the first AI unit and the model reasoning results of the second AI unit.

[0208] In this embodiment, the first information can be used to simultaneously request performance monitoring of the AI ​​service and request the network side to perform model inference for the second AI unit. Alternatively, the first information can be used to request the network to perform performance monitoring of the AI ​​service, and the third information can be used to request the network side to perform model inference for the second AI unit.

[0209] In this embodiment of the application, the second AI unit is an AI unit paired with the first AI unit, or it can be described as the second AI unit being aligned with the first AI unit, or it can be described as the second AI unit being matched with the first AI unit.

[0210] Optionally, the alignment of the first AI unit and the second AI unit may mean that the first AI unit and the second AI unit are sub-units of the target unit, or it may mean that when the first AI unit and the second AI unit perform joint reasoning, the reasoning accuracy meets the preset requirements. For example, the joint reasoning accuracy of the first AI unit and the second AI unit is greater than or equal to the first preset threshold, or the error of the joint reasoning is less than or equal to the second preset threshold.

[0211] Optionally, the input to the second AI unit includes the model inference result of the first AI unit;

[0212] After the second device receives the first information or the third information sent by the first device, the method further includes:

[0213] The second device acquires the first AI service data sent by the first device, the first AI service data including the model inference result of the first AI unit; the second device performs model inference of the second AI unit based on the model inference result of the first AI unit to obtain the model inference result of the second AI unit.

[0214] In some embodiments of this application, a first device first performs model inference for a first AI unit to obtain the model inference result of the first AI unit, and a second device uses the model inference result of the first AI unit as input data for model inference of a second AI unit. Based on the model inference result of the first AI unit, the second device obtains the model inference result of the second AI unit.

[0215] Optionally, the input to the first AI unit includes the model inference result of the second AI unit;

[0216] After the second device receives the first information or the third information sent by the first device, the method further includes:

[0217] The second device sends the model inference results of the second AI unit to the first device;

[0218] The second device acquires second AI service data sent by the first device. The second AI service data includes at least one of the following: the model inference result of the first AI unit; and a third AI unit, wherein the third AI unit is an AI unit related to the first AI unit.

[0219] The second device obtains the second information based on the second AI service data.

[0220] As one implementation method, the third AI unit serves as a proxy model for the first AI unit.

[0221] In some embodiments of this application, the second device first performs model inference on the second AI unit, and the first device then performs model inference on the first AI unit based on the model inference result of the second AI unit to obtain the model inference result of the first AI unit. It can be understood that the model inference result of the second AI unit has no physical meaning and is merely an intermediate parameter in the model inference process. This prevents the first AI unit from being used by other unauthorized requesters.

[0222] Optionally, the reasoning result of the second AI unit satisfies any one of the following conditions:

[0223] The reasoning result of the second AI unit does not have corresponding physical parameters;

[0224] The reasoning results of the second AI unit have corresponding physical parameters.

[0225] In one embodiment, when the output of the second AI unit has corresponding physical parameters, the first device can parse the physical parameters corresponding to the output of the second AI unit, while the second device cannot parse the physical parameters corresponding to the output of the second AI unit; or,

[0226] When the output of the second AI unit has corresponding physical parameters, both the first device and the second device can parse the physical parameters corresponding to the output of the second AI unit.

[0227] The reasoning result of the second AI unit may include any one of the following:

[0228] The output of the second AI unit has no physical meaning.

[0229] The physical meaning of the output of the second AI unit cannot be interpreted by the second device, but can be understood by the first device.

[0230] The physical meaning of the output of the second AI unit can be interpreted by both the second and first devices.

[0231] In one implementation, the reasoning result of the second AI unit is the output result of the second AI unit. This output result has no physical meaning, which helps to protect the second AI unit from being used by devices other than the first device and can be used to restrict the users of the second AI unit.

[0232] In one implementation, the reasoning result of the second AI unit is the output result of the second AI unit. The second device cannot interpret the physical meaning of the output result, but the first device can know the physical meaning of the output result. This is beneficial to protecting the second AI unit from being used by devices other than the first device. It can be used to restrict the users of the second AI unit. Compared with an output result without physical meaning, it is beneficial for the first device to perform verification of the transmission result.

[0233] In one implementation, the reasoning result of the second AI unit is the output result of the second AI unit. Both the second device and the first device can parse the physical meaning of the output result, which is beneficial for sharing the output result of the second AI unit with other requesters.

[0234] In this embodiment, the reasoning result of the second AI unit does not have corresponding physical parameters, so even if a device other than the first device obtains the reasoning result of the second AI unit, it cannot parse the reasoning result. This is beneficial to protect the second AI unit from being used by devices other than the first device, thereby limiting the users of the second AI unit.

[0235] In this embodiment, the reasoning result of the second AI unit has corresponding physical parameters, which is beneficial for the first device to verify whether the reasoning result of the second AI unit is accurate, and can better support sharing the second AI unit with other devices.

[0236] Optionally, the first information includes a first identifier of the model inference task associated with the performance monitoring task requested by the first information, and the second device obtains the third information sent by the first device, including:

[0237] Based on the first identifier, the third information is obtained, wherein the third information includes the first identifier of the model reasoning task.

[0238] Based on this first identifier, the second device is able to associate the performance monitoring task of the first information request with the corresponding inference task.

[0239] Optionally, the first information is further used to request the second device to perform model inference for the second AI unit;

[0240] Before the second device receives the first information sent by the first device, the method further includes:

[0241] The second device obtains the fourth information sent by the first device, which is used to request the network side to perform model inference for the second AI unit;

[0242] The first information includes task update information, which is used to replace the model inference task requested by the fourth information with the model inference and performance monitoring task requested by the first information.

[0243] Optionally, the task update information includes a second identifier for the model inference task of the fourth information request and a third identifier for the model inference and performance monitoring tasks of the first information request.

[0244] It should be noted that the model inference task corresponding to the second identifier can be the same as the inference task in the third identifier. However, the second identifier is no longer used to identify and manage the inference task; instead, the third identifier is used to identify and manage the inference task.

[0245] In some embodiments of this application, the first device may first send a fourth message requesting the network side to perform model inference for the second AI unit, and then send the first message, modifying the model inference task requested by the fourth message into a model inference and performance monitoring task, thereby achieving the purpose of flexibly changing the monitoring task.

[0246] Optionally, the first information includes at least one of the following:

[0247] The second AI unit;

[0248] Description information related to the second AI unit;

[0249] The data characteristic information of the second AI unit, for example, the data characteristic ID of the second AI unit.

[0250] In this embodiment of the application, when the first information is used to request the network side to perform model inference of the second AI unit, the relevant information of the second AI unit is sent so that the network side can perform model inference of the second AI unit based on the relevant information of the second AI unit.

[0251] Optionally, before the second device sends the second information to the first device, the method further includes:

[0252] The second device acquires a third AI unit sent by the first device, wherein the third AI unit is an AI unit related to the first AI unit; for example, the third AI unit is a proxy model of the first AI unit.

[0253] The performance monitoring results related to the AI ​​service include the performance monitoring results of the third AI unit and the performance monitoring results associated with the second AI function; or, the performance monitoring truth value related to the AI ​​service includes the performance monitoring truth value associated with the third AI unit.

[0254] The second AI function is the AI ​​function corresponding to the third AI unit.

[0255] In this embodiment, the first device sends the aforementioned third AI unit to the second device, enabling the second device to obtain the performance monitoring result information of the aforementioned AI service based on the performance monitoring result of the third AI unit. The second device does not need to obtain the performance monitoring result information of the aforementioned AI service based on the inference result of the first AI unit, thereby avoiding the first device sending the inference result of the first AI unit to the second device and reducing signaling transmission overhead.

[0256] Optionally, the first information is information transmitted between a first network function and / or a second network function in the first device and the second device; and / or,

[0257] The second information is information transmitted between the second network function in the first device and the second device; and / or,

[0258] The third information is the information transmitted between the first device and the first network function; and / or,

[0259] The fourth information is the information transmitted between the first device and the first network function;

[0260] The first network function is used to convert external network requirements into internal requirements.

[0261] The second network function is used for at least one of business processing and the implementation of business policy rules;

[0262] The third and fourth information are used to request the network side to perform model inference for the second AI unit.

[0263] The aforementioned first network function can also be described as a first entity, and the aforementioned second network function can also be described as an eighth network element. The network functions involved in the embodiments of this application are described below with reference to Table 1. Please refer to Table 1 above for descriptions of the relevant network functions; they will not be repeated here.

[0264] Optionally, the second device acquires the first information sent by the first device, including:

[0265] The first network function in the second device obtains the first message sent by the first device. The first message is used to request resource allocation for AI services. The first message includes the first information, and the first information also includes at least one of the following: data characteristic information of the first AI unit and AI function corresponding to the first AI unit.

[0266] The method further includes:

[0267] The first network function in the second device sends a second message to the first device. The second message is a response message to the first message. The second message is carried by control plane signaling and includes at least one of the following: communication resource node information, AI resource node information, data characteristic information of the first AI unit, and a first indication; wherein, the first indication is used to indicate whether the data characteristic information of the first AI unit matches the data characteristic information that the network can collect.

[0268] In this embodiment of the application, the first device interacts with the first network function in the second device to obtain the resources configured for the AI ​​services of the first device by the network side.

[0269] Optionally, before the first network function in the second device sends the second message to the first device, it further includes:

[0270] The first network function sends a third message to the third network function. The third message is used for resource requests for AI services. The third message includes at least one of the following: data source information of the first AI unit, data characteristic information of the first AI unit, relevant monitoring request information of the first AI unit, data characteristic information associated with the data source information of the first AI unit, data characteristic information of the second AI unit, data characteristic information associated with the data source information of the second AI unit, and a first identifier; wherein, the first identifier is the identifier of the model inference task associated with the performance monitoring task of the first information request.

[0271] The first network function obtains a fourth message sent by the third network function, the fourth message being a response message to the third message, and the fourth message including AI resource node information configured according to the third message;

[0272] The third network function is used to manage service resources. Specifically, this third network function can be the aforementioned seventh network element.

[0273] Optionally, before the first network function in the second device sends the second message to the first device, it further includes:

[0274] The first network function sends a fifth message to the fourth network function. The fifth message is used to request model-related information. The fifth message includes at least one of the following: the AI ​​function corresponding to the first AI unit, the data characteristic information of the first AI unit, the relevant monitoring request information of the first AI unit, the second AI unit, the function corresponding to the second AI unit, the data characteristic information of the second AI unit, the relevant description information of the second AI unit, and the first identifier; wherein, the first identifier is the identifier of the model inference task associated with the performance monitoring task of the first information request.

[0275] The first network function obtains the sixth message sent by the fourth network function, the sixth message including model-related information, or including model-related information and the first indication;

[0276] The fourth network function is used for AI model management.

[0277] The fourth network function can be specifically referred to as the sixth network element mentioned above.

[0278] Optionally, the method in this application embodiment further includes:

[0279] The first network function sends a seventh message to the fifth network function. The seventh message is used to obtain the data source information of the first AI unit. The seventh message includes at least one of the following: data characteristic information of the first AI unit, the first identifier, and historical data source information associated with the data characteristic information of the first AI unit.

[0280] The second network function acquires the eighth message sent by the fifth network function, the eighth message including the data source information of the first AI unit;

[0281] The fifth network function is used for data management.

[0282] The fifth network function can be specifically referred to as the ninth network element mentioned above.

[0283] The information transmission method of this application will be described in detail below with reference to the embodiments. In the following embodiments, the first device is described as the requester and the second device is described as the network. The network may include at least one network element other than the twelfth network element in Table 1.

[0284] Example 1: Business monitoring is performed based on the relationship between AI services and AI units. This allows for the deployment of different AI units for monitoring different AI services, thereby preventing the deterioration of AI service performance.

[0285] Method 1: AI business-related model inference is performed by the requester;

[0286] As shown in Figure 5a, the process includes:

[0287] Step 1: The requester sends the first message to the network, which is used to request performance monitoring of the AI ​​service.

[0288] Optionally, the first information may further include at least one of the AI ​​function of the first AI unit and the data source request information of the first AI unit.

[0289] Step 2: The network and the requester interact with the input data of the first AI unit.

[0290] Step 2 is optional. If the input data of the first AI unit is collected by the network, then in step 2 the network sends the input data of the first AI unit to the requester.

[0291] Step 3: The requester performs model inference for the first AI unit and obtains the model inference result of the first AI unit.

[0292] Step 4: The requester and the network interact with AI business data, which includes at least one of the following:

[0293] The model inference results of the first AI unit;

[0294] True values ​​for performance monitoring of AI business;

[0295] Monitor KPIs;

[0296] Monitoring events;

[0297] Timestamp.

[0298] Step 5: The requester obtains the performance monitoring results of the AI ​​business.

[0299] In this method 1, the performance monitoring results of the AI ​​business are directly determined by the first AI unit, which can also be understood as the performance monitoring results of the AI ​​business can be obtained based on the one-sided reasoning of the first AI unit.

[0300] Method 2: AI business-related results are completed by joint reasoning between the requester and the network. Based on the order in which the requester and the network perform AI unit reasoning, Method 2 is divided into Method 2-1 and Method 2-2.

[0301] Method 2-1: The requester first performs model inference for the first AI unit, and then the network performs model inference for the second AI unit.

[0302] As shown in Figure 5b, the process includes:

[0303] Step 1: The requester sends the first message to the network, which is used to request performance monitoring of the AI ​​service.

[0304] Optionally, the first information may further include at least one of the AI ​​function of the first AI unit and the data source request information of the first AI unit.

[0305] Step 2: The requester performs model inference for the first AI unit and obtains the model inference result of the first AI unit.

[0306] Step 3: The requester sends AI service data to the network. The AI ​​service data includes the model inference results of the first AI unit.

[0307] Step 4: The network collects data and uses it as model input for the second AI unit.

[0308] Step 4 is optional.

[0309] Step 5: The network performs model inference for the second AI unit and obtains the model inference result of the second AI unit.

[0310] If step 4 is executed, the model input of the second AI unit includes the model inference result of the first AI unit and the data collected from the network. If step 4 is not executed, the model input of the second AI unit includes the model inference result of the first AI unit.

[0311] Step 6: Obtain the truth value from the network.

[0312] As one implementation, the network receives requests from AI units related to monitoring. These requests can indicate that obtaining truth values ​​relies on model inference from network-side AI units. Based on the functionality of a first or second AI unit, the network can find an AI unit that obtains truth values. Through the AI ​​unit obtained via inference, the network acquires the performance monitoring truth values.

[0313] Step 7: The network interacts with the requester with AI business data, which includes at least one of the following: monitoring events, monitoring KPIs, timestamps, and truth values.

[0314] Step 8: The requester obtains the performance monitoring results of the AI ​​business.

[0315] In this method 2-1, the relationship between the first AI unit and the second AI unit includes the following three cases:

[0316] Scenario 1: The first AI unit can be aligned with the second AI unit. For example, the first and second AI units are sub-units of a certain AI unit, and this process is used to monitor the inference performance of the entire AI unit.

[0317] Scenario 2: The first AI unit and the second AI unit were historically aligned, and now it is necessary to determine whether they are still aligned through business monitoring.

[0318] Scenario 3: It is unknown whether the first AI unit and the second AI unit are aligned. It is now necessary to determine whether they are still aligned through business monitoring.

[0319] Optionally, in this method 2-1, the output of the second AI unit has physical meaning.

[0320] Optionally, the reasoning result of the second AI unit satisfies any one of the following conditions:

[0321] The reasoning result of the second AI unit does not have corresponding physical parameters;

[0322] The reasoning results of the second AI unit have corresponding physical parameters.

[0323] In one embodiment, when the output of the second AI unit has corresponding physical parameters, the first device can parse the physical parameters corresponding to the output of the second AI unit, while the second device cannot parse the physical parameters corresponding to the output of the second AI unit; or,

[0324] When the output of the second AI unit has corresponding physical parameters, both the first device and the second device can parse the physical parameters corresponding to the output of the second AI unit.

[0325] The reasoning result of the second AI unit may include any one of the following:

[0326] The output of the second AI unit has no physical meaning.

[0327] The physical meaning of the output of the second AI unit cannot be interpreted by the second device, but can be understood by the first device.

[0328] The physical meaning of the output of the second AI unit can be interpreted by both the second and first devices.

[0329] In one implementation, the reasoning result of the second AI unit is the output result of the second AI unit. This output result has no physical meaning, which helps to protect the second AI unit from being used by devices other than the first device and can be used to restrict the users of the second AI unit.

[0330] In one implementation, the reasoning result of the second AI unit is the output result of the second AI unit. The second device cannot interpret the physical meaning of the output result, but the first device can know the physical meaning of the output result. This is beneficial to protecting the second AI unit from being used by devices other than the first device. It can be used to restrict the users of the second AI unit. Compared with an output result without physical meaning, it is beneficial for the first device to perform verification of the transmission result.

[0331] In one implementation, the reasoning result of the second AI unit is the output result of the second AI unit. Both the second device and the first device can parse the physical meaning of the output result, which is beneficial for sharing the output result of the second AI unit with other requesters.

[0332] In this embodiment, the reasoning result of the second AI unit does not have corresponding physical parameters, so even if a device other than the first device obtains the reasoning result of the second AI unit, it cannot parse the reasoning result. This is beneficial to protect the second AI unit from being used by devices other than the first device, thereby limiting the users of the second AI unit.

[0333] In this embodiment, the reasoning result of the second AI unit has corresponding physical parameters, which is beneficial for the first device to verify whether the reasoning result of the second AI unit is accurate, and can better support sharing the second AI unit with other devices.

[0334] In this method 2-1, the performance monitoring results of the AI ​​business are first inferred by the first AI unit and then determined by the second AI unit.

[0335] Method 2-2: First, the network performs model inference for the second AI unit, and then the requester performs model inference for the first AI unit;

[0336] As shown in Figure 5c, the process includes:

[0337] Step 1: The requester sends the first message to the network, which is used to request performance monitoring of the AI ​​service.

[0338] Optionally, the first information may further include at least one of the AI ​​functions of the first AI unit, the second AI unit, and the data source request information of the first AI unit.

[0339] Step 2: The network collects data and uses the collected data as the model input for the second AI unit.

[0340] Step 2 is optional.

[0341] Step 3: The network performs model inference for the second AI unit.

[0342] If step 2 is performed, the data collected in step 2 will be used as the model input for the second AI unit; if the requesting party provides input data for the second AI unit to the network before step 3, the data provided by the requesting party can be used as the input data for the second AI unit.

[0343] Step 4: The network sends the model inference results of the second AI unit to the requester.

[0344] Step 5: The requester performs model reasoning for the first AI unit.

[0345] Specifically, the data obtained in step 4 is used as the model input for the first AI unit.

[0346] Optionally, the reasoning result of the second AI unit satisfies any one of the following conditions:

[0347] The reasoning result of the second AI unit does not have corresponding physical parameters;

[0348] The reasoning results of the second AI unit have corresponding physical parameters.

[0349] In one embodiment, when the output of the second AI unit has corresponding physical parameters, the first device can parse the physical parameters corresponding to the output of the second AI unit, while the second device cannot parse the physical parameters corresponding to the output of the second AI unit; or,

[0350] When the output of the second AI unit has corresponding physical parameters, both the first device and the second device can parse the physical parameters corresponding to the output of the second AI unit.

[0351] The reasoning result of the second AI unit may include any one of the following:

[0352] The output of the second AI unit has no physical meaning.

[0353] The physical meaning of the output of the second AI unit cannot be interpreted by the second device, but can be understood by the first device.

[0354] The physical meaning of the output of the second AI unit can be interpreted by both the second and first devices.

[0355] In one implementation, the reasoning result of the second AI unit is the output result of the second AI unit. This output result has no physical meaning, which helps to protect the second AI unit from being used by devices other than the first device and can be used to restrict the users of the second AI unit.

[0356] In one implementation, the reasoning result of the second AI unit is the output result of the second AI unit. The second device cannot interpret the physical meaning of the output result, but the first device can know the physical meaning of the output result. This is beneficial to protecting the second AI unit from being used by devices other than the first device. It can be used to restrict the users of the second AI unit. Compared with an output result without physical meaning, it is beneficial for the first device to perform verification of the transmission result.

[0357] In one implementation, the reasoning result of the second AI unit is the output result of the second AI unit. Both the second device and the first device can parse the physical meaning of the output result, which is beneficial for sharing the output result of the second AI unit with other requesters.

[0358] In this embodiment, the reasoning result of the second AI unit does not have corresponding physical parameters, so even if a device other than the first device obtains the reasoning result of the second AI unit, it cannot parse the reasoning result. This is beneficial to protect the second AI unit from being used by devices other than the first device, thereby limiting the users of the second AI unit.

[0359] In this embodiment, the reasoning result of the second AI unit has corresponding physical parameters, which is beneficial for the first device to verify whether the reasoning result of the second AI unit is accurate, and can better support sharing the second AI unit with other devices.

[0360] Optionally, the model input of the first AI unit may also include other data collected by the requester.

[0361] Step 6: The requester sends AI service data to the network, the AI ​​service data including at least one of the following: the model inference result of the first AI unit; related units of the first AI unit.

[0362] Step 7: Obtain the truth value from the network.

[0363] Step 8: The network interacts with the requester with AI business data, which includes at least one of the following: monitoring events, monitoring KPIs, timestamps, and truth values.

[0364] Step 9: The requester obtains the performance monitoring results of the AI ​​business.

[0365] In this method 2-2, the relationship between the first AI unit and the second AI unit includes the following three cases:

[0366] Scenario 1: The first AI unit can be aligned with the second AI unit. For example, the first and second AI units are sub-units of a certain AI unit, and this process is used to monitor the inference performance of the entire AI unit.

[0367] Scenario 2: The first AI unit and the second AI unit were historically aligned, and now it is necessary to determine whether they are still aligned through business monitoring.

[0368] Scenario 3: It is unknown whether the first AI unit and the second AI unit are aligned. It is now necessary to determine whether they are aligned through business monitoring.

[0369] In this method 2-2, the performance monitoring results of the AI ​​business are first inferred by the second AI unit, and then obtained by the first AI unit.

[0370] Example 2: This application describes multiple implementations of the monitoring request process. For example, the monitoring request can be sent separately, or the monitoring request and the inference request can be sent simultaneously.

[0371] Implementation method 1: Send monitoring requests separately;

[0372] As shown in Figure 6a, the process includes:

[0373] Step 1: The requester sends the first message to the network, which is used to request performance monitoring of the AI ​​service.

[0374] The first information includes at least one of the following:

[0375] Monitoring type;

[0376] Monitoring and feedback methods;

[0377] Monitoring cycle;

[0378] Monitoring indicators;

[0379] Monitoring events;

[0380] Parameters related to monitoring events.

[0381] In this embodiment, the monitoring types include monitoring based on the inference results of the first AI unit, monitoring based on a proxy model, and monitoring based on truth values. The proxy model is the proxy model of the first AI unit.

[0382] The aforementioned monitoring feedback method can be configured as periodic feedback or event-triggered feedback. Event-triggered feedback helps reduce interaction overhead; periodic feedback helps obtain more complete monitoring information. In this embodiment, the aforementioned monitoring feedback method can be dynamically requested based on the requester's different states (e.g., signal quality high / low, battery level high / low).

[0383] The aforementioned monitoring period can be configured as a long period or a short period. A long period helps reduce transmission overhead, while a short period helps obtain more complete monitoring information. In this embodiment, the aforementioned monitoring period can be dynamically requested based on the requester's different states (such as signal quality level or battery level).

[0384] Step 2: The network sends a response message containing the first information to the requester.

[0385] Optionally, the response message includes an identifier of the performance monitoring task of the first information request.

[0386] Implementation method 2: Monitoring requests and inference requests are sent simultaneously;

[0387] As shown in Figure 6b, the process includes:

[0388] Step 1: The requester sends the first message to the network, which is used to request performance monitoring of the AI ​​service and to request the network side to perform model inference for the second AI unit.

[0389] Optionally, the first information includes at least one of the following:

[0390] Monitoring type;

[0391] Monitoring and feedback methods;

[0392] Monitoring cycle;

[0393] Monitoring indicators;

[0394] Monitoring events;

[0395] Parameters related to monitoring events;

[0396] The second AI unit;

[0397] Description information related to the second AI unit;

[0398] The data characteristic information of the second AI unit.

[0399] Optionally, the second AI unit includes at least one of the following:

[0400] The executable file corresponding to the second AI unit;

[0401] The reference model structure indication for the second AI unit;

[0402] The order of model parameters for the second AI unit;

[0403] The parameter file for the second AI unit;

[0404] Instructions from the second AI unit;

[0405] Modification layer instructions for the second AI unit;

[0406] The layer indicator corresponding to the second AI unit, for example, the second AI unit corresponds to the first 1 to K layers.

[0407] Optionally, the descriptive information related to the second AI unit includes at least one of the following:

[0408] The effective duration of the second AI unit;

[0409] The configuration environment of the second AI unit includes: deep learning frameworks (TensorFlow, PyTorch, etc.), supported library versions, etc.

[0410] The second AI unit's model input information indicator is used to indicate data items, data format, data precision, etc.

[0411] The second AI unit's model output information indicator is used to indicate the accuracy, dimension, and storage format of the output data.

[0412] Step 2: The network sends a response message containing the first information to the requester.

[0413] Optionally, the response message includes performance monitoring of the first information request and an identifier of the model inference task.

[0414] Implementation method 3: Monitoring requests and inference requests are sent separately;

[0415] As shown in Figure 6c, the process includes:

[0416] Step 1: The requester sends a third message to the network, which is used to request the network to perform model inference for the second AI unit;

[0417] The third information includes at least one of the following:

[0418] The second AI unit;

[0419] Description information related to the second AI unit;

[0420] The data characteristic information of the second AI unit.

[0421] Optionally, the second AI unit can refer to the second AI unit in the above implementation method 2, and the description information related to the second AI unit can refer to the description information related to the AI ​​unit in the above implementation method 2, which will not be repeated here.

[0422] Step 2: The network sends a response message containing third-party information to the requester.

[0423] Optionally, the response message includes a first identifier.

[0424] Step 3: The requester sends the first message to the network, which is used to request performance monitoring of the AI ​​service;

[0425] The first information includes at least one of the following:

[0426] The first identifier is an identifier for the model inference task associated with the performance monitoring task of the first information request;

[0427] Monitoring type;

[0428] Monitoring and feedback methods;

[0429] Monitoring cycle;

[0430] Monitoring indicators;

[0431] Monitoring events;

[0432] Parameters related to monitoring events.

[0433] Step 4: The network sends a response message containing the first information to the requester.

[0434] Optionally, the response message includes an identifier of the performance monitoring task of the first information request.

[0435] Implementation method 4: First send an inference request, then modify the inference request into a monitoring request and an inference request;

[0436] As shown in Figure 6d, the process includes:

[0437] Step 1: The requester sends a fourth message to the network, which is used to request the network to perform model inference for the second AI unit;

[0438] The fourth piece of information includes at least one of the following:

[0439] The second AI unit;

[0440] Description information related to the second AI unit;

[0441] The data characteristic information of the second AI unit.

[0442] Optionally, the second AI unit can refer to the second AI unit in the above implementation method 2, and the description information related to the second AI unit can refer to the description information related to the AI ​​unit in the above implementation method 2, which will not be repeated here.

[0443] Step 2: The network sends a response message containing the fourth piece of information to the requester.

[0444] Optionally, the response message includes a second identifier.

[0445] Step 3: The requester sends first information to the network, which is used to modify or replace the model inference task requested by the fourth information with the model inference and performance monitoring task requested by the first information.

[0446] The first information includes at least one of the following:

[0447] The second identifier of the model reasoning task in the fourth information request;

[0448] Monitoring type;

[0449] Monitoring and feedback methods;

[0450] Monitoring cycle;

[0451] Monitoring indicators;

[0452] Monitoring events;

[0453] Parameters related to monitoring events.

[0454] Step 4: The network sends a response message containing the first information to the requester.

[0455] Optionally, the response message includes a third identifier for the model inference and performance monitoring task of the first information request.

[0456] Example 3: The user plane or data plane interaction process corresponding to the three performance monitoring schemes in Example 1 is described.

[0457] Monitoring Solution 1: Monitoring based on inference results;

[0458] Method 1: The performance monitoring results of AI services are directly determined by the first AI unit, which can also be understood as the performance monitoring results of AI services being obtained based on the one-sided reasoning of the first AI unit;

[0459] As shown in Figure 7a, the process includes:

[0460] Step 1: The requester performs model reasoning for the first AI unit.

[0461] Step 2: The requester sends AI business data to the network, which includes the model inference results of the first AI unit.

[0462] To address privacy concerns, before sending the model inference results of the first AI unit to the network, if the inference results of the first AI unit have physical meaning, then the model inference results of the first AI unit will be anonymized. If the inference results of the first AI unit do not have physical meaning, then the model inference results of the first AI unit do not need to be anonymized.

[0463] Optionally, the desensitization process may include adding a certain amount of random noise.

[0464] Step 3: Obtain the truth value from the network.

[0465] Step 4: The network sends AI business data to the requester, which includes at least one of the following:

[0466] Monitoring events;

[0467] Monitor KPIs;

[0468] Timestamp.

[0469] Optionally, the monitoring KPIs are obtained based on the model inference results and truth values ​​of the first AI unit.

[0470] Step 5: The requester obtains the performance monitoring results of the AI ​​business.

[0471] In Method 1, the requester sends the inference results of the first AI unit to the network, the network calculates the monitoring KPIs, and after monitoring the event, feeds back the relevant information to the requester.

[0472] Method 2-1: The performance-related AI unit model inference for AI business is first determined by the first AI unit inference, and then by the second AI unit inference.

[0473] As shown in Figure 7b, the process includes:

[0474] Step 1: The requester performs model reasoning for the first AI unit.

[0475] Step 2: The requester sends AI business data to the network, which includes the model inference results of the first AI unit.

[0476] Step 3: Optionally, the network collects data and uses the collected data as model input for the second AI unit.

[0477] Step 4: The network performs model inference for the second AI unit and obtains the model inference result of the second AI unit.

[0478] If step 3 is executed, the model input of the second AI unit includes the model inference result of the first AI unit and the data collected from the network. If step 3 is not executed, the model input of the second AI unit includes the model inference result of the first AI unit.

[0479] The network side can obtain the third AI unit through a requester or an OTT server associated with the requester. The signaling carried can be based on the user plane, data plane, or control plane.

[0480] Step 5: Obtain the truth value from the network.

[0481] Step 6: The network sends AI business data to the requester, which includes at least one of the following:

[0482] Monitoring events;

[0483] Monitor KPIs;

[0484] Timestamp.

[0485] Optionally, the aforementioned monitoring KPIs are obtained based on the true values ​​obtained from the network and the model inference results of the second AI unit.

[0486] Step 7: The requester obtains the performance monitoring results of the AI ​​business.

[0487] Method 2-2: The AI ​​unit model inference related to the performance of AI business is first obtained by the second AI unit and then by the first AI unit.

[0488] As shown in Figure 7c, the process includes:

[0489] Step 1: Optionally, the network collects data and uses the collected data as model input for the second AI unit.

[0490] Step 2: The network performs model inference for the second AI unit.

[0491] If step 1 is performed, the data collected in step 1 is used as the model input for the second AI unit; if the requesting party provides input data for the second AI unit to the network before step 2, the data provided by the requesting party can be used as the input data for the second AI unit.

[0492] The network side can obtain the second AI unit by sending it through the requester or an OTT server associated with the requester. The signaling carried can be based on the user plane, data plane, or control plane.

[0493] Step 3: The network sends the model inference results of the second AI unit to the requester.

[0494] Step 4: The requester performs model reasoning for the first AI unit.

[0495] Specifically, the data obtained in step 3 is used as the model input for the first AI unit.

[0496] Optionally, the model input of the first AI unit may also include other data collected by the requester.

[0497] Optionally, the reasoning result of the second AI unit satisfies any one of the following conditions:

[0498] The reasoning result of the second AI unit does not have corresponding physical parameters;

[0499] The reasoning results of the second AI unit have corresponding physical parameters.

[0500] In one embodiment, when the output of the second AI unit has corresponding physical parameters, the first device can parse the physical parameters corresponding to the output of the second AI unit, while the second device cannot parse the physical parameters corresponding to the output of the second AI unit; or,

[0501] When the output of the second AI unit has corresponding physical parameters, both the first device and the second device can parse the physical parameters corresponding to the output of the second AI unit.

[0502] The reasoning result of the second AI unit may include any one of the following:

[0503] The output of the second AI unit has no physical meaning.

[0504] The physical meaning of the output of the second AI unit cannot be interpreted by the second device, but can be understood by the first device.

[0505] The physical meaning of the output of the second AI unit can be interpreted by both the second and first devices.

[0506] In one implementation, the reasoning result of the second AI unit is the output result of the second AI unit. This output result has no physical meaning, which helps to protect the second AI unit from being used by devices other than the first device and can be used to restrict the users of the second AI unit.

[0507] In one implementation, the reasoning result of the second AI unit is the output result of the second AI unit. The second device cannot interpret the physical meaning of the output result, but the first device can know the physical meaning of the output result. This is beneficial to protecting the second AI unit from being used by devices other than the first device. It can be used to restrict the users of the second AI unit. Compared with an output result without physical meaning, it is beneficial for the first device to perform verification of the transmission result.

[0508] In one implementation, the reasoning result of the second AI unit is the output result of the second AI unit. Both the second device and the first device can parse the physical meaning of the output result, which is beneficial for sharing the output result of the second AI unit with other requesters.

[0509] In this embodiment, the reasoning result of the second AI unit does not have corresponding physical parameters, so even if a device other than the first device obtains the reasoning result of the second AI unit, it cannot parse the reasoning result. This is beneficial to protect the second AI unit from being used by devices other than the first device, thereby limiting the users of the second AI unit.

[0510] In this embodiment, the reasoning result of the second AI unit has corresponding physical parameters, which is beneficial for the first device to verify whether the reasoning result of the second AI unit is accurate, and can better support sharing the second AI unit with other devices.

[0511] Step 5: The requester sends AI business data to the network, which includes the model inference results of the first AI unit.

[0512] Step 6: Obtain the truth value from the network.

[0513] Optionally, the network retrieves the truth value associated with the first AI unit.

[0514] Step 7: The network sends AI business data to the requester, which includes at least one of the following:

[0515] Monitoring events;

[0516] Monitor KPIs;

[0517] Timestamp.

[0518] Optionally, the monitoring KPIs are obtained based on the true values ​​and the model inference results of the first AI unit.

[0519] Step 8: The requester obtains the performance monitoring results of the AI ​​business.

[0520] The above monitoring scheme 1 involves the requester sending the inference result of the first AI unit to the network, the network obtaining the truth value, calculating the monitoring KPI based on the obtained truth value, and feeding back relevant information to the requester after monitoring the event.

[0521] Monitoring Solution 2: Monitoring based on the agent model;

[0522] Method 1:

[0523] As shown in Figure 7d, the process includes:

[0524] Step 1: The requester sends at least one of the following to the network: a third AI unit and description information related to the third AI unit, wherein the third AI unit is an AI unit related to the first AI unit.

[0525] In this embodiment of the application, there is a fixed difference in inference accuracy between the third AI unit and the first AI unit, and this difference can be known on the first device side.

[0526] For example, the third AI unit is a proxy model for the first AI unit.

[0527] Optionally, in step 1, the requester also sends a third AI to the network;

[0528] The third AI unit includes at least one of the following:

[0529] The executable file corresponding to the third AI unit;

[0530] The reference model structure indication for the third AI unit;

[0531] The order of model parameters in the third AI unit;

[0532] The parameter file for the third AI unit;

[0533] Instructions from the third AI unit;

[0534] Modification layer instructions for the third AI unit;

[0535] The layer indicator corresponding to the third AI unit, for example, is used to indicate the first 1 to K layers;

[0536] Optionally, the descriptive information related to the third AI unit includes at least one of the following:

[0537] The effective duration of the third AI unit;

[0538] The configuration environment of the third AI unit, such as deep learning frameworks (TensorFlow, PyTorch, etc.) and supported library versions;

[0539] The model input information indicator of the third AI unit, for example, is used to indicate data items, data format, data precision, etc.

[0540] The third AI unit's model output information indicator, for example, is used to indicate the accuracy of the output data, data dimensions, storage format, etc.

[0541] The network side can obtain the third AI unit through a requester or an OTT server associated with the requester. The signaling carried can be based on the user plane, data plane, or control plane.

[0542] Step 2: Collect the model input data of the first AI unit via the network.

[0543] Step 3: The network sends AI service data to the requester, the AI ​​service data including the model input data of the first AI unit.

[0544] Step 4: The requester performs model reasoning for the first AI unit.

[0545] Step 5: The network performs model inference for the third AI unit.

[0546] Step 6: Obtain the truth value from the network.

[0547] Specifically, the network obtains truth values ​​associated with the third AI unit, or truth values ​​associated with the function of the third AI unit.

[0548] Step 7: The requester obtains AI business data from the network. This AI business data includes the monitoring results of the third AI unit, and the monitoring results of the third AI unit include at least one of the following:

[0549] The monitoring KPIs of the third AI unit;

[0550] Monitoring events of the third AI unit;

[0551] Timestamp.

[0552] Optionally, the monitoring results of the third AI unit are obtained based on the model inference results of the third AI unit and the truth values ​​associated with the third AI unit.

[0553] Step 8: The requester obtains the performance monitoring results of the AI ​​service based on the monitoring results of the third AI unit.

[0554] In this embodiment of the application, there is a correlation between the monitoring results of the third AI unit and the monitoring results of the first AI unit. Based on the monitoring results of the third AI unit and the correlation, the requester can obtain the monitoring results of the first AI unit, perform monitoring based on the monitoring results of the first AI unit, and obtain the performance monitoring results of the AI ​​service.

[0555] Method 2: The performance-related AI unit model inference results of AI business are first obtained by the second AI unit inference, and then by the first AI unit inference.

[0556] As shown in Figure 7e, the process includes:

[0557] Step 0: The requester sends at least one of the following to the network: a third AI unit and description information related to the third AI unit, wherein the third AI unit is an AI unit related to the first AI unit.

[0558] This step is the same as step 1 in method 1 of monitoring scheme 2.

[0559] Step 1: The network collects data as input for the model of the second AI unit.

[0560] Step 1 is optional.

[0561] Step 2: The network performs model inference for the second AI unit.

[0562] Optionally, if step 1 was performed, the data collected in step 1 is used as the model input for the second AI unit. Optionally, if the requesting party provided input data for the second AI unit to the network before step 2, the data provided by the requesting party can be used as the input data for the second AI unit.

[0563] The network side can obtain the second AI unit by sending it through the requester or an OTT server associated with the requester. The signaling carried can be based on the user plane, data plane, or control plane.

[0564] Step 3: The network sends the inference results of the second AI unit to the requester.

[0565] Step 4: The requester performs model reasoning for the first AI unit.

[0566] Specifically, the data obtained in step 3 is used as the model input.

[0567] Step 5: The network performs model inference for the third AI unit.

[0568] Specifically, the model output of the second AI unit obtained in step 2 is used as the model input of the third AI unit.

[0569] Step 6: Obtain the truth value from the network.

[0570] The network retrieves the truth value associated with the third AI unit.

[0571] Step 7: The requester obtains AI business data from the network. This AI business data includes the monitoring results of the third AI unit, and the monitoring results of the third AI unit include at least one of the following:

[0572] The monitoring KPIs of the third AI unit;

[0573] Monitoring events of the third AI unit;

[0574] Timestamp.

[0575] Optionally, the monitoring results of the third AI unit are obtained based on the model inference results of the third AI unit and the truth values ​​associated with the third AI unit.

[0576] Step 8: The requester obtains the performance monitoring results of the AI ​​service based on the monitoring results of the third AI unit.

[0577] In this embodiment of the application, there is a correlation between the monitoring results of the third AI unit and the monitoring results of the first AI unit. Based on the monitoring results of the third AI unit and the correlation, the requester can obtain the monitoring results of the first AI unit, perform monitoring based on the monitoring results of the first AI unit, and obtain the performance monitoring results of the AI ​​service.

[0578] Monitoring scheme 1 involves the requester sending the inference result of the first AI unit to the network. The network retrieves the truth value and calculates monitoring KPIs based on it. After a monitoring event, the network sends relevant information back to the requester. During continuous inference, the requester needs to continuously send the inference result of the first AI unit to the network. Monitoring scheme 2, on the other hand, involves the requester sending the relevant AI units of the first AI unit to the network all at once, thus avoiding multiple uplink transmissions during the inference phase to send the inference result of the first AI unit.

[0579] Monitoring Solution 3: Monitoring based on truth value interaction;

[0580] Method 1: The performance-related inference results of AI unit models for AI business are directly determined by the first AI unit;

[0581] As shown in Figure 7f, the process includes:

[0582] Step 1: The requester performs model inference for the first AI unit;

[0583] Step 2: Obtain truth information from the network.

[0584] Step 3: The requester obtains AI business data from the network. The AI ​​business data includes truth information, which includes at least one of the following:

[0585] The truth value associated with the first AI unit;

[0586] The truth value associated with the AI ​​function corresponding to the first AI unit;

[0587] Timestamp.

[0588] To protect privacy, network providers can anonymize truth information before providing it to requesters. For example, the anonymized truth information may only contain truth values ​​for some important results, rather than all truth values.

[0589] Step 4: Based on the truth information obtained in Step 3, the requester obtains the performance monitoring results of the AI ​​business.

[0590] Method 2: The performance-related AI unit model inference results of AI business are first obtained by the second AI unit inference, and then by the first AI unit inference.

[0591] As shown in Figure 7g, the process includes:

[0592] Step 1: The network collects data and uses the collected data as the model input for the second AI unit.

[0593] Step 1 is optional.

[0594] Step 2: The network performs model inference for the second AI unit.

[0595] If step 1 is performed, the data collected in step 1 is used as the model input for the second AI unit; if the requesting party provides input data for the second AI unit to the network before step 2, the data provided by the requesting party can be used as the input data for the second AI unit.

[0596] Step 3: The network sends the model inference results of the second AI unit to the requester.

[0597] Step 4: The requester performs model reasoning for the first AI unit.

[0598] Specifically, the data obtained in step 3 is used as the model input for the first AI unit. This can be understood as the model inference result of the second AI unit having no physical meaning, serving as an intermediate parameter in the model inference process. This prevents the first AI unit from being used by other unauthorized requesters.

[0599] Optionally, the model input of the first AI unit may also include other data collected by the requester.

[0600] Step 5: Obtain truth information from the network.

[0601] Step 6: The requester obtains AI business data from the network. The AI ​​business data includes truth information, which includes at least one of the following:

[0602] The truth value associated with the first AI unit;

[0603] The truth value associated with the AI ​​function corresponding to the first AI unit;

[0604] Timestamp.

[0605] Step 7: Based on the truth information obtained in Step 6, the requester obtains the performance monitoring results of the AI ​​business.

[0606] In this monitoring scheme 3, the network side provides truth-related data, and the requester calculates the monitoring results themselves, which can save uplink overhead and avoid excessive signaling interaction during the monitoring phase.

[0607] Example 4: This example 4 is described in conjunction with the control plane flow of external AI services. The first device in this example is specifically a UE.

[0608] Example 4-1: Combined with Implementation Method 1 (monitoring request sent separately) in Example 2 above;

[0609] As shown in Figure 8a, the process includes:

[0610] Step 1: The UE sends a first message to the first entity. The first message is used to request resource allocation for the AI ​​service. The first message includes the first information, which is used to request performance monitoring of the AI ​​service.

[0611] Optionally, the resources include at least one of communication resources, data resources, computing power resources, and AI resources.

[0612] Optionally, the first message can be a request message or a subscription message.

[0613] Optionally, the first information includes at least one of the following:

[0614] Monitoring type;

[0615] Monitoring and feedback methods;

[0616] Monitoring cycle;

[0617] Monitoring indicators;

[0618] Monitoring events;

[0619] Parameters related to monitoring events;

[0620] Data characteristic ID of the first AI unit;

[0621] The AI ​​function corresponding to the first AI unit.

[0622] Optionally, the first entity may be a newly established fifth network element, such as an AI service management function or a collaborative control function. Alternatively, the first entity may be co-located with an existing entity, such as co-located with a second network element (communication management function) or co-located with a fourth network element (policy billing function).

[0623] Optionally, the aforementioned first information can be forwarded via a Non-Access Stratum (NAS) interface.

[0624] Step 2: The first entity and the fourth network element determine the AI ​​strategy.

[0625] Optionally, the fourth network element is a policy-based billing function.

[0626] Step 3a: The first entity sends a fifth message to the sixth network element. This fifth message is used to obtain model-related information. The model-related information includes at least one of the following:

[0627] The AI ​​function corresponding to the first AI unit;

[0628] Data characteristic ID of the first AI unit;

[0629] The first AI unit's relevant monitoring request information.

[0630] Optionally, the aforementioned sixth network element is an AI model management function.

[0631] By sending the AI ​​function corresponding to the first AI unit, the sixth network element can identify the relevant AI unit being monitored.

[0632] Since the configuration for monitoring data collection needs to match the data characteristic ID, sending the data characteristic ID of the first AI unit helps the sixth network element determine whether it can collect monitoring data related to the first AI unit, or the input data required for monitoring data and inference.

[0633] By sending the relevant monitoring request information of the first AI unit, it is possible to indicate to the sixth network element whether it is necessary to obtain the truth value based on the AI ​​unit, and to help the sixth network element determine the corresponding AI unit so as to obtain the truth value in subsequent processes.

[0634] Step 3b (optional): The sixth network element determines whether the data characteristic ID of the first AI unit matches the data characteristics that can be collected by the network.

[0635] For example, based on the network configuration associated with the data characteristic ID, it can be determined whether the data characteristics that the network can collect match the network configuration. If the data characteristics that the network can collect match the data characteristic ID of the first AI unit, then the subsequent steps are executed.

[0636] Step 3c: The sixth network element sends a sixth message to the first entity. This sixth message is used in response to the fifth message and indicates model-related information. The sixth message includes at least one of the following:

[0637] AI units related to monitoring;

[0638] First indication; the first indication is used to indicate whether the data characteristic information of the first AI unit matches the data characteristic information that the network can collect.

[0639] The data characteristic ID of the first AI unit.

[0640] Step 4a: The first entity sends a seventh message to the ninth network element, the seventh message being used to obtain the data source information of the first AI unit.

[0641] This ninth network element can be a data plane management function.

[0642] The seventh message can be a request message or a subscription message. It is used to obtain the data source and determine the data source information; optionally, the seventh message includes the data characteristic ID of the first AI unit.

[0643] Optionally, the seventh message is determined based on the description information related to the first AI unit in the first information.

[0644] When the first entity selects the data plane management function instance, it considers at least one of the following factors:

[0645] Data Network Name (DNN);

[0646] Single Network Slice Selection Assistance Information (S-NSSAI);

[0647] AI business identifier;

[0648] Supported AI services;

[0649] Service area information;

[0650] Data source information.

[0651] Step 4b: The ninth network element sends an eighth message to the first entity, which indicates the data source information. Optionally, the eighth message includes the associated data characteristic ID.

[0652] Step 5: The first entity sends a third message to the seventh network element. The third message is for AI service resource request.

[0653] Optionally, this seventh network element is an AI resource management function.

[0654] Optionally, the AI ​​resource request information includes at least one of the following:

[0655] Model information, such as the model or its download address;

[0656] Data source information, such as the data source address;

[0657] Data characteristic ID of the first AI unit;

[0658] Data characteristic ID associated with data source information;

[0659] AI business computational load, such as computing power size and type;

[0660] Access-stratum (AS) address;

[0661] Step 6: The seventh network element selects the appropriate eighth network element based on the AI ​​resource request message.

[0662] Optionally, the eighth network element is an AI resource node, and the seventh network element determines a suitable AI resource node based on the AI ​​resource request message and the AI ​​strategy.

[0663] Step 7a: The seventh network element sends a ninth message to the eighth network element. The ninth message includes AI resource configuration information, which is used to guide the AI ​​resource nodes to process and forward AI service data.

[0664] The AI ​​resource configuration information includes at least one of the following:

[0665] Data source information, such as the data source address;

[0666] Data characteristic ID of the first AI unit;

[0667] Data characteristic ID associated with data source information;

[0668] AI business detection information;

[0669] AI business processing rules;

[0670] AS address;

[0671] Step 7b: The eighth network element sends the tenth message to the seventh network element in response to the AI ​​resource configuration.

[0672] Step 8: The seventh network element sends a fourth message to the first entity in response to the third message, or this fourth message may be referred to as an AI resource request response message. The fourth message contains information about the eighth network element (AI resource node), and the information of the eighth network element (AI resource node) includes at least one of the following:

[0673] IP address;

[0674] ID information;

[0675] Fully Qualified Domain Name (FQDN);

[0676] Among them, the seventh network element establishes AI support based on the tenth message.

[0677] Step 9: Request communication resources and establish communication bearer.

[0678] Step 10: The first entity sends a second message to the first device. The second message is used in response to the first message to notify the UE of the result of the AI ​​service request, and includes, but is not limited to, at least one of the following:

[0679] Communication resource nodes;

[0680] AI resource node information;

[0681] First instruction;

[0682] Data characteristic ID of the first AI unit;

[0683] Task identifier.

[0684] Example 4-2: Combined with Implementation Method 2 in Example 2 above (simultaneous sending of monitoring requests and inference requests);

[0685] As shown in Figure 8b, the process includes:

[0686] Step 1: The UE sends a first message to the first entity. The first message is used to request resource allocation for the AI ​​service. The first message includes the first information, which is used to request performance monitoring of the AI ​​service and to request model inference of the second AI unit.

[0687] Optionally, the resources include at least one of communication resources, data resources, computing power resources, and AI resources.

[0688] Optionally, the first message can be a request message or a subscription message.

[0689] Optionally, the first information includes at least one of the following:

[0690] Monitoring type;

[0691] Monitoring and feedback methods;

[0692] Monitoring cycle;

[0693] Monitoring indicators;

[0694] Monitoring events;

[0695] Parameters related to monitoring events;

[0696] Data characteristic ID of the first AI unit;

[0697] The AI ​​function corresponding to the first AI unit;

[0698] The second AI unit is an AI unit that performs inference on the network side;

[0699] The data characteristic ID of the second AI unit;

[0700] Description information related to the second AI unit.

[0701] Optionally, the first entity may be a newly established fifth network element, such as an AI service management function or a collaborative control function. Alternatively, the first entity may be co-located with an existing entity, such as co-located with a second network element (communication management function) or co-located with a fourth network element (policy billing function).

[0702] Optionally, the aforementioned first information can be forwarded via a Non-Access Stratum (NAS) interface.

[0703] Optionally, the second AI unit includes at least one of the following:

[0704] The executable file corresponding to the second AI unit;

[0705] The reference model structure indication for the second AI unit;

[0706] The order of model parameters for the second AI unit;

[0707] The parameter file for the second AI unit;

[0708] Instructions from the second AI unit;

[0709] Modification layer instructions for the second AI unit;

[0710] The layer indicator corresponding to the second AI unit.

[0711] The descriptive information related to the second AI unit includes at least one of the following:

[0712] The effective duration of the second AI unit;

[0713] The configuration environment of the second AI unit, such as deep learning frameworks (TensorFlow, PyTorch, etc.) and supported library versions;

[0714] The second AI unit's model input information indicator is used to indicate data items, data format, data precision, etc.

[0715] The second AI unit's model output information indicator is used to indicate the accuracy, dimension, and storage format of the output data.

[0716] Step 2: The first entity and the fourth network element determine the AI ​​strategy.

[0717] Step 2 is the same as step 2 in the above embodiment 4-1.

[0718] Step 3a: The first entity sends a fifth message to the sixth network element. This fifth message is used to obtain model-related information. The model-related information includes at least one of the following:

[0719] The AI ​​function corresponding to the first AI unit;

[0720] Data characteristic ID of the first AI unit;

[0721] Relevant monitoring request information for the first AI unit;

[0722] Second AI unit;

[0723] The AI ​​function corresponding to the second AI unit;

[0724] The data characteristic ID of the second AI unit;

[0725] Description information related to the second AI unit.

[0726] Optionally, the aforementioned sixth network element is an AI model management function.

[0727] By sending the AI ​​function corresponding to the first AI unit, the sixth network element can identify the relevant AI unit being monitored.

[0728] Since the configuration for monitoring data collection needs to match the data characteristic ID, sending the data characteristic ID of the first AI unit helps the sixth network element determine whether monitoring data related to the first AI unit can be collected.

[0729] By sending the relevant monitoring request information of the first AI unit, it is possible to indicate to the sixth network element whether it is necessary to obtain the truth value based on the AI ​​unit, and to help the sixth network element determine the corresponding AI unit so as to obtain the truth value in subsequent processes.

[0730] By sending the aforementioned second AI unit, the sixth network can become aware of the AI ​​unit that needs to perform inference and can manage the lifecycle of that AI unit on the sixth network element side.

[0731] By sending the AI ​​function corresponding to the second AI unit, the sixth network element can learn the AI ​​function corresponding to the AI ​​unit that needs to be inferred on the network side.

[0732] By sending the data characteristic ID of the second AI unit, the sixth network element can learn the data characteristics of the AI ​​unit that needs to be inferred on the network side. Optionally, the sixth network element can determine whether the network configuration can support the data collection of the corresponding data characteristics.

[0733] By sending the descriptive information related to the second AI unit, the sixth network element can learn about the operation, configuration, input / output interfaces, and other information of the AI ​​unit that needs to be inferred on the network side.

[0734] Steps 3b to 4b in this embodiment are the same as steps 3b to 4b in embodiment 4-1.

[0735] Step 5: The first entity sends a third message to the seventh network element. The third message is for AI service resource request.

[0736] Optionally, this seventh network element is an AI resource management function.

[0737] Optionally, the AI ​​resource request information includes at least one of the following:

[0738] Model information, such as the model or its download address;

[0739] Data source information, such as the data source address;

[0740] Data characteristic ID of the first AI unit;

[0741] Data characteristic ID associated with data source information;

[0742] AI business computational load, such as computing power size and type;

[0743] Access-stratum (AS) address;

[0744] The first identifier is an identifier for the model inference task associated with the performance monitoring task of the first information request.

[0745] The aforementioned first identifier enables the association of the performance monitoring task of the first information request with the AI ​​resource nodes of the previous inference task or service, which is beneficial for related data interaction.

[0746] Steps 6 to 10 in this embodiment are the same as steps 6 to 10 in embodiment 4-1.

[0747] Example 4-3: Combined with implementation method 3 in Example 2 above (monitoring requests and inference requests are sent separately);

[0748] As shown in Figure 8c, the process includes:

[0749] Step 1: The UE sends a first message to the first entity. The first message is used to request resource allocation for the AI ​​service. The first message includes the first information, which is used to request performance monitoring of the AI ​​service. The first information includes a first identifier, which is the identifier of the model inference task associated with the performance monitoring task requested by the first information.

[0750] Optionally, the resources include at least one of communication resources, data resources, computing power resources, and AI resources.

[0751] Optionally, the first message can be a request message or a subscription message.

[0752] Optionally, the first information further includes at least one of the following:

[0753] Monitoring type;

[0754] Monitoring and feedback methods;

[0755] Monitoring cycle;

[0756] Monitoring indicators;

[0757] Monitoring events;

[0758] Parameters related to monitoring events;

[0759] Data characteristic ID of the first AI unit;

[0760] The AI ​​function corresponding to the first AI unit.

[0761] The aforementioned first identifier enables the association of the performance monitoring task of the first information request with the AI ​​resource nodes of the previous inference task or service, which is beneficial for related data interaction.

[0762] Optionally, the first entity may be a newly established fifth network element, such as an AI service management function or a collaborative control function. Alternatively, the first entity may be co-located with an existing entity, such as co-located with a second network element (communication management function) or co-located with a fourth network element (policy billing function).

[0763] Optionally, the aforementioned first information can be forwarded via a Non-Access Stratum (NAS) interface.

[0764] Step 2: The first entity and the fourth network element determine the AI ​​strategy.

[0765] Step 2 is the same as step 2 in the above embodiment 4-1.

[0766] Step 3a: The first entity sends a fifth message to the sixth network element. This fifth message is used to obtain model-related information. The model-related information includes at least one of the following:

[0767] The AI ​​function corresponding to the first AI unit;

[0768] Data characteristic ID of the first AI unit;

[0769] Relevant monitoring request information for the first AI unit;

[0770] The first identifier is the identifier of the model inference task associated with the performance monitoring task of the first information request.

[0771] The first identifier enables the management of the performance monitoring task of the first information request and the second AI unit used in the previous inference task, thereby facilitating the sixth network element to monitor the associated inference task.

[0772] Optionally, the aforementioned sixth network element is an AI model management function.

[0773] By sending the AI ​​function corresponding to the first AI unit, the sixth network element can identify the relevant AI unit being monitored.

[0774] Since the configuration for monitoring data collection needs to match the data characteristic ID, sending the data characteristic ID of the first AI unit helps the sixth network element determine whether monitoring data related to the first AI unit can be collected.

[0775] By sending the relevant monitoring request information of the first AI unit, the sixth network element can be informed that the acquisition of the truth value depends on the model inference of the network-side AI unit.

[0776] Steps 3b to 3c in this embodiment are the same as steps 3b to 3c in embodiment 4-1.

[0777] Step 4a: The first entity sends a seventh message to the ninth network element, the seventh message being used to obtain the data source information of the first AI unit.

[0778] This ninth network element can be a data plane management function.

[0779] The seventh message can be a request message or a subscription message. It is used to obtain the data source and determine the data source information; optionally, the seventh message includes at least one of the following:

[0780] The data characteristic ID of the first AI unit; the data characteristic ID of the first AI unit can indicate the data characteristic ID required by the first AI unit, which is helpful for the ninth network element to determine the data source information;

[0781] A first identifier, based on which the performance monitoring task and the associated inference task can be instructed to use the same data source information;

[0782] The data source information associated with the data characteristic ID of the first AI unit can be used to indicate historical data source information that matches the data characteristic ID of the first AI unit.

[0783] Optionally, the seventh message is determined based on the description information related to the first AI unit in the first information.

[0784] When the first entity selects the data plane management function instance, it considers at least one of the following factors:

[0785] Data Network Name (DNN);

[0786] Single Network Slice Selection Assistance Information (S-NSSAI);

[0787] AI business identifier;

[0788] Supported AI services;

[0789] Service area information;

[0790] Data source information.

[0791] Step 4b in this embodiment is the same as step 4b in embodiment 4-1.

[0792] Step 5: The first entity sends a third message to the seventh network element. The third message is for AI service resource request.

[0793] Optionally, this seventh network element is an AI resource management function.

[0794] Optionally, the AI ​​resource request information includes at least one of the following:

[0795] Model information, such as the model or its download address;

[0796] Data source information, such as the data source address;

[0797] Data characteristic ID of the first AI unit;

[0798] Data characteristic ID associated with data source information;

[0799] The first identifier is an identifier for the model inference task associated with the performance monitoring task of the first information request;

[0800] AI business computational load, such as computing power size and type;

[0801] Access-stratum (AS) address;

[0802] Steps 6 to 9 in this embodiment are the same as steps 6 to 9 in embodiment 4-1.

[0803] Step 10: The first entity sends a second message to the first device. The second message is used in response to the first message to notify the UE of the result of the AI ​​service request, and includes, but is not limited to, at least one of the following:

[0804] Communication resource nodes;

[0805] AI resource node information;

[0806] Data characteristic ID of the first AI unit;

[0807] Task identifier.

[0808] Example 4-4: Combined with implementation method 4 in Example 2 above (first send the inference request, then modify the inference request into a monitoring request and an inference request);

[0809] As shown in Figure 8d, the process includes:

[0810] Step 1: The UE sends a first message to the first entity. The first message is used to request resource allocation for the AI ​​service. The first message includes the first information, which is used to request that the model inference task be modified into a performance monitoring and model inference task. The first information includes the associated inference task ID.

[0811] The associated inference task ID can be understood as the ID of the inference task of the fourth information request, i.e., the second identifier.

[0812] The aforementioned associated inference task IDs can be linked to previous inference tasks / services, facilitating the monitoring of associated inference tasks. Furthermore, using these associated inference task IDs eliminates the need for processes such as confirming data feature IDs.

[0813] Optionally, the resources include at least one of communication resources, data resources, computing power resources, and AI resources.

[0814] Optionally, the first message can be a request message or a subscription message.

[0815] Optionally, the first information further includes at least one of the following:

[0816] Monitoring type;

[0817] Monitoring and feedback methods;

[0818] Monitoring cycle;

[0819] Monitoring indicators;

[0820] Monitoring events;

[0821] Parameters related to monitoring events.

[0822] The second identifier mentioned above can be used to associate the performance monitoring task of the first information request with the AI ​​resource nodes of the previous inference task or service, which is beneficial for related data interaction.

[0823] Optionally, the first entity may be a newly established fifth network element, such as an AI service management function or a collaborative control function. Alternatively, the first entity may be co-located with an existing entity, such as co-located with a second network element (communication management function) or co-located with a fourth network element (policy billing function).

[0824] Optionally, the aforementioned first information can be forwarded via a Non-Access Stratum (NAS) interface.

[0825] Step 2: The first entity and the fourth network element determine the AI ​​strategy.

[0826] Step 2 is the same as step 2 in the above embodiment 4-1.

[0827] Step 3a: The first entity sends a fifth message to the sixth network element. This fifth message is used to obtain model-related information. The model-related information includes at least one of the following:

[0828] Relevant monitoring request information for the first AI unit;

[0829] The associated inference task ID (second identifier).

[0830] This second identifier enables the management of the performance monitoring task of the first information request and the second AI unit used in the previous inference task, thereby facilitating the sixth network element to monitor the associated inference task.

[0831] Steps 3c to 4b in this embodiment are the same as steps 3c to 4b in embodiment 4-1.

[0832] Step 5: The first entity sends a third message to the seventh network element. The third message is for AI service resource request.

[0833] Optionally, this seventh network element is an AI resource management function.

[0834] Optionally, the AI ​​resource request information includes at least one of the following:

[0835] Model information, such as the model or its download address;

[0836] Data source information, such as the data source address;

[0837] The associated reasoning task ID (second identifier);

[0838] AI business computational load, such as computing power size and type;

[0839] Access-stratum (AS) address;

[0840] Steps 6 to 9 in this embodiment are the same as steps 6 to 9 in embodiment 4-1.

[0841] Step 10: The first entity sends a second message to the first device. The second message is used in response to the first message to notify the UE of the result of the AI ​​service request, and includes, but is not limited to, at least one of the following:

[0842] Communication resource nodes;

[0843] AI resource node information;

[0844] Third identifier.

[0845] Example 2 describes various implementation methods for the monitoring request process, including separate requests and requests together with inference requests. Example 4 is a refinement of the process for external AI services, which allows the network to provide monitoring services during the execution of AI services, thus avoiding the deterioration of AI service performance.

[0846] Example 5: Combine the user plane or data plane process of external AI services with three monitoring schemes and control plane processes.

[0847] Example 5-1: A monitoring scheme based on the interaction of inference results from the first AI unit, wherein the model inference related to AI business is first performed by the network for the model inference of the second AI unit.

[0848] As shown in Figure 9a, the process includes:

[0849] Method 1: Feed back the reasoning results and monitoring results separately;

[0850] Step 1: The UE sends AI service data to the eighth network element, and the AI ​​service data includes at least one of the following:

[0851] Second AI unit;

[0852] Description information of the second AI unit;

[0853] The data characteristic ID of the second AI unit.

[0854] Step 2: The eighth network element obtains intranet data from the tenth network element.

[0855] Optionally, the interaction signaling between the eighth network element and the tenth network element may also include: the data characteristic ID of the first AI unit, and / or the data characteristic ID corresponding to the data source.

[0856] The eighth network element determines the data source information in at least one of the following ways:

[0857] Obtain from AI resource allocation information;

[0858] Obtain from local configuration.

[0859] Step 3: The eighth network element performs model inference for the second AI unit.

[0860] Step 4: The eighth network element sends AI service data to the first device. The AI ​​service data includes at least one of the following: the inference result of the second AI unit; the identifier of the second AI unit; the data characteristic ID of the second AI unit.

[0861] Step 5: The UE performs model inference for the first AI unit.

[0862] Step 6: The UE sends AI service data to the eighth network element, which includes the inference results of the first AI unit.

[0863] Step 7: The eighth network element obtains the true value and calculates the performance monitoring results.

[0864] Optionally, the eighth network element obtains a request from the monitoring-related AI unit. This request can be used to indicate that truth value acquisition requires model inference from the network-side AI unit. Based on the function of the first AI unit or the function of the second AI unit, an AI unit for obtaining truth value can be found. Through the AI ​​unit obtained through inference, the performance monitoring truth value is obtained.

[0865] Specifically, the model input data for the first AI unit is determined based on the data collected in step 3.

[0866] Step 8: The eighth network element sends AI service data to the UE. The AI ​​service data includes at least one of the following: monitoring events; monitoring KPIs; timestamps.

[0867] Step 9: AI task completion notification and ACK.

[0868] Method 2: Provide feedback on reasoning results and monitoring results together.

[0869] Step 1: The UE sends AI service data to the eighth network element, and the AI ​​service data includes at least one of the following:

[0870] Second AI unit;

[0871] Description information of the second AI unit;

[0872] The data characteristic ID of the second AI unit.

[0873] Step 2: The eighth network element obtains intranet data from the tenth network element.

[0874] Optionally, the interaction signaling between the eighth network element and the tenth network element may also include: the data characteristic ID of the first AI unit, and / or the data characteristic ID corresponding to the data source.

[0875] The eighth network element determines the data source information in at least one of the following ways:

[0876] Obtain from AI resource allocation information;

[0877] Obtain from local configuration.

[0878] Step 3: The eighth network element performs model inference for the second AI unit.

[0879] Step 4: The UE sends AI service data to the eighth network element, which includes the historical inference results of the first AI unit.

[0880] Step 5: The eighth network element obtains the truth value.

[0881] Optionally, the eighth network element obtains a request from the monitoring-related AI unit. This request can be used to indicate that truth value acquisition requires model inference from the network-side AI unit. Based on the function of the first AI unit or the function of the second AI unit, an AI unit for obtaining truth value can be found. Through the AI ​​unit obtained through inference, the performance monitoring truth value is obtained.

[0882] Optionally, if the eighth network element obtains the historical inference result of the first AI unit, the performance monitoring result is calculated based on the historical inference result and the truth value.

[0883] Step 6: The eighth network element sends AI service data to the UE. The AI ​​service data includes at least one of the following: monitoring events; monitoring KPIs; timestamps; model inference results of the second AI unit; performance monitoring results.

[0884] Step 7: The UE performs model inference for the first AI unit.

[0885] Step 8: AI task completion notification and ACK.

[0886] Example 5-2: A monitoring scheme based on the interaction of the third unit, wherein the model inference related to AI business is first performed by the network device for the model inference of the second AI unit;

[0887] As shown in Figure 9b, the process includes:

[0888] Method 1: Feedback the reasoning results and monitoring results separately.

[0889] Step 1: The UE sends AI service data to the eighth network element. The AI ​​service data includes at least one of the following: the second AI unit; the description information of the second AI unit; and the data characteristic ID of the second AI unit.

[0890] Step 2: The eighth network element obtains intranet data from the tenth network element.

[0891] Optionally, the interaction signaling between the eighth network element and the tenth network element may also include: the data characteristic ID of the first AI unit, and / or the data characteristic ID corresponding to the data source.

[0892] The eighth network element determines the data source information in at least one of the following ways:

[0893] Obtain from AI resource allocation information;

[0894] Obtain from local configuration.

[0895] Step 3: The eighth network element performs model inference for the second AI unit.

[0896] Step 4: The eighth network element sends AI service data to the UE. The AI ​​service data includes at least one of the following: the inference result of the second AI unit; the identifier of the second AI unit; the data characteristic ID of the second AI unit.

[0897] Step 5: The UE performs model inference for the first AI unit.

[0898] Optionally, the UE performs model inference for the first AI unit based on the model inference results of the second AI unit.

[0899] Step 6: The UE sends AI service data to the eighth network element, including at least one of the following: the third AI unit; the description information of the third AI unit; the data characteristic ID of the third AI unit.

[0900] Step 7: The eighth network element performs model inference for the third AI unit and obtains the inference results related to the first AI unit.

[0901] Step 8: The eighth network element obtains the true value and calculates the monitoring results.

[0902] Optionally, the eighth network element obtains a request from the monitoring-related AI unit. This request can be used to indicate that truth value acquisition requires model inference from the network-side AI unit. Based on the function of the first AI unit or the function of the second AI unit, an AI unit for obtaining truth value can be found. Through the AI ​​unit obtained through inference, the performance monitoring truth value is obtained.

[0903] Specifically, the model input data for the first AI unit is determined based on the data collected in step 3.

[0904] The monitoring results include at least one of the following: monitoring events, monitoring KPIs, and timestamps.

[0905] Step 9: The eighth network element sends AI service data to the UE, including at least one of the following: monitoring events; monitoring KPIs; timestamps.

[0906] Step 10: AI task completion notification and ACK.

[0907] Method 2: Provide feedback on both the reasoning results and the monitoring results;

[0908] Step 1: The UE sends AI service data to the eighth network element. The AI ​​service data includes at least one of the following: a second AI unit; description information of the second AI unit; data characteristic ID of the second AI unit; a third AI unit; description information of the third AI unit; data characteristic ID of the third AI unit.

[0909] Step 2: The eighth network element obtains intranet data from the tenth network element.

[0910] Optionally, the interaction signaling between the eighth network element and the tenth network element may also include: the data characteristic ID of the first AI unit, and / or the data characteristic ID corresponding to the data source.

[0911] The eighth network element determines the data source information in at least one of the following ways:

[0912] Obtain from AI resource allocation information;

[0913] Obtain from local configuration.

[0914] Step 3: The eighth network element performs model inference for the second AI unit.

[0915] Step 4: The eighth network element performs model inference for the third AI unit.

[0916] Optionally, the eighth network element obtains the reasoning results related to the first AI unit based on the model reasoning of the third AI unit.

[0917] Step 5: The eighth network element obtains the true value and calculates the monitoring results.

[0918] Optionally, the eighth network element obtains a request from the monitoring-related AI unit. This request can be used to indicate that truth value acquisition requires model inference from the network-side AI unit. Based on the functions of the first or second AI unit, an AI unit for obtaining truth values ​​can be found. Through the AI ​​unit obtained through inference, the performance monitoring truth value is acquired.

[0919] The monitoring results include at least one of the following: monitoring events, monitoring KPIs, and timestamps.

[0920] Step 6: The eighth network element sends AI service data to the UE, including at least one of the following: the inference result of the second AI unit; the identifier of the second AI unit; the data characteristic ID of the second AI unit; monitoring events; monitoring KPIs; timestamps.

[0921] Step 7: The UE performs model inference for the first AI unit.

[0922] Optionally, the UE performs model inference of the first AI unit based on the model inference result of the second AI unit, and obtains the performance monitoring result of the AI ​​service based on the model inference result of the first AI unit and the monitoring result of the transmission of the eighth network element.

[0923] Step 8: AI task completion notification and ACK.

[0924] Example 5-3: A monitoring method based on network-provided truth values, wherein the model inference related to AI services is first performed by the network device as the model inference of the second AI unit, and the inference task and the monitoring task are executed in different eighth network elements.

[0925] As shown in Figure 9c, the process includes:

[0926] Method 1: Feedback the reasoning results and monitoring results separately.

[0927] Step 1: The UE sends AI service data to the eighth network element. The AI ​​service data includes at least one of the following: the second AI unit; the description information of the second AI unit; and the data characteristic ID of the second AI unit.

[0928] Step 2: The eighth network element obtains intranet data from the tenth network element.

[0929] Optionally, the interaction signaling between the eighth network element and the tenth network element may also include: the data characteristic ID of the first AI unit, and / or the data characteristic ID corresponding to the data source.

[0930] The eighth network element determines the data source information in at least one of the following ways:

[0931] Obtain from AI resource allocation information;

[0932] Obtain from local configuration.

[0933] Step 3: The eighth network element performs model inference for the second AI unit.

[0934] Step 4: The eighth network element sends AI service data to the first device. The AI ​​service data includes at least one of the following: the inference result of the second AI unit; the identifier of the second AI unit; the data characteristic ID of the second AI unit.

[0935] Step 5: The UE performs model inference for the first AI unit.

[0936] Step 6: The eighth network element 2 obtains network data from the tenth network element.

[0937] Optionally, the interaction signaling between the eighth network element 2 and the tenth network element may also include at least one of the data characteristic ID of the first AI unit and the data characteristic ID corresponding to the data source.

[0938] The eighth network element determines the data source information in at least one of the following ways:

[0939] Obtain from AI resource allocation information;

[0940] Obtain from local configuration.

[0941] Step 7: The eighth network element 2 obtains the truth information and calculates the monitoring results.

[0942] Optionally, the process by which the eighth network element 2 obtains truth information is the same as the process by which the eighth network obtains truth information.

[0943] Specifically, the model input data for the first AI unit is determined based on the data collected in step 3.

[0944] Optionally, the truth information includes at least one of the following: truth values ​​related to the second AI unit; truth values ​​related to the first AI unit; truth values ​​related to the first AI function; and truth values ​​related to the second AI function.

[0945] Among them, the first AI function is the AI ​​function corresponding to the first AI unit, and the second AI function is the AI ​​function corresponding to the second AI unit.

[0946] Step 8: The eighth network element 2 sends AI service data to the UE, which includes truth information. Optionally, the AI ​​service data also includes a timestamp.

[0947] Step 9: The UE calculates the service monitoring results based on the AI ​​service data obtained in Step 8.

[0948] Step 10: AI task completion notification and ACK.

[0949] Method 2: Providing feedback on reasoning results and monitoring results together

[0950] Step 1: The UE sends AI service data to the eighth network element. The AI ​​service data includes at least one of the following: the second AI unit; the description information of the second AI unit; and the data characteristic ID of the second AI unit.

[0951] Step 2: The eighth network element obtains intranet data from the tenth network element.

[0952] Optionally, the interaction signaling between the eighth network element and the tenth network element may also include: the data characteristic ID of the first AI unit, and / or the data characteristic ID corresponding to the data source.

[0953] The eighth network element determines the data source information in at least one of the following ways:

[0954] Obtain from AI resource allocation information;

[0955] Obtain from local configuration.

[0956] Step 3: The eighth network element performs model inference for the second AI unit.

[0957] Step 4: The eighth network element 2 obtains network data from the tenth network element.

[0958] Step 5: The eighth network element 2 obtains truth information.

[0959] Step 6: The eighth network element receives truth information from the eighth network element 2 and calculates the monitoring results.

[0960] Optionally, the process by which the eighth network element 2 obtains truth information is the same as the process by which the eighth network obtains truth information.

[0961] Specifically, the model input data for the first AI unit is determined based on the data collected in step 3.

[0962] Optionally, the truth information includes at least one of the following:

[0963] Truth values ​​related to the second AI unit; truth values ​​related to the first AI unit; truth values ​​related to the first AI function; truth values ​​related to the second AI function; timestamp.

[0964] Step 7: The eighth network element sends AI service data to the UE. The AI ​​service data includes at least one of the following: the inference result of the second AI unit; the identifier of the second AI unit; the data characteristic ID of the second AI unit; and truth information.

[0965] Step 8: UE calculates monitoring KPIs.

[0966] Step 9: AI task completion notification and ACK.

[0967] Example 3 shows the general user interaction flow corresponding to the three performance monitoring schemes. Example 5 is a refinement of the process of external AI services, which enables performance monitoring during the execution of AI services and avoids the deterioration of AI service performance.

[0968] The AI ​​unit / AI model in this application embodiment may also be referred to as AI unit, AI model, machine learning (ML) model, ML unit, AI structure, AI function, AI characteristic, machine learning model, neural network, neural network function, neural network functionality, etc. Alternatively, the AI ​​unit / AI model may refer to a processing unit capable of implementing specific algorithms, formulas, processing flows, capabilities, etc., related to AI. Or, the AI ​​unit / AI model may be a processing method, algorithm, function, module, or unit for a specific dataset. Alternatively, the AI ​​unit / AI model may be a processing method, algorithm, function, module, or unit running on AI / ML related hardware such as GPU, NPU, TPU, and ASIC. This invention does not impose specific limitations in these respects. Optionally, the specific dataset includes the input and / or output of the AI ​​unit / AI model.

[0969] Optionally, the identifier of the AI ​​unit / AI model may be an AI model identifier, an AI structure identifier, an AI algorithm identifier, or an identifier of a specific dataset associated with the AI ​​unit / AI model, or an identifier of a specific scenario, environment, channel characteristics, or device related to the AI / ML, or an identifier of a function, feature, capability, or module related to the AI / ML. This application does not specifically limit this.

[0970] The information transmission method provided in this application can be executed by an information transmission device. This application uses an information transmission device executing the information transmission method as an example to illustrate the information transmission device provided in this application.

[0971] This application provides an information transmission device. As an example, the information transmission device may be a communication device or a component within a communication device, such as a chip. The communication device may be a terminal, a network-side device, or a server, etc. Exemplarily, the terminal may include, but is not limited to, the type of terminal 11 listed above, and the network-side device may include, but is not limited to, the type of network-side device 12 listed above. This application does not impose specific limitations.

[0972] The information transmission device includes a receiving module, a transmitting module, and a processing module. These modules can be implemented in software or hardware. When implemented in hardware, the processing module can be implemented by a processor. For example, the processor can include general-purpose processors, special-purpose processors, such as a Central Processing Unit (CPU), microprocessor, Digital Signal Processor (DSP), Artificial Intelligence (AI) processor, Graphics Processing Unit (GPU), Application Specific Integrated Circuit (ASIC), Network Processor (NP), Field Programmable Gate Array (FPGA), or other programmable logic devices, gate circuits, transistors, discrete hardware components, etc. The receiving and transmitting modules can be implemented by a communication interface, which can include one or more of the following: transceiver, pins, circuits, bus, radio frequency unit, etc.

[0973] Specifically, referring to Figure 10, when the information transmission device is a terminal or a component in a terminal, the information transmission device 1000 includes a first sending module 1001, which is used to send first information to a second device. The first information is used to request performance monitoring of artificial intelligence (AI) services.

[0974] The first receiving module 1002 is used to obtain the second information sent by the second device, the second information including information related to the performance monitoring result of the AI ​​service or information related to the performance monitoring truth value of the AI ​​service.

[0975] The second device includes network devices.

[0976] Optionally, the apparatus in this application embodiment further includes:

[0977] The second sending module is used to send first AI service data to the second device, wherein the first AI service data includes the model inference results of the first AI unit;

[0978] The second information is obtained based on the model reasoning results of the first AI unit.

[0979] Optionally, the first information is further used to request the network side to perform model inference for the second AI unit; or, the device further includes:

[0980] The third sending module is used to send third information to the second device, the third information being used to request the network side to perform model inference for the second AI unit;

[0981] The second AI unit is an AI unit paired with the first AI unit, and the first AI unit is the AI ​​unit used by the first device for model inference.

[0982] Optionally, the apparatus in this application embodiment further includes:

[0983] The fifth sending module is used to send first AI service data to the second device, the first AI service data including the model inference results of the first AI unit.

[0984] Optionally, the apparatus in this application embodiment further includes:

[0985] The fifth receiving module is used to acquire the model inference results of the second AI unit sent by the second device;

[0986] The first processing module is used to perform model inference of the first AI unit based on the model inference result of the second AI unit, and obtain the model inference result of the first AI unit;

[0987] The sixth sending module is used to send second AI service data to the second device. The second AI service data includes at least one of the following: the model inference result of the first AI unit; and a third AI unit, wherein the third AI unit is an AI unit related to the first AI unit.

[0988] Optionally, when the first device sends the third information to the second device, the first information includes:

[0989] The first identifier of the model inference task associated with the performance monitoring task of the first information request.

[0990] Optionally, the first information is also used to request the network side to perform model inference for the second AI unit;

[0991] The device further includes:

[0992] The seventh sending module is used to send fourth information to the second device, the fourth information being used to request the network side to perform model inference for the second AI unit;

[0993] The first information includes task update information, which is used to replace the model inference task requested by the fourth information with the model inference and performance monitoring task requested by the first information.

[0994] Optionally, the first information includes at least one of the following:

[0995] The second AI unit;

[0996] Description information related to the second AI unit;

[0997] The data characteristic information of the second AI unit.

[0998] Optionally, the performance monitoring result related information of the AI ​​service includes the performance monitoring result of the first AI unit, or the performance monitoring truth value related information of the AI ​​service includes at least one of the performance monitoring truth value associated with the first AI unit and the performance monitoring truth value associated with the first AI function;

[0999] Wherein, the first AI unit is the AI ​​unit of the first device that performs model inference, and the first AI function is the AI ​​function corresponding to the first AI unit.

[1000] Optionally, the device further includes:

[1001] The eighth sending module is used to send a third AI unit to the second device, wherein the third AI unit is an AI unit related to the first AI unit;

[1002] The performance monitoring results related to the AI ​​service include the performance monitoring results of the third AI unit and the performance monitoring results associated with the second AI function; or, the performance monitoring truth value related to the AI ​​service includes the performance monitoring truth value associated with the third AI unit.

[1003] The second AI function is the AI ​​function corresponding to the third AI unit.

[1004] Optionally, the device further includes:

[1005] The second processing module is used to obtain the performance monitoring result of the first AI unit based on the performance monitoring result of the third AI unit or the performance monitoring true value associated with the third AI unit.

[1006] Optionally, the performance monitoring results include at least one of the following:

[1007] Performance monitoring KPIs;

[1008] Monitoring events;

[1009] Timestamp.

[1010] Optionally, the first information includes at least one of the following:

[1011] Monitoring type;

[1012] Monitoring and feedback methods;

[1013] Monitoring cycle;

[1014] Monitoring indicators;

[1015] Monitoring events;

[1016] Parameters related to monitoring events.

[1017] Optionally,

[1018] The first information is information transmitted between the first network function and / or the second network function in the first device and the second device; and / or,

[1019] The second information is information transmitted between the second network function in the first device and the second device; and / or,

[1020] The third information is the information transmitted between the first device and the first network function; and / or,

[1021] The fourth information is the information transmitted between the first device and the first network function;

[1022] The first network function is used to convert external network requirements into internal requirements.

[1023] The second network function is used for at least one of business processing and the implementation of business policy rules;

[1024] The third and fourth information are used to request the network side to perform model inference for the second AI unit.

[1025] Optionally, the first sending module is used to:

[1026] Send a first message to the first network function in the second device. The first message is used to request resource allocation for AI services. The first message includes the first information. The first information also includes at least one of the following: data characteristic information of the first AI unit and AI function corresponding to the first AI unit.

[1027] The device further includes:

[1028] The sixth receiving module is used to acquire a second message sent by the first network function in the second device. The second message is a response message to the first message. The second message is carried by control plane signaling and includes at least one of the following: communication resource node information, AI resource node information, data characteristic information of the first AI unit, and a first indication; wherein the first indication is used to indicate whether the data characteristic information of the first AI unit matches the data characteristic information that the network can collect.

[1029] Referring to Figure 11, when the information transmission device is a network-side device or a component of a network-side device, the information transmission device 1100 includes a second receiving module 1101, which is used to obtain first information sent by the first device. The first information is used to request performance monitoring of artificial intelligence (AI) services.

[1030] The fourth sending module 1102 is used to send second information to the first device, the second information including information related to the performance monitoring result of the AI ​​service or information related to the performance monitoring truth value of the AI ​​service;

[1031] The first device includes a terminal, an application function (AF) or an OTT server.

[1032] Optionally, the device further includes:

[1033] The third receiving module is used to acquire the first AI service data sent by the first device, wherein the first AI service data includes the model inference results of the first AI unit;

[1034] The second information is obtained based on the model reasoning results of the first AI unit.

[1035] Optionally, the first information is also used to request the network side to perform model inference for the second AI unit;

[1036] Alternatively, the device may further include:

[1037] The fourth receiving module is used to obtain the third information sent by the first device, the third information being used to request the network side to perform model inference of the second AI unit;

[1038] The second AI unit is an AI unit paired with the first AI unit, and the first AI unit is the AI ​​unit used by the first device for model inference.

[1039] Optionally, the device further includes:

[1040] The seventh receiving module is used to acquire the first AI service data sent by the first device, the first AI service data including the model inference result of the first AI unit; the second device performs model inference of the second AI unit based on the model inference result of the first AI unit to obtain the model inference result of the second AI unit.

[1041] Optionally, the device further includes:

[1042] The ninth sending module is used to send the model inference results of the second AI unit to the first device;

[1043] The eighth receiving module is used to acquire the second AI service data sent by the first device. The second AI service data includes at least one of the following: the model inference result of the first AI unit; and the third AI unit, which is an AI unit related to the first AI unit.

[1044] The third processing module is used to obtain the second information based on the second AI business data.

[1045] Optionally, the first information includes a first identifier of the model inference task associated with the performance monitoring task requested by the first information, and the fourth receiving module is used to:

[1046] Based on the first identifier, the third information is obtained, wherein the third information includes the first identifier of the model reasoning task.

[1047] Optionally, the first information is further used to request the second device to perform model inference for the second AI unit;

[1048] The device also includes:

[1049] The ninth receiving module is used to obtain the fourth information sent by the first device, the fourth information being used to request the network side to perform model inference for the second AI unit;

[1050] The first information includes task update information, which is used to replace the model inference task requested by the fourth information with the model inference and performance monitoring task requested by the first information.

[1051] Optionally, the first information includes at least one of the following:

[1052] The second AI unit;

[1053] Description information related to the second AI unit;

[1054] The data characteristic information of the second AI unit.

[1055] Optionally, the performance monitoring result related information of the AI ​​service includes the performance monitoring result of the first AI unit, or the performance monitoring truth value related information of the AI ​​service includes at least one of the performance monitoring truth value associated with the first AI unit and the performance monitoring truth value associated with the first AI function;

[1056] Wherein, the first AI unit is the AI ​​unit of the first device that performs model inference, and the first AI function is the AI ​​function corresponding to the first AI unit.

[1057] Optionally, the device further includes:

[1058] The tenth receiving module is used to acquire the third AI unit sent by the first device, wherein the third AI unit is an AI unit related to the first AI unit;

[1059] The performance monitoring results related to the AI ​​service include the performance monitoring results of the third AI unit and the performance monitoring results associated with the second AI function; or, the performance monitoring truth value related to the AI ​​service includes the performance monitoring truth value associated with the third AI unit.

[1060] The second AI function is the AI ​​function corresponding to the third AI unit.

[1061] Optionally, the performance monitoring results include at least one of the following:

[1062] Performance monitoring KPIs;

[1063] Monitoring events;

[1064] Timestamp.

[1065] Optionally, the first information includes at least one of the following:

[1066] Monitoring type;

[1067] Monitoring and feedback methods;

[1068] Monitoring cycle;

[1069] Monitoring indicators;

[1070] Monitoring events;

[1071] Parameters related to monitoring events.

[1072] Optionally, the first information is information transmitted between a first network function and / or a second network function in the first device and the second device; and / or,

[1073] The second information is information transmitted between the second network function in the first device and the second device; and / or,

[1074] The third information is the information transmitted between the first device and the first network function; and / or,

[1075] The fourth information is the information transmitted between the first device and the first network function;

[1076] The first network function is used to convert external network requirements into internal requirements.

[1077] The second network function is used for at least one of business processing and the implementation of business policy rules;

[1078] The third and fourth information are used to request the network side to perform model inference for the second AI unit.

[1079] Optionally, the second receiving module is used to obtain a first message sent by the first device through a first network function in the second device. The first message is used to request resource allocation for AI services. The first message includes the first information, and the first information further includes at least one of the following: data characteristic information of the first AI unit and AI function corresponding to the first AI unit.

[1080] The method further includes:

[1081] The tenth sending module is used to send a second message to the first device through the first network function in the second device. The second message is a response message to the first message. The second message is carried by control plane signaling and includes at least one of the following: communication resource node information, AI resource node information, data characteristic information of the first AI unit, and a first indication; wherein, the first indication is used to indicate whether the data characteristic information of the first AI unit matches the data characteristic information that the network can collect.

[1082] Optionally, the device further includes:

[1083] The eleventh sending module is used to send a third message to the third network function through the first network function. The third message is used for resource requests for AI services. The third message includes at least one of the following: data source information of the first AI unit, data characteristic information of the first AI unit, relevant monitoring request information of the first AI unit, data characteristic information associated with the data source information of the first AI unit, data characteristic information of the second AI unit, data characteristic information associated with the data source information of the second AI unit, and a first identifier; wherein, the first identifier is the identifier of the model inference task associated with the performance monitoring task of the first information request.

[1084] The eleventh receiving module is used to obtain a fourth message sent by the third network function through the first network function. The fourth message is a response message to the third message and includes AI resource node information configured according to the third message.

[1085] The third network function is used to manage business resources.

[1086] Optionally, the device further includes:

[1087] The twelfth sending module is used to send a fifth message to the fourth network function through the first network function. The fifth message is used to request model-related information. The fifth message includes at least one of the following: the AI ​​function corresponding to the first AI unit, the data characteristic information of the first AI unit, the relevant monitoring request information of the first AI unit, the second AI unit, the function corresponding to the second AI unit, the data characteristic information of the second AI unit, the relevant description information of the second AI unit, and the first identifier; wherein, the first identifier is the identifier of the model inference task associated with the performance monitoring task of the first information request.

[1088] The twelfth receiving module is used for the first network function to obtain the sixth message sent by the fourth network function, the sixth message including model-related information, or including model-related information and the first indication;

[1089] The fourth network function is used for AI model management.

[1090] Optionally, the device further includes:

[1091] The thirteenth sending module is used to send a seventh message to the fifth network function through the first network function. The seventh message is used to obtain the data source information of the first AI unit. The seventh message includes at least one of the following: data characteristic information of the first AI unit, the first identifier, and historical data source information associated with the data characteristic information of the first AI unit.

[1092] The thirteenth receiving module is used to obtain the eighth message sent by the fifth network function through the second network function, the eighth message including the data source information of the first AI unit;

[1093] The fifth network function is used for data management.

[1094] In this embodiment, a first device sends first information to a second device, the first information being a request for performance monitoring of an artificial intelligence (AI) service. The first device then receives second information sent by the second device, the second information including information related to the performance monitoring results of the AI ​​service or information related to the performance monitoring truth value of the AI ​​service. The first device includes a terminal, an application function (AF) server, or an OTT server; the second device includes a network device. This allows the first device to monitor the model inference of the AI ​​unit based on information provided by the network, thereby determining whether the model inference accuracy of the AI ​​unit has deteriorated, and thus ensuring the inference performance of the AI ​​unit related to the AI ​​service on the first device side.

[1095] The information transmission device provided in this application embodiment can implement the various processes implemented in the method embodiments of Figures 3 to 9c and achieve the same technical effect. To avoid repetition, it will not be described again here.

[1096] As shown in Figure 12, this application embodiment also provides a communication device 1200, including a processor 1201 and a memory 1202. The memory 1202 stores a program or instructions that can run on the processor 1201. For example, when the communication device 1200 is a terminal, the program or instructions executed by the processor 1201 implement the various steps of the above-described information transmission method embodiment and achieve the same technical effect. When the communication device m00 is a network-side device, the program or instructions executed by the processor 1201 implement the various steps of the above-described information transmission method embodiment and achieve the same technical effect. To avoid repetition, this will not be described again here.

[1097] This application also provides a terminal, including a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps in the method embodiment shown in FIG3. This terminal embodiment corresponds to the above-described terminal-side method embodiment, and all implementation processes and methods of the above-described method embodiments can be applied to this terminal embodiment and can achieve the same technical effect. The terminal may be the information transmission device shown in FIG10. Specifically, FIG13 is a schematic diagram of the hardware structure of a terminal implementing an embodiment of this application.

[1098] The terminal 1300 includes, but is not limited to, at least some of the following components: radio frequency unit 1301, network module 1302, audio output unit 1303, input unit 1304, sensor 1305, display unit 1306, user input unit 1307, interface unit 1308, memory 1309, and processor 1310.

[1099] Those skilled in the art will understand that terminal 1300 may also include a power supply (such as a battery) for powering various components. The power supply can be logically connected to processor 1310 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The terminal structure shown in Figure 13 does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[1100] It should be understood that, in this embodiment, the input unit 1304 may include a graphics processor 13041 and a microphone 13042. The graphics processor 13041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1306 may include a display panel 13061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1307 includes a touch panel 13071 and at least one of other input devices 13072. The touch panel 13071 is also called a touch screen. The touch panel 13071 may include a touch detection device and a touch controller. Other input devices 13072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[1101] In this embodiment, after receiving downlink data from the network-side device, the radio frequency unit 1301 can transmit it to the processor 1310 for processing; in addition, the radio frequency unit 1301 can send uplink data to the network-side device. Typically, the radio frequency unit 1301 includes, but is not limited to, antennas, amplifiers, transceivers, couplers, low-noise amplifiers, duplexers, etc.

[1102] The memory 1309 can be used to store software programs or instructions, as well as various data. The memory 1309 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1309 may include volatile memory or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1309 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[1103] Processor 1310 may include one or more processing units; optionally, processor 1310 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1310.

[1104] The radio frequency unit 1301 is used to send first information to the second device, the first information being used to request performance monitoring of the artificial intelligence (AI) service; and to obtain second information sent by the second device, the second information including information related to the performance monitoring result of the AI ​​service or information related to the performance monitoring truth value of the AI ​​service.

[1105] In this embodiment, a first device sends first information to a second device, the first information being a request for performance monitoring of an artificial intelligence (AI) service. The first device then receives second information sent by the second device, the second information including information related to the performance monitoring results of the AI ​​service or information related to the performance monitoring truth values ​​of the AI ​​service. The first device includes a terminal, an application function (AF) server, or an OTT server; the second device includes a network device. In this solution, the network provides the first device with information related to the performance monitoring results of the AI ​​service or information related to the performance monitoring truth values. This allows the first device to perform AI unit inference on the AI ​​service based on the information provided by the network, without needing to collect truth values ​​again. This effectively avoids the problem of low inference accuracy of the AI ​​unit due to incomplete truth values ​​collected by the first device, thereby ensuring the inference performance of the AI ​​units related to the AI ​​service on the first device side.

[1106] It is understood that the implementation process of each implementation method mentioned in this embodiment can refer to the relevant description of the method embodiment and achieve the same or corresponding technical effect. To avoid repetition, it will not be described again here.

[1107] This application also provides a network-side device, including a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps of the method embodiment shown in FIG4. This network-side device embodiment corresponds to the above-described network-side device method embodiment. All implementation processes and methods of the above-described method embodiments can be applied to this network-side device embodiment and can achieve the same technical effect.

[1108] Specifically, this application embodiment also provides a network-side device, which can be the information transmission device shown in FIG11. As shown in FIG14, the network-side device 1400 includes: an antenna 141, a radio frequency device 142, a baseband device 143, a processor 144, and a memory 145. The antenna 141 is connected to the radio frequency device 142. In the uplink direction, the radio frequency device 142 receives information through the antenna 141 and sends the received information to the baseband device 143 for processing. In the downlink direction, the baseband device 143 processes the information to be transmitted and sends it to the radio frequency device 142. The radio frequency device 142 processes the received information and transmits it through the antenna 141.

[1109] The method executed by the network-side device in the above embodiments can be implemented in the baseband device 143, which includes a baseband processor.

[1110] The baseband device 143 may include at least one baseband board, on which multiple chips are disposed, as shown in FIG14. One of the chips is, for example, a baseband processor, which is connected to the memory 145 via a bus interface to call the program in the memory 145 and execute the network device operation shown in the above method embodiment.

[1111] The network-side device may also include a network interface 146, such as a Common Public Radio Interface (CPRI).

[1112] Specifically, the network-side device 1400 in this application embodiment further includes: instructions or programs stored in memory 145 and executable on processor 144. The processor 144 calls the instructions or programs in memory 145 to execute the methods executed by each module shown in FIG11 and achieve the same technical effect. To avoid repetition, it will not be described in detail here.

[1113] Specifically, this application also provides a network-side device. As shown in FIG15, the network-side device 1500 includes a processor 1501, a network interface 1502, and a memory 1503. The network-side device may be the information transmission device shown in FIG11. The network interface 1502 is, for example, a common public radio interface (CPRI).

[1114] Specifically, the network-side device 1500 in this application embodiment further includes: instructions or programs stored in memory 1503 and executable on processor 1501. Processor 1501 calls the instructions or programs in memory 1503 to execute the methods executed by each module shown in FIG11 and achieve the same technical effect. To avoid repetition, it will not be described in detail here.

[1115] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described information transmission method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[1116] The processor mentioned above is the processor in the terminal described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk. In some examples, the readable storage medium may be a non-transient readable storage medium.

[1117] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described information transmission method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[1118] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[1119] This application also provides a computer program / program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described information transmission method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[1120] This application also provides an information transmission system, including: a terminal and a network-side device, wherein the terminal can be used to execute the steps of the information transmission method executed by the first device as described above, and the network-side device can be used to execute the steps of the information transmission method executed by the second device as described above.

[1121] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[1122] From the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of computer software products plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes several instructions to cause the terminal or network-side device to execute the methods described in the various embodiments of this application.

[1123] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other implementations under the guidance of this application without departing from the spirit and scope of the claims. All of these implementations are within the protection scope of this application.

Claims

An information transmission method, the method comprising: The first device sends a first message to the second device, the first message being used to request performance monitoring of the artificial intelligence (AI) service. The first device acquires the second information sent by the second device, the second information including information related to the performance monitoring results of the AI ​​service or information related to the true value of the performance monitoring of the AI ​​service; The first device includes a terminal, an application function (AF) or an OTT server; the second device includes network equipment. The method of claim 1, wherein, After the first device sends the first information to the second device, the method further includes: The first device sends first AI service data to the second device, the first AI service data including the model inference results of the first AI unit; The second information is obtained based on the model reasoning results of the first AI unit. The method of claim 1, wherein, The first information is also used to request the network side to perform model inference for the second AI unit; or, the method further includes: The first device sends a third message to the second device, the third message being used to request the network side to perform model inference for the second AI unit; The second AI unit is an AI unit paired with the first AI unit, and the first AI unit is the AI ​​unit used by the first device for model inference. The method according to claim 3, wherein After the first device sends the first information or the third information to the second device, the method further includes: The first device sends first AI service data to the second device, the first AI service data including the model inference results of the first AI unit. The method according to claim 3, wherein, After the first device sends the first information or the third information to the second device, the method further includes: The first device acquires the model inference results of the second AI unit sent by the second device; The first device performs model reasoning of the first AI unit based on the model reasoning result of the second AI unit, and obtains the model reasoning result of the first AI unit; The first device sends second AI service data to the second device. The second AI service data includes at least one of the following: the model inference result of the first AI unit; and a third AI unit, which is an AI unit related to the first AI unit. The method according to any one of claims 3 to 5, wherein When the first device sends the third information to the second device, the first information includes: The first identifier of the model inference task associated with the performance monitoring task of the first information request. The method of claim 1, wherein, The first information is also used to request the network side to perform model inference for the second AI unit; Before the first device sends the first information to the second device, the method further includes: The first device sends a fourth message to the second device, the fourth message being used to request the network side to perform model inference for the second AI unit; The first information includes task update information, which is used to replace the model inference task requested by the fourth information with the model inference and performance monitoring task requested by the first information. The method according to any one of claims 3 to 7, wherein The first information includes at least one of the following: The second AI unit; Description information related to the second AI unit; The data characteristic information of the second AI unit. The method according to any one of claims 1 to 8, wherein The performance monitoring results related to the AI ​​service include the performance monitoring results of the first AI unit, or the performance monitoring truth value related to the AI ​​service includes at least one of the performance monitoring truth value associated with the first AI unit and the performance monitoring truth value associated with the first AI function. Wherein, the first AI unit is the AI ​​unit of the first device that performs model inference, and the first AI function is the AI ​​function corresponding to the first AI unit. The method according to claim 2 or 5, wherein Before the first device receives the second information sent by the second device, the method further includes: The first device sends a third AI unit to the second device, wherein the third AI unit is an AI unit related to the first AI unit; The performance monitoring results related to the AI ​​service include the performance monitoring results of the third AI unit and the performance monitoring results associated with the second AI function; or, the performance monitoring truth value related to the AI ​​service includes the performance monitoring truth value associated with the third AI unit. The second AI function is the AI ​​function corresponding to the third AI unit. The method of claim 10, wherein, Also includes: The first device obtains the performance monitoring result of the first AI unit based on the performance monitoring result of the third AI unit or the performance monitoring true value associated with the third AI unit. The method according to claim 9 or 10, wherein The performance monitoring results include at least one of the following: Performance monitoring KPIs; Monitoring events; Timestamp. The method according to any one of claims 1 to 12, wherein The first information includes at least one of the following: Monitoring type; Monitoring and feedback methods; Monitoring cycle; Monitoring indicators; Monitoring events; Parameters related to monitoring events. The method according to any one of claims 1 to 13, wherein, The first information is information transmitted between the first network function and / or the second network function in the first device and the second device; And / or, The second information is information transmitted between the second network function in the first device and the second device; And / or, The third information is the information transmitted between the first device and the first network function; And / or, The fourth information is the information transmitted between the first device and the first network function; The first network function is used to convert external network requirements into internal requirements. The second network function is used for at least one of business processing and the implementation of business policy rules; The third and fourth information are used to request the network side to perform model inference for the second AI unit. The method of claim 14, wherein, The first device sends first information to the second device, including: The first device sends a first message to the first network function in the second device. The first message is used to request resource allocation for AI services. The first message includes the first information, and the first information also includes at least one of the following: data characteristic information of the first AI unit and AI function corresponding to the first AI unit. The method further includes: The second message sent by the first network function in the second device is obtained. The second message is a response message to the first message. The second message is carried by control plane signaling and includes at least one of the following: communication resource node information, AI resource node information, data characteristic information of the first AI unit, and a first indication; wherein, the first indication is used to indicate whether the data characteristic information of the first AI unit matches the data characteristic information that the network can collect. An information transmission method, the method comprising: The second device obtains the first information sent by the first device, which is used to request performance monitoring of the artificial intelligence (AI) service. The second device sends second information to the first device, the second information including information related to the performance monitoring results of the AI ​​service or information related to the true value of the performance monitoring of the AI ​​service; The first device includes a terminal, an application function (AF) or an OTT server; the second device includes network equipment. The method of claim 16, wherein, Before the second device sends the second information to the first device, the method further includes: The second device acquires the first AI service data sent by the first device, the first AI service data including the model inference results of the first AI unit; The second information is obtained based on the model reasoning results of the first AI unit. The method of claim 16, wherein, The first information is also used to request the network side to perform model inference for the second AI unit; Alternatively, the method may further include: The second device obtains the third information sent by the first device, which is used to request the network side to perform model inference for the second AI unit; The second AI unit is an AI unit paired with the first AI unit, and the first AI unit is the AI ​​unit used by the first device for model inference. The method according to claim 18, wherein, After the second device receives the first information or the third information sent by the first device, the method further includes: The second device acquires the first AI service data sent by the first device, the first AI service data including the model inference result of the first AI unit; the second device performs model inference of the second AI unit based on the model inference result of the first AI unit to obtain the model inference result of the second AI unit. The method of claim 18, wherein, After the second device receives the first information or the third information sent by the first device, the method further includes: The second device sends the model inference results of the second AI unit to the first device; The second device acquires second AI service data sent by the first device. The second AI service data includes at least one of the following: the model inference result of the first AI unit; and a third AI unit, wherein the third AI unit is an AI unit related to the first AI unit. The second device obtains the second information based on the second AI service data. The method according to any one of claims 18 to 20, wherein The first information includes a first identifier of the model inference task associated with the performance monitoring task requested by the first information, and the third information received by the second device from the first device includes: Based on the first identifier, the third information is obtained, wherein the third information includes the first identifier of the model reasoning task. The method of claim 16, wherein, The first information is also used to request the second device to perform model inference for the second AI unit; Before the second device receives the first information sent by the first device, the method further includes: The second device obtains the fourth information sent by the first device, which is used to request the network side to perform model inference for the second AI unit. The first information includes task update information, which is used to replace the model inference task requested by the fourth information with the model inference and performance monitoring task requested by the first information. The method according to any one of claims 18 to 22, wherein The first information includes at least one of the following: The second AI unit; Description information related to the second AI unit; The data characteristic information of the second AI unit. The method according to any one of claims 16 to 23, wherein The performance monitoring results related to the AI ​​service include the performance monitoring results of the first AI unit, or the performance monitoring truth value related to the AI ​​service includes at least one of the performance monitoring truth value associated with the first AI unit and the performance monitoring truth value associated with the first AI function. Wherein, the first AI unit is the AI ​​unit of the first device that performs model inference, and the first AI function is the AI ​​function corresponding to the first AI unit. The method of claim 17 or 20, wherein, Before the second device sends the second information to the first device, the method further includes: The second device acquires the third AI unit sent by the first device, wherein the third AI unit is an AI unit related to the first AI unit; The performance monitoring results related to the AI ​​service include the performance monitoring results of the third AI unit and the performance monitoring results associated with the second AI function; or, the performance monitoring truth value related to the AI ​​service includes the performance monitoring truth value associated with the third AI unit. The second AI function is the AI ​​function corresponding to the third AI unit. The method of claim 24 or 25, wherein, The performance monitoring results include at least one of the following: Performance monitoring KPIs; Monitoring events; Timestamp. The method of any one of claims 16 to 26, wherein The first information includes at least one of the following: Monitoring type; Monitoring and feedback methods; Monitoring cycle; Monitoring indicators; Monitoring events; Parameters related to monitoring events. The method according to any one of claims 16 to 27, wherein, The first information is information transmitted between the first network function and / or the second network function in the first device and the second device; And / or, The second information is information transmitted between the second network function in the first device and the second device; And / or, The third information is the information transmitted between the first device and the first network function; And / or, The fourth information is the information transmitted between the first device and the first network function; The first network function is used to convert external network requirements into internal requirements. The second network function is used for at least one of business processing and the implementation of business policy rules; The third and fourth information are used to request the network side to perform model inference for the second AI unit. The method of claim 28, wherein, The second device obtains the first information sent by the first device, including: The first network function in the second device acquires the first message sent by the first device, the first message being used for requesting resource allocation for an AI service, and the first message including the first information, the first information further including at least one of the following: data characteristic information of the first AI unit and an AI function corresponding to the first AI unit; The method further includes: The first network function in the second device sends a second message to the first device, the second message being a response message of the first message, the second message being carried through control plane signaling, and the second message including at least one of the following: communication resource node information, AI resource node information, data characteristic information of the first AI unit, and a first indication; wherein the first indication is used to indicate whether the data characteristic information of the first AI unit matches data characteristic information that can be collected by the network. The method of claim 29, wherein, Before the first network function in the second device sends the second message to the first device, the method further includes: The first network function sends a third message to a third network function, the third message being used for resource request of an AI service, and the third message including at least one of the following: data source information of the first AI unit, data characteristic information of the first AI unit, related monitoring request information of the first AI unit, data characteristic information associated with the data source information of the first AI unit, data characteristic information of a second AI unit, data characteristic information associated with data source information of the second AI unit, and a first identifier; wherein the first identifier is an identifier of a model inference task associated with a performance monitoring task requested by the first information; The first network function acquires a fourth message sent by the third network function, the fourth message being a response message of the third message, and the fourth message including AI resource node information configured according to the third message; The third network function is configured to manage service resources. The method of claim 29 or 30, wherein, Before the first network function in the second device sends the second message to the first device, the method further includes: The first network function sends a fifth message to a fourth network function, the fifth message being used for requesting acquisition of model-related information, and the fifth message including at least one of the following: an AI function corresponding to the first AI unit, data characteristic information of the first AI unit, related monitoring request information of the first AI unit, the second AI unit, a function corresponding to the second AI unit, data characteristic information of the second AI unit, description information related to the second AI unit, and the first identifier; wherein the first identifier is an identifier of a model inference task associated with a performance monitoring task requested by the first information; The first network function acquires a sixth message sent by the fourth network function, the sixth message including model-related information, or including model-related information and the first indication; The fourth network function is configured to manage AI models. The method of claim 30 or 31, wherein, The method further includes: The first network function sends a seventh message to a fifth network function, the seventh message being used to obtain data source information of the first AI unit, the seventh message comprising at least one of the following: data characteristic information of the first AI unit, the first identifier, historical data source information associated with the data characteristic information of the first AI unit; The second network function obtains an eighth message sent by the fifth network function, the eighth message comprising the data source information of the first AI unit; The fifth network function is configured to perform data management. An information transmission apparatus comprises: A first sending module configured to send first information to a second device, the first information being used to request performance monitoring of an artificial intelligence (AI) service; A first receiving module configured to obtain second information sent by the second device, the second information comprising performance monitoring result related information of the AI service or performance monitoring truth value related information of the AI service; The second device comprises a network device. The apparatus of claim 33, wherein Further comprising: A second sending module configured to send first AI service data to the second device, the first AI service data comprising model inference result of a first AI unit; The second information is obtained according to the model inference result of the first AI unit. The apparatus of claim 33, wherein The first information is also used to request the network side to perform model inference of a second AI unit; or the apparatus further comprises: A third sending module configured to send third information to the second device, the third information being used to request the network side to perform model inference of a second AI unit; The second AI unit is an AI unit paired with the first AI unit, and the first AI unit is an AI unit used by a first device to perform model inference. An information transmission apparatus comprises: A second receiving module configured to obtain first information sent by a first device, the first information being used to request performance monitoring of an artificial intelligence (AI) service; A fourth sending module configured to send second information to the first device, the second information comprising performance monitoring result related information of the AI service or performance monitoring truth value related information of the AI service; The first device comprises a terminal, an application function (AF) or an over-the-top (OTT) server. The apparatus of claim 36, wherein Further comprising: A third receiving module configured to obtain first AI service data sent by the first device, the first AI service data comprising model inference result of a first AI unit; The second information is obtained according to the model inference result of the first AI unit. The apparatus of claim 36, wherein The first information is also used to request the network side to perform model inference of a second AI unit; Or the apparatus further comprises: A fourth receiving module configured to obtain third information sent by the first device, the third information being used to request the network side to perform model inference of a second AI unit; The second AI unit is an AI unit paired with the first AI unit, and the first AI unit is an AI unit used by the first device to perform model inference. A communication device comprising a processor and a memory, said memory storing a program or instructions executable on said processor, said program or instructions, when executed by said processor, implementing the steps of the information transmission method according to any one of claims 1 to 15, or implementing the steps of the information transmission method according to any one of claims 16 to 32. A readable storage medium, on which a program or instructions are stored, said program or instructions, when executed by a processor, implementing the steps of the information transmission method according to any one of claims 1 to 15, or implementing the steps of the information transmission method according to any one of claims 16 to 32. A computer program product comprising computer instructions, said computer instructions, when executed by a processor, implementing the steps of the information transmission method according to any one of claims 1 to 15, or implementing the steps of the information transmission method according to any one of claims 16 to 32.

Images